A real-time obstacle avoidance unmanned aerial vehicle path planning method

Abstract

The application discloses a kind of unmanned aerial vehicle flight path planning methods of real-time obstacle avoidance, comprising the following steps: using airborne multi-sensor to detect environment, the position, speed and direction of real-time capture environmental obstacle;Using convolutional neural network and recurrent neural network to process the data collected by sensor, predict the future motion trajectory of dynamic obstacle;Establish environment model, the risk of the environment around unmanned aerial vehicle is quantitatively evaluated, and the model is dynamically updated when new obstacle is detected;Using deep reinforcement learning based on generating preliminary flight trajectory under global perspective;According to the real-time environmental information provided by obstacle detection module, re-plan trajectory to avoid dynamic obstacle;Unmanned aerial vehicle group makes collaborative decision collectively according to the experience sharing of multi-agent and maximum reciprocity reward mechanism, ensure the safety of the whole group.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of unmanned aerial vehicles, and particularly relates to a real-time obstacle avoidance unmanned aerial vehicle path planning method. BACKGROUND

[0002] In the path planning of unmanned aerial vehicle clusters under energy constraints, obstacles in complex environments, including static and dynamic obstacles, pose a major challenge to the safe flight of unmanned aerial vehicles. The unmanned aerial vehicle group needs to face the changing environment and obstacles during task execution, while maintaining efficient energy use and task target realization. In particular, in multi-task scenarios, the unmanned aerial vehicle not only needs to maintain the accuracy and robustness of flight, but also needs to perform real-time obstacle detection and avoidance.

[0003] To achieve this goal, the technical solution of the path planning method needs to have autonomy and collaboration. Unmanned aerial vehicles can perceive the surrounding environment in real time by using on-board multi-sensor (such as laser radar, camera, etc.), identify static obstacles (such as buildings, terrain, etc.) and dynamic obstacles (such as moving vehicles, other unmanned aerial vehicles, etc.). This environmental perception information must be quickly and accurately integrated and combined with advanced path planning algorithms for dynamic obstacle avoidance.

[0004] The technical solution of obstacle avoidance path planning not only considers the obstacle avoidance needs of a single unmanned aerial vehicle, but also focuses on the collaborative cooperation of unmanned aerial vehicle clusters under energy constraints to maximize global efficiency. This collaborative obstacle avoidance and energy-aware planning strategy can significantly improve the safety and execution efficiency of unmanned aerial vehicle groups in complex and dynamic environments, thereby ensuring stable flight and effective energy management in diverse tasks.

[0005] Therefore, it is a very practical work to study the real-time obstacle avoidance unmanned aerial vehicle path planning method to ensure the safety of the unmanned aerial vehicle group when facing complex and changing environments. SUMMARY

[0006] Therefore, the goal of the present application is to propose a real-time obstacle avoidance unmanned aerial vehicle path planning method, comprising the following steps:

[0007] Step 1, use on-board multi-sensor to detect the environment, and capture the position, speed and direction of obstacles in the environment in real time;

[0008] Step 2, use convolutional neural networks and recurrent neural networks to process the data collected by the sensors to predict the future motion trajectory of dynamic obstacles;

[0009] Step 3, establish an environmental model to quantitatively evaluate the risk of the environment around the unmanned aerial vehicle, and dynamically update the model when new obstacles are detected;

[0010] Step 4: Generate a preliminary flight trajectory from a global perspective using deep reinforcement learning.

[0011] Step 5: Replan the trajectory to avoid dynamic obstacles based on the real-time environmental information provided by the obstacle detection module;

[0012] In step 6, the drone group makes collaborative decisions collectively based on the multi-agent experience sharing and maximum reciprocal reward mechanism to ensure the safety of the entire group.

[0013] Specifically, the method of using a convolutional neural network and a recurrent neural network to process data collected by sensors and predict the future motion trajectory of a dynamic obstacle includes the following steps:

[0014] Acquire environmental data from multiple sensors, including image data from cameras and point cloud data from lidar;

[0015] The normalized image data and point cloud data are input into the convolutional neural network to extract spatial features;

[0016] The spatial features extracted from the convolutional neural network are input into the recurrent neural network for time series modeling to learn the movement pattern of obstacles;

[0017] Through recursive operations over multiple time steps, the recursive neural network predicts the motion trajectory of dynamic obstacles.

[0018] Specifically, the method of generating a preliminary flight trajectory from a global perspective based on deep reinforcement learning includes the following steps:

[0019] State definition, the state of each drone includes its position, speed, remaining power, and information about surrounding obstacles. Let the state of the i-th drone be s i , whose state vector includes: s i =[x i ,y i ,z i ,v i ,E i ,O i ], where: (x i ,y i ,z i ) represents the position coordinates of the drone, v i Indicates the speed of the drone, E i Represents the remaining energy of the drone, O i Represents the information of surrounding obstacles, including distance, direction, and speed. In order to achieve collaboration between multiple agents, the global state S is used to represent the state of all drones and the environment: S = [s1, s2, ..., s N ], where N is the number of drones;

[0020] Action definition, the action of each drone includes its next displacement or speed adjustment. Let the action of the i-th drone be a i , its action vector is: a i =[Δx i ,Δy i ,Δz i ,Δv i ], where Δx i ,Δy i ,Δz i Represents the displacement increment in three dimensions, Δv i represents the velocity increment, and the global action A represents the joint action of all drones, A=[a1,a2,…,a N ];

[0021] Reward function definition: The reward function is used to measure the quality of the trajectory and encourage energy-efficient and obstacle-avoiding trajectory planning. Let the reward function of the i-th drone be r i , which is defined as follows: i =-αd i -βE i +γC i -δR i , where d i Indicates the distance between the drone and the target point. The shorter the distance, the higher the reward. E i Represents energy consumption. The lower the energy consumption, the higher the reward. C i It represents the cooperation utility with other drones. The higher the utility, the higher the reward. R i Represents the risk distance to the obstacle. The closer to the obstacle, the greater the penalty. α, β, γ, δ: coefficients for weighing various rewards;

[0022] During the training phase, the actions of each agent are controlled by a decentralized actor network, while the critic network is centralized and can obtain the global state S and global action A. The parameters of each actor network are determined by the policy μ i (s i |θ i ) indicates that the critic network uses Q(S,A|φ) to evaluate the overall reward. Q(S,A|φ) is the critic network used to evaluate the value of the global action;

[0023] The actor network uses the data in the experience replay buffer to update the policy parameters θ of the i-th drone. i , Represents the experience replay buffer, which stores past state-action-reward-next state samples; the critic network updates the critic network parameters φ using the Bellman equation;

[0024] During execution, each UAV only uses its own local observation s i and the trained actor network to make action decisions and achieve autonomous flight.

[0025] Preferably, before trajectory planning, the UAVs with more energy are assigned to tasks that require higher energy, thereby balancing energy consumption. In the deployment phase, the initial energy E of each UAV is evaluated. i , assign tasks to them to ensure the optimal overall energy utilization. In order to plan energy-efficient paths, the critic network is used to evaluate the energy consumption and reward of each path and select the path that maximizes the global reward. The path optimization function is defined as:

[0026]

[0027] in, The optimal action is to minimize energy consumption and maximize coordination with other drones. Based on the evaluation results of the critic network, the flight action with the lowest energy consumption and that can meet the mission requirements is selected, and the position of the drone is updated at each time step:

[0028] s i,t+1 =s i,t +a i

[0029] Among them, s i,t+1 is the state of the i-th UAV at time step t+1, a i The currently selected action.

[0030] Specifically, the replanning of the trajectory to avoid dynamic obstacles based on the real-time environmental information provided by the obstacle detection module includes the following steps:

[0031] Obtain surrounding environment information through the obstacle detection module, including the real-time position, speed, and direction of obstacles;

[0032] Define the risk function R of each obstacle relative to the drone ij , used to quantify the threat level of obstacles to drones: Among them, d ij is the distance between UAV j and obstacle i, v ij is the relative speed of the obstacle to the UAV. The larger the value of the risk function, the higher the threat level of the obstacle.

[0033] For highly dependent drones, each other's actions are considered when making obstacle avoidance decisions to avoid conflicts. A policy function is used to determine the optimal action a of drone i. i : Among them, C(s i ,a) is the energy consumption cost, λ is the adjustment coefficient, s i represents the environmental state of the i-th UAV, and a represents the optional action of the i-th UAV, which is used to balance the relationship between obstacle avoidance risk and energy consumption;

[0034] Plan the optimal path for a drone to avoid obstacles using a fast-exploring random tree algorithm or a neural network-based approach.

[0035] Furthermore, the method of using a fast exploration random tree algorithm or a neural network-based method to plan the optimal path for a drone to avoid obstacles includes the following steps:

[0036] Initialize the fast exploration random tree and set the current position of the drone s start As the root node of the fast exploration random tree, the target position is s goal , using the information provided by the obstacle detection module to define the obstacle area And set it as the unreachable area in path planning;

[0037] Fast exploration of random tree path expansion: randomly sample a new location point s in the planning space rand , find the distance s rand The nearest tree node s near , and along the s near To s rand Direction expansion, get new node s new : Where ∈ is the expansion step size.

[0038] Obstacle avoidance judgment: judge the newly expanded node s new Is it in an obstacle area? If Then add the node to the fast exploration random tree;

[0039] Repeat the above expansion process until the path is extended from the root node to the target location s goal ;

[0040] Obstacle trajectory predicted using neural network Future collision checking of paths generated by a fast-exploring random tree algorithm;

[0041] For each path point s j , calculate the path point s j With obstacles Distance at time step t+1

[0042] If the distance is less than the safety distance threshold d safe , it is considered that there is a collision risk, and the path needs to be replanned;

[0043] For the path segment that may collide, the rapid exploration random tree algorithm is used to resample and expand from the current node to find a new path to avoid obstacles;

[0044] The generated path is optimized using path smoothing techniques to reduce energy consumption and improve the feasibility of the path;

[0045] The optimized path P after path smoothing smooth is represented as: Where P i represents the i-th point on the path, and the goal is to minimize the distance between path points to obtain a smoother path, and n represents the number of path points.

[0046] Specifically, the UAV group makes a collaborative decision according to the experience sharing and maximum reciprocity reward mechanism of multiple agents, including the following steps:

[0047] Each UAV stores its observation, action, reward, and state change information as experience during task execution, and the experience of the i-th UAV at time step t is:

[0048]

[0049] Where s represents the state at time step t, a represents the action performed, r represents the immediate reward, s' represents the new state after performing the action;

[0050] All UAVs store their experiences in a shared experience pool The shared experience pool is used for learning and optimization when making decisions;

[0051] When a dynamic obstacle is detected, a batch B of experiences is randomly sampled from the shared experience pool for learning, which is used to ensure the diversity and robustness of the learning process;

[0052] Define the reciprocity reward R rec , which measures the contribution of each UAV to other UAVs:

[0053]

[0054] Where is the point mutual information, which is used to quantify the degree of information sharing between UAV i and UAV j. is the contribution or impact of drone i’s action on drone j;

[0055] The total reward of each drone includes its own immediate reward and reciprocal rewards

[0056]

[0057] Each drone updates its policy network based on shared experience and maximum reciprocal reward mechanism. Used to determine the best action in: is the state-action value function, which is used to estimate the possible reward after taking action a;

[0058] Trajectory generation and adjustment: Based on the local movements of all drones, a collective trajectory is generated to ensure the safe passage of the entire drone group;

[0059] Path optimization objective,The goal of path optimization is to maximize the total reward of the group while minimizing energy consumption and avoiding obstacles:

[0060]

[0061] Among them, P is the candidate path; is the total energy consumption of the i-th UAV in T time steps, α is the weight coefficient of energy consumption,

[0062] The trajectory P generated by the drone according to the collective decision i Perform flights while maintaining communication with other drones to obtain mutual status information;

[0063] When a new obstacle or environmental change is detected, the drone swarm re-evaluates its current trajectory and updates its actions, adjusting them in real time by again sampling from the shared experience pool and combining it with the current maximum reciprocal reward.

[0064] The beneficial effects of the present invention are as follows: In order to effectively respond to dynamic environmental changes, this technical solution combines multi-agent deep reinforcement learning with methods such as rapid exploration random trees to ensure decentralized collaborative decision-making within the drone cluster. Each drone relies on local information for path planning based on a partial observation Markov decision process, and improves the response speed and decision-making quality of the entire cluster through a shared experience pool. In order to ensure the safety of the drone cluster and efficient use of energy, the solution also combines the maximum reciprocal reward mechanism to evaluate the cooperation effect between drones to achieve global optimal task execution and energy allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 The overall flow chart of a real-time obstacle avoidance UAV trajectory planning method is shown. DETAILED DESCRIPTION

[0066] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely some, rather than all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0067] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0068] like Figure 1 As shown, this embodiment proposes a real-time obstacle avoidance UAV trajectory planning method, including the following steps:

[0069] Step 1: Use multiple onboard sensors to detect the environment and capture the position, speed, and direction of obstacles in the environment in real time.

[0070] Step 2: Use convolutional neural networks and recurrent neural networks to process the data collected by sensors and predict the future movement trajectory of dynamic obstacles;

[0071] Step 3: Build an environmental model to quantitatively assess the risks of the drone’s surroundings and dynamically update the model when new obstacles are detected.

[0072] Step 4: Generate a preliminary flight trajectory from a global perspective using deep reinforcement learning.

[0073] Step 5: Replan the trajectory to avoid dynamic obstacles based on the real-time environmental information provided by the obstacle detection module;

[0074] In step 6, the drone group makes collaborative decisions collectively based on the multi-agent experience sharing and maximum reciprocal reward mechanism to ensure the safety of the entire group.

[0075] This approach uses data collected by multiple sensors (such as lidar, visual sensors, and infrared sensors) to process and predict the future trajectory of dynamic obstacles through a combination of CNNs and RNNs. CNNs extract spatial features from input images or point cloud data, while RNNs capture the temporal characteristics of dynamic obstacles, thereby predicting their future trajectory.

[0076] Specifically, the method of using a convolutional neural network and a recurrent neural network to process data collected by sensors and predict the future motion trajectory of a dynamic obstacle includes the following steps:

[0077] Acquire environmental data from multiple sensors, including image data from cameras and point cloud data from lidar;

[0078] The normalized image data and point cloud data are input into the convolutional neural network to extract spatial features;

[0079] The spatial features extracted from the convolutional neural network are input into the recurrent neural network for time series modeling to learn the movement pattern of obstacles;

[0080] Through recursive operations over multiple time steps, the recursive neural network predicts the motion trajectory of dynamic obstacles.

[0081] By combining CNN and RNN methods, we can extract spatial and temporal features from data collected by multiple sensors, effectively predicting the future motion trajectory of dynamic obstacles. Convolutional neural networks (CNNs) are used for feature extraction, while recurrent neural networks (RNNs), particularly LSTMs or GRUs, are used to capture time series characteristics. This combined approach provides an intelligent obstacle detection and avoidance solution for energy-constrained drones, ensuring safe navigation in complex and dynamic environments.

[0082] Multi-agent trajectory planning is performed from a global perspective, combined with an energy-efficient deployment strategy to generate preliminary flight trajectories. This approach ensures that each drone completes its mission using the optimal path while coordinating with other drones. The steps include defining states, actions, and rewards, training, and applying energy-efficient path planning.

[0083] Specifically, the method of generating a preliminary flight trajectory from a global perspective based on deep reinforcement learning includes the following steps:

[0084] State definition, the state of each drone includes its position, speed, remaining power, and information about surrounding obstacles. Let the state of the i-th drone be s i , whose state vector includes: s i =[x i ,y i ,z i ,v i ,E i ,O i ], where: (x i ,y i ,z i ) represents the position coordinates of the drone, v i Indicates the speed of the drone, Ei Represents the remaining energy of the drone, O i Represents the information of surrounding obstacles, including distance, direction, and speed. In order to achieve collaboration between multiple agents, the global state S is used to represent the state of all drones and the environment: S = [s1, s2, ..., s N ], where N is the number of drones;

[0085] Action definition, the action of each drone includes its next displacement or speed adjustment. Let the action of the i-th drone be a i , its action vector is: a i =[Δx i ,Δy i ,Δz i ,Δv i ], where Δx i ,Δy i ,Δz i Represents the displacement increment in three dimensions, Δv i represents the velocity increment, and the global action A represents the joint action of all drones, A=[a1,a2,…,a N ];

[0086] Reward function definition: The reward function is used to measure the quality of the trajectory and encourage energy-efficient and obstacle-avoiding trajectory planning. Let the reward function of the i-th drone be r i , which is defined as follows: i =-αd i -βE i +γC i -δR i , where d i Indicates the distance between the drone and the target point. The shorter the distance, the higher the reward. E i Represents energy consumption. The lower the energy consumption, the higher the reward. C i It represents the cooperation utility with other drones. The higher the utility, the higher the reward. R i Represents the risk distance to the obstacle. The closer to the obstacle, the greater the penalty. α, β, γ, δ: coefficients for weighing various rewards;

[0087] During the training phase, the actions of each agent are controlled by a decentralized actor network, while the critic network is centralized and can obtain the global state S and global action A. The parameters of each actor network are determined by the policy μ i (s i |θ i ) indicates that the critic network uses Q(S,A|φ) to evaluate the overall reward. Q(S,A|φ) is the critic network used to evaluate the value of the global action;

[0088] The actor network uses the data in the experience replay buffer to update the policy parameters θ of the i-th drone. i , Represents the experience replay buffer, which stores past state-action-reward-next state samples; the critic network updates the critic network parameters φ using the Bellman equation;

[0089] During execution, each UAV only uses its own local observation s i and the trained actor network to make action decisions and achieve autonomous flight.

[0090] By defining global states, actions, and reward functions, this solution enables multi-agent collaboration and path optimization. Through centralized training and decentralized execution, preliminary flight trajectories can be generated from a global perspective. Combined with energy-efficient deployment strategies, this solution ensures the swarm can complete its mission with optimal energy efficiency. In complex dynamic environments, this solution enables coordinated obstacle avoidance and efficient flight for drones through intelligent path planning.

[0091] Preferably, before trajectory planning, the UAVs with more energy are assigned to tasks that require higher energy, thereby balancing energy consumption. In the deployment phase, the initial energy E of each UAV is evaluated. i , assign tasks to them to ensure the optimal overall energy utilization. In order to plan energy-efficient paths, the critic network is used to evaluate the energy consumption and reward of each path and select the path that maximizes the global reward. The path optimization function is defined as:

[0092]

[0093] in, The optimal action is to minimize energy consumption and maximize coordination with other drones. Based on the evaluation results of the critic network, the flight action with the lowest energy consumption and that can meet the mission requirements is selected, and the position of the drone is updated at each time step:

[0094] s i,t+1 =s i,t +a i

[0095] Among them, s i,t+1 is the state of the i-th UAV at time step t+1, a i The currently selected action.

[0096] In energy-constrained drone swarm trajectory planning, dynamic obstacle avoidance is crucial for ensuring safe operation in complex environments. By combining real-time environmental information provided by the obstacle detection module and leveraging a multi-agent mutual information mechanism, the drone swarm can efficiently implement dynamic obstacle avoidance.

[0097] Specifically, the replanning of the trajectory to avoid dynamic obstacles based on the real-time environmental information provided by the obstacle detection module includes the following steps:

[0098] Obtain surrounding environment information through the obstacle detection module, including the real-time position, speed, and direction of obstacles;

[0099] Define the risk function R of each obstacle relative to the drone ij , used to quantify the threat level of obstacles to drones: Among them, d ij is the distance between UAV j and obstacle i, v ij is the relative speed of the obstacle to the UAV. The larger the value of the risk function, the higher the threat level of the obstacle.

[0100] For highly dependent drones, each other's actions are considered when making obstacle avoidance decisions to avoid conflicts. A policy function is used to determine the optimal action a of drone i. i : Among them, C(s i ,a) is the energy consumption cost, λ is the adjustment coefficient, s i represents the environmental state of the i-th UAV, and a represents the optional action of the i-th UAV, which is used to balance the relationship between obstacle avoidance risk and energy consumption;

[0101] Plan the optimal path for a drone to avoid obstacles using a fast-exploring random tree algorithm or a neural network-based approach.

[0102] In energy-constrained UAV swarm trajectory planning, a combination of a rapidly exploring random tree algorithm and a neural network-based trajectory prediction method enables real-time planning for dynamic obstacle avoidance, ensuring the safety of the UAVs. The rapidly exploring random tree algorithm is an efficient path-finding algorithm that can quickly find a path from the starting point to the end point, while the neural network is used to predict the future trajectory of dynamic obstacles, thereby optimizing the obstacle avoidance path.

[0103] Furthermore, the method of using a fast exploration random tree algorithm or a neural network-based method to plan the optimal path for a drone to avoid obstacles includes the following steps:

[0104] Initialize the fast exploration random tree and set the current position of the drone s startAs the root node of the fast exploration random tree, the target position is s goal , using the information provided by the obstacle detection module to define the obstacle area And set it as the unreachable area in path planning;

[0105] Fast exploration of random tree path expansion: randomly sample a new location point s in the planning space rand , find the distance s rand The nearest tree node s near , and along the s near To s rand Direction expansion, get new node s new : Where ∈ is the expansion step size.

[0106] Obstacle avoidance judgment: judge the newly expanded node s new Is it in an obstacle area? If Then add the node to the fast exploration random tree;

[0107] Repeat the above expansion process until the path is extended from the root node to the target location s goal ;

[0108] Obstacle trajectory predicted using neural network Future collision checking of paths generated by a fast-exploring random tree algorithm;

[0109] For each path point s j , calculate the path point s j With obstacles The distance at time step t+l

[0110] If the distance is less than the safety distance threshold d safe , it is considered that there is a collision risk and the path needs to be replanned;

[0111] For path segments that may collide, a fast-exploring random tree algorithm is used to resample and expand from the current node to find a new path to avoid obstacles;

[0112] The generated path is optimized using path smoothing technology to reduce energy consumption and improve path feasibility;

[0113] The optimized path P after path smoothing smooth Expressed as: Among them, P i represents the i-th point on the path. The goal is to minimize the distance between path points to obtain a smoother path. n represents the number of path points.

[0114] The primary purpose of the experience replay mechanism is to enhance drones' learning capabilities in complex environments by storing and reusing past experience. This allows them to draw on this experience to derive countermeasures when faced with new obstacles. This embodiment combines the experience replay mechanism with a maximum reciprocal reward algorithm to reoptimize drone trajectories. Experience replay is used to store and reuse past experience, while the maximum reciprocal reward algorithm ensures cooperation among drone groups. This allows trajectory optimization to not only consider individual objectives but also improve overall performance.

[0115] Specifically, the drone swarm makes collaborative decisions collectively based on multi-agent experience sharing and a maximum reciprocal reward mechanism, including the following steps:

[0116] During the execution of the mission, each drone stores its observations, actions, rewards, and state change information as experience. The experience of the i-th drone at time step t is:

[0117]

[0118] in, represents the state at time step t, Indicates the action to be performed. Indicates immediate reward, Indicates the new state after executing the action;

[0119] All drones store their experience into a shared experience pool A shared experience pool is used for learning and optimization during decision making;

[0120] When a dynamic obstacle is detected, Randomly sample a batch B of experience to learn: Where M is the batch size, which is used to ensure the diversity and robustness of the learning process;

[0121] Define the reciprocal reward R rec , the reciprocal reward measures the contribution of each drone to other drones:

[0122]

[0123] in, is the point mutual information, which is used to quantify the degree of information sharing between UAV i and UAV j. is the contribution or impact of drone i’s action on drone j;

[0124] The total reward of each drone includes its own immediate reward and reciprocal rewards

[0125]

[0126] Each drone updates its policy network based on shared experience and maximum reciprocal reward mechanism. Used to determine the best action in: is the state-action value function, which is used to estimate the possible reward after taking action a;

[0127] Trajectory generation and adjustment, based on the local actions of all drones, a collective trajectory is generated to ensure that the entire drone group can pass safely. Let the trajectory of the i-th drone be P i , which consists of a series of location points:

[0128]

[0129] Where T is the number of planning time steps;

[0130] Path optimization objective,The goal of path optimization is to maximize the total reward of the group while minimizing energy consumption and avoiding obstacles:

[0131]

[0132] Among them, P is the candidate path; is the total energy consumption of the i-th UAV in T time steps, α is the weight coefficient of energy consumption,

[0133] The trajectory P generated by the drone according to the collective decision i Perform flights while maintaining communication with other drones to obtain mutual status information;

[0134] When a new obstacle or environmental change is detected, the drone swarm re-evaluates its current trajectory and updates its actions, adjusting them in real time by again sampling from the shared experience pool and combining it with the current maximum reciprocal reward.

[0135] In energy-constrained drone swarm trajectory planning, when dynamic obstacles are detected, multi-agent experience sharing and a maximum reciprocal reward mechanism are utilized to ensure the swarm can make collective, coordinated decisions. The experience sharing mechanism helps the swarm improve its learning ability in complex environments, while the maximum reciprocal reward mechanism ensures cooperation among the drones, maximizing the overall mission benefit. By maximizing rewards and minimizing energy consumption during trajectory planning, the swarm achieves efficient and safe dynamic obstacle avoidance.

[0136] As used herein, the word "preferred" is intended to serve as an example, instance, or illustration. Any aspect or design described herein as "preferred" is not necessarily to be construed as advantageous over other aspects or designs. Rather, the use of the word "preferred" is intended to present concepts in a concrete manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X employs A or B" is intended to mean any of the naturally inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied in any of the foregoing examples.

[0137] Moreover, although the present disclosure has been shown and described with respect to one or implementation, those skilled in the art will think of equivalent variations and modifications based on reading and understanding of this specification and the accompanying drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the above-mentioned components (such as elements, etc.), the terms used to describe such components are intended to correspond to any component (unless otherwise indicated) that performs the specified function of the component (such as it is functionally equivalent), even if structurally different from the disclosed structure that performs the function in the exemplary implementation of the present disclosure shown herein. In addition, although the specific features of the present disclosure have been disclosed with respect to only one of several implementations, such features can be combined with one or other features of other implementations that can be desired and advantageous for a given or specific application. Moreover, insofar as the terms "including", "having", "containing" or their variations are used in specific embodiments or claims, such terms are intended to be included in a manner similar to the term "comprising".

[0138] The functional units in the embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or multiple or more units may be integrated into a single module. The aforementioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The aforementioned storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc. The aforementioned devices or systems may execute the storage method in the corresponding method embodiment.

[0139] In summary, the above embodiment is one implementation method of the present invention, but the implementation method of the present invention is not limited to the described embodiment. Any other changes, modifications, substitutions, combinations, and simplifications that deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A real-time obstacle avoidance UAV trajectory planning method, characterized in that: The steps include: Step 1: Use multiple onboard sensors to detect the environment and capture the position, speed, and direction of obstacles in the environment in real time. Step 2: Use convolutional neural networks and recurrent neural networks to process the data collected by sensors and predict the future movement trajectory of dynamic obstacles; Step 3: Build an environmental model to quantitatively assess the risks of the drone’s surroundings and dynamically update the model when new obstacles are detected. Step 4: Generate a preliminary flight trajectory from a global perspective using deep reinforcement learning. Step 5: Replan the trajectory to avoid dynamic obstacles based on the real-time environmental information provided by the obstacle detection module; Step 6: The drone swarm makes collaborative decisions based on the multi-agent experience sharing and maximum reciprocity reward mechanism to ensure the safety of the entire swarm. The drone swarm makes collaborative decisions collectively based on the multi-agent experience sharing and maximum reciprocity reward mechanism, including the following steps: During the execution of the mission, each drone stores its observation, action, reward and state change information as experience. UAVs at time step The experience is: in, Represents the time step status, Indicates the action to be performed. Indicates immediate reward, Represents the new state after executing the action; All drones store their experience into a shared experience pool , a shared experience pool is used for learning and optimization during decision making; When a dynamic obstacle is detected, Randomly sample a batch from Learning from the experience of others is used to ensure the diversity and robustness of the learning process; Defining Reciprocal Rewards , the reciprocal reward measures the contribution of each drone to other drones: in, is the point mutual information, which is used to quantify the UAV and drones The degree of information sharing between Because of drones Actions on drones the contribution or impact generated; The total reward of each drone includes its own immediate reward and reciprocal rewards : Each drone updates its policy network based on shared experience and maximum reciprocal reward mechanism. Used to determine the best action : ,in: is the state-action value function, used to estimate the action to be taken the potential returns that may be obtained later; Trajectory generation and adjustment: Based on the local movements of all drones, a collective trajectory is generated to ensure the safe passage of the entire drone group; Path optimization objective,The goal of path optimization is to maximize the total reward of the group while minimizing energy consumption and avoiding obstacles: in, is the candidate path; For the A drone in The total energy consumption in a time step is is the weight coefficient of energy consumption, Drones follow the trajectory generated by collective decision-making Perform flights while maintaining communication with other drones to obtain mutual status information; When a new obstacle or environmental change is detected, the drone swarm re-evaluates its current trajectory and updates its actions, adjusting them in real time by again sampling from the shared experience pool and combining it with the current maximum reciprocal reward.

2. A real-time obstacle avoidance UAV trajectory planning method according to claim 1, characterized in that: The method of using a convolutional neural network and a recurrent neural network to process data collected by sensors and predict the future motion trajectory of a dynamic obstacle includes the following steps: Acquire environmental data from multiple sensors, including image data from cameras and point cloud data from lidar; The normalized image data and point cloud data are input into the convolutional neural network to extract spatial features; The spatial features extracted from the convolutional neural network are input into the recurrent neural network for time series modeling to learn the movement pattern of obstacles; Through recursive operations over multiple time steps, the recursive neural network predicts the motion trajectory of dynamic obstacles.

3. The real-time obstacle avoidance UAV trajectory planning method according to claim 1, characterized in that: The method of generating a preliminary flight trajectory from a global perspective based on deep reinforcement learning includes the following steps: State definition, the state of each drone includes its position, speed, remaining power, and information about surrounding obstacles. The status of the drone is , whose state vector includes: ,in, Indicates the position coordinates of the drone, Indicates the speed of the drone, Indicates the remaining energy of the drone, Represents information about surrounding obstacles, including distance, direction, and speed. In order to achieve collaboration between multiple agents, a global state is used Represents the state of all drones and the environment: ,in, is the number of drones; Action definition, the action of each drone includes its next displacement or speed adjustment. The action of the drone is , its action vector is: ,in represents the displacement increment in three dimensions, Indicates velocity increment, global action represents the joint action of all drones, ; Reward function definition, the reward function is used to measure the quality of the trajectory, encouraging energy-efficient and obstacle-avoiding trajectory planning. The reward function of the drone is , defined as follows: ,in, Indicates the distance between the drone and the target point. The shorter the distance, the higher the reward. Indicates energy consumption. The lower the energy consumption, the higher the reward. It represents the cooperation utility with other drones. The higher the utility, the higher the reward. Indicates the risk distance to the obstacle. The closer to the obstacle, the greater the penalty. The coefficient used to weigh various rewards; During the training phase, the actions of each agent are controlled by a decentralized actor network, while the critic network is centralized and has access to the global state. and global actions , the parameters of each actor network are determined by the strategy Indicates that the critic network uses To evaluate the overall reward, It is a critic network used to evaluate the value of global actions; Among them, the actor network uses the data in the experience replay buffer to update the Strategy parameters of each drone , Represents the experience replay buffer, which stores past state-action-reward-next state samples; the critic network updates the critic network parameters using the Bellman equation ; During execution, each drone uses only its own local observations and the trained actor network to make action decisions and achieve autonomous flight.

4. The real-time obstacle avoidance UAV trajectory planning method according to claim 3 is characterized in that: Before trajectory planning, drones with more energy are assigned to tasks that require more energy, thereby balancing energy consumption. During the deployment phase, the initial energy of each drone is evaluated and tasks are assigned to ensure optimal overall energy utilization. To plan energy-efficient paths, a critic network is used to evaluate the energy consumption and reward of each path and select the path that maximizes the global reward. The path optimization function is defined as: in, The optimal action is to minimize energy consumption and maximize coordination with other drones. Based on the evaluation results of the critic network, the flight action with the lowest energy consumption and that can meet the mission requirements is selected, and the position of the drone is updated at each time step: in, is the i-th drone time step The state of The currently selected action.

5. The real-time obstacle avoidance UAV trajectory planning method according to claim 1, characterized in that: The method of replanning the trajectory to avoid dynamic obstacles based on the real-time environmental information provided by the obstacle detection module includes the following steps: Obtain surrounding environment information through the obstacle detection module, including the real-time position, speed, and direction of obstacles; Define the risk function of each obstacle relative to the drone , used to quantify the threat level of obstacles to drones: ,in, For drones With obstacles distance, is the relative speed of the obstacle to the UAV. The larger the value of the risk function, the higher the threat level of the obstacle. For highly dependent drones, consider each other's actions when making obstacle avoidance decisions to avoid conflicts, and use a policy function to decide the drones The best action : ,in, is the energy consumption cost, is the adjustment coefficient, Indicates the The environmental status of the drone, Indicates the The optional actions of the drone are used to balance the relationship between obstacle avoidance risk and energy consumption; Plan the optimal path for a drone to avoid obstacles using a fast-exploring random tree algorithm or a neural network-based approach.

6. The real-time obstacle avoidance UAV trajectory planning method according to claim 5, characterized in that: The method of using a fast exploration random tree algorithm or a neural network-based method to plan the optimal path for a drone to avoid obstacles includes the following steps: Initialize a fast exploration random tree and set the current position of the drone As the root node of the fast exploration random tree, the target position is , using the information provided by the obstacle detection module to define the obstacle area , and set it as the unreachable area in path planning; Rapid Exploration of Random Tree Path Extensions: Randomly sample a new location in the planning space , find the distance The nearest tree node , and along Towards Direction expansion to obtain new nodes : ,in, is the expansion step length; Obstacle avoidance judgment: judging newly expanded nodes Is it in an obstacle area? If , then add the node to the fast exploration random tree; Repeat the above expansion process until the path is extended from the root node to the target location ; Obstacle trajectory predicted using neural network ,future collision checking of paths generated by the fast exploration random tree algorithm; For each waypoint , calculate the path points With obstacles At time step distance ; If the distance is less than the safety distance threshold , it is considered that there is a collision risk and the path needs to be replanned; For path segments that may collide, a fast-exploring random tree algorithm is used to resample and expand from the current node to find a new path to avoid obstacles; The generated path is optimized using path smoothing technology to reduce energy consumption and improve path feasibility; Optimized path after path smoothing Expressed as: in, Indicates the first points, the goal is to minimize the distance between path points to obtain a smoother path, Indicates the number of path points.

Images