UAV high-speed navigation and obstacle avoidance method, system, terminal and storage medium

Through deep reinforcement learning methods, the drone navigation and obstacle avoidance problems are modeled as Markov decision-making models, and the neural network strategy is optimized by using path progress rewards, security rewards and terminal rewards to solve the high-speed navigation and scrambled environments of drones. The problem of obstacle avoidance has achieved rapid and effective task completion and high generalization capabilities.

CN118466557BActive Publication Date: 2024-11-26BEIJING INST OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410917392.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-10
Publication Date
2024-11-26
Estimated Expiration
2044-07-10

AI Technical Summary

Technical Problem

The existing high-speed navigation and obstacle avoidance methods of drone are difficult to effectively reach a series of prescribed waypoints in a messy environment with minimal time and no collision. The traditional method decouples planning and control, resulting in a large impact on model errors and is robust. Poor sex.

Method used

Using deep reinforcement learning methods, the drone navigation and obstacle avoidance problems are modeled as Markov decision-making models. Through the pre-constructed simulation environment, neural network strategies are trained to obtain the observation information of the drone and calculate the path progress reward and safety reward. and terminal rewards to optimize neural network strategies, avoid potential collision risks and improve navigation efficiency.

Benefits of technology

It realizes the rapid and effective completion of tasks in a messy environment, avoids collision risks, improves the generalization ability of neural network strategies in new scenarios, reduces the coupling error between planning and tracking processes, and improves high-speed navigation and avoidance. Dysfunction function.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118466557B_ABST
    Figure CN118466557B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, system, terminal and storage medium for high-speed navigation and obstacle avoidance of unmanned aerial vehicles, and relates to the field of unmanned aerial vehicle control technology. The neural network strategy in the present invention can output effective execution actions according to the observation information of the unmanned aerial vehicle. The training method is: obtaining the simulated observation information of the unmanned aerial vehicle through a simulation environment with randomly changing complexity; sampling simulation execution actions through a neural network strategy; calculating the simulation state information at the next moment and the timely reward at the current moment through a Markov decision model, including path progress reward, safety reward, and terminal reward; calculating the discount reward based on the timely reward, and optimizing the neural network strategy using the discount reward. The present invention encourages the unmanned aerial vehicle to move quickly along the route, avoid collision risks, and complete the task more quickly and effectively by setting three kinds of rewards. The training adopts a simulation environment with randomly changing complexity, which also effectively improves the generalization ability of the neural network strategy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle control technology, and in particular to a method, system, terminal and storage medium for high-speed navigation and obstacle avoidance of an unmanned aerial vehicle. Background Art

[0002] High-speed navigation and obstacle avoidance in a cluttered environment is a challenging task that pushes the drone to the physical limits of its speed and acceleration. In addition, obstacles in cluttered environments introduce highly non-convex optimization problems that are difficult to solve with traditional methods. Traditional frameworks decouple planning and control, which increases the impact of model errors, leading to suboptimal results or even crashes.

[0003] At present, there are several research directions for high-speed navigation and obstacle avoidance methods for UAVs:

[0004] 1. Polynomials or B-splines are used to represent the trajectory, but polynomial trajectory planning essentially smoothes the trajectory, which limits the maneuverability of the drone and makes it impossible to achieve the optimal high-speed navigation strategy;

[0005] 2. Turn the intermediate waypoints into constraints and shorten the trajectory flight time by optimizing the sampling interval between each trajectory. However, this method will lead to longer optimization time and slower response when there are more obstacles in a cluttered environment. In addition, this method decouples planning and control, resulting in poor robustness to UAV model errors.

[0006] 3. Use topological path planning combined with a tracker based on deep reinforcement learning (DRL) to achieve obstacle avoidance in cluttered environments. However, in cluttered environments, topological path planning usually takes a lot of time, making it difficult to navigate and avoid obstacles in real time. On the other hand, existing obstacle avoidance methods are too simple and it is difficult to achieve 100% obstacle avoidance when there are many obstacles.

[0007] In summary, existing high-speed navigation and obstacle avoidance methods are difficult to effectively solve the problem of drones reaching a series of specified waypoints in a cluttered environment in the shortest time and without collision.

[0008] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0009] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a method, system, terminal and storage medium for high-speed navigation and obstacle avoidance of unmanned aerial vehicles are provided, aiming to solve the problem that the existing high-speed navigation and obstacle avoidance methods are difficult to effectively solve the problem that unmanned aerial vehicles in a cluttered environment can reach a series of specified waypoints in the shortest time and without collision.

[0010] The technical solution adopted by the present invention to solve the problem is as follows:

[0011] In a first aspect, an embodiment of the present invention provides a method for high-speed navigation and obstacle avoidance of a drone, the method comprising:

[0012] Obtain observation information from drones;

[0013] Inputting the observation information into a pre-trained neural network strategy to obtain the execution action of the drone;

[0014] The training method of the neural network strategy includes:

[0015] Model the UAV navigation and obstacle avoidance problem as a Markov decision model in advance;

[0016] Acquire simulated observation information of the UAV through a pre-constructed simulation environment, wherein the complexity of the simulation environment may vary randomly;

[0017] Performing action sampling according to the simulation observation information through the neural network strategy to obtain the simulation execution action of the drone;

[0018] Calculating the simulation state information of the drone at the next moment and the timely reward at the current moment through the Markov decision model based on the simulation execution action and the simulation state information of the drone at the current moment, wherein the timely reward includes a path progress reward, a safety reward, and a terminal reward; and calculating a discount reward based on the timely reward;

[0019] Determine whether the discounted reward reaches the desired goal, and if not, optimize the neural network strategy according to the discounted reward.

[0020] In one embodiment, the observation information includes the drone position, the drone attitude, the drone linear velocity, the drone angular velocity, the relative position of the drone's current position and several target points that the drone is about to pass, and the relative position of several obstacles within the drone's observation range and the nearest obstacles;

[0021] The execution action includes the combined thrust and angular velocity of the drone.

[0022] In one embodiment, the elements of the Markov decision model include:

[0023] The state space reflects the set of states of the drone;

[0024] An action space, reflecting the action set of the drone, is used to sample the actions of the neural network strategy to obtain the simulation execution action;

[0025] A state transfer equation is used to calculate the simulation state information of the drone at the next moment;

[0026] A reward function, used to calculate the instantaneous reward at the current moment;

[0027] Initial state distribution, used to sample the initial state information of the UAV;

[0028] A discount factor, reflecting the importance of future rewards, is used to calculate the discounted rewards.

[0029] In one embodiment, the calculation method of the path progress reward includes:

[0030] Calculate the first distance value by using the coordinates of the target point at the current moment and the coordinates of the drone at the current moment;

[0031] Calculate the second distance value through the coordinates of the target point at the current moment and the coordinates of the drone at the previous moment;

[0032] The path progress reward is calculated using the first distance value, the second distance value, and the angular velocity of the drone at the current moment.

[0033] In one embodiment, the calculation method of the safety reward includes:

[0034] The safety reward is calculated according to the distance value between the drone and the obstacle at the current moment.

[0035] In one embodiment, the method for calculating the terminal reward includes:

[0036] Determine whether the drone has collided. If a collision occurs, generate a collision penalty value;

[0037] Determine whether the drone has completed the flight mission. If the flight mission is completed, generate a mission reward value;

[0038] The terminal reward is calculated according to the collision penalty value and the task reward value.

[0039] In one embodiment, the simulation environment includes several parallel simulation environments, wherein each of the simulation environments runs independently, and the environmental parameters and UAV parameters therein can be randomly sampled; the environmental parameters are used to reflect obstacle information and / or route information, and the UAV parameters are used to reflect the UAV status.

[0040] In a second aspect, an embodiment of the present invention further provides a drone high-speed navigation and obstacle avoidance system, the system comprising:

[0041] Information acquisition module, used to obtain observation information of the UAV;

[0042] An action generation module, used to input the observation information into a pre-trained neural network strategy to obtain an execution action of the drone;

[0043] A strategy training module, which is used to pre-model the UAV navigation and obstacle avoidance problem as a Markov decision model;

[0044] Acquire simulated observation information of the UAV through a pre-constructed simulation environment, wherein the complexity of the simulation environment may vary randomly;

[0045] Performing action sampling according to the simulation observation information through the neural network strategy to obtain the simulation execution action of the drone;

[0046] Calculating the simulation state information of the drone at the next moment and the timely reward at the current moment through the Markov decision model based on the simulation execution action and the simulation state information of the drone at the current moment, wherein the timely reward includes a path progress reward, a safety reward, and a terminal reward; and calculating a discount reward based on the timely reward;

[0047] Determine whether the discounted reward reaches the desired goal, and if not, optimize the neural network strategy according to the discounted reward.

[0048] In a third aspect, an embodiment of the present invention further provides a terminal, comprising a memory and one or more processors; the memory stores one or more programs; the program includes instructions for executing any of the above-described methods for high-speed navigation and obstacle avoidance of a UAV; and the processor is used to execute the program.

[0049] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a plurality of instructions are stored, wherein the instructions are suitable for being loaded and executed by a processor to implement the steps of any of the above-mentioned methods for high-speed navigation and obstacle avoidance of unmanned aerial vehicles.

[0050] Beneficial effects of the present invention: The embodiments of the present invention encourage drones to move quickly along the predetermined route through path progress rewards, safety rewards, and terminal rewards, avoiding potential collision risks, and completing tasks more quickly and efficiently. In addition, the complexity of the simulation environment used in neural network strategy training can change randomly, which effectively improves the generalization ability of neural network strategies in new scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1It is a flow chart of a method for high-speed navigation and obstacle avoidance of a UAV provided by an embodiment of the present invention.

[0053] Figure 2 It is a schematic diagram of the training process of the neural network strategy provided by an embodiment of the present invention.

[0054] Figure 3 This is a diagram showing the effects of Example 1 provided by an embodiment of the present invention.

[0055] Figure 4 This is a diagram showing the effect of Example 2 provided in an embodiment of the present invention.

[0056] Figure 5 It is a module schematic diagram of a UAV high-speed navigation and obstacle avoidance system provided by an embodiment of the present invention.

[0057] Figure 6 It is a principle block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The present invention discloses a method, system, terminal and storage medium for high-speed navigation and obstacle avoidance of a drone. In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0060] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as herein.

[0061] In view of the above-mentioned defects of the prior art, the present invention provides a method for high-speed navigation and obstacle avoidance of a drone, the method comprising: obtaining observation information of the drone; inputting the observation information into a pre-trained neural network strategy to obtain the execution action of the drone; the training method of the neural network strategy comprises: pre-modeling the navigation and obstacle avoidance problem of the drone as a Markov decision model; obtaining the simulated observation information of the drone through a pre-built simulation environment, wherein the complexity of the simulation environment can change randomly; sampling actions according to the simulated observation information through the neural network strategy to obtain the simulated execution action of the drone; calculating the simulated state information of the drone at the next moment and the timely reward at the current moment based on the simulated execution action and the simulated state information of the drone at the current moment through the Markov decision model, wherein the timely reward includes a path progress reward, a safety reward and a terminal reward; calculating a discount reward according to the timely reward; judging whether the discount reward reaches the expected goal, if not, optimizing the neural network strategy according to the discount reward. The present invention encourages drones to move forward quickly according to the predetermined route through path progress rewards, safety rewards and terminal rewards, avoids potential collision risks, and completes tasks more quickly and efficiently. In addition, the complexity of the simulation environment used for neural network strategy training can change randomly, which effectively improves the generalization ability of neural network strategies in new scenarios.

[0062] like Figure 1 As shown, the method includes:

[0063] Step S100: Obtain observation information of the drone.

[0064] Specifically, the observation information of the drone can reflect the current environment information of the drone and its own state information. In order to solve the problem of high-speed navigation and obstacle avoidance of the drone, this embodiment needs to first obtain the observation information of the drone, and then correctly determine what action the drone should perform.

[0065] Step S200: input the observation information into a pre-trained neural network strategy to obtain the execution action of the drone.

[0066] Specifically, in order to quickly obtain the current execution action of the drone, this embodiment pre-builds and trains a neural network strategy. The input data of the neural network strategy is the observation information of the drone, and the output data is the current execution action of the drone. Since the neural network strategy has been pre-trained with big data and has learned the complex mapping relationship between input data and output data, it can quickly and accurately analyze what action the drone should currently perform in order to better complete the flight mission.

[0067] In one implementation, the observation information includes the drone position, drone attitude, drone linear velocity, drone angular velocity, the relative position of the drone's current position and several target points that it is about to pass, and the relative position of the drone's current position and several obstacles within the drone's observation range that are closest to it; the execution action includes the drone's combined thrust and angular velocity.

[0068] like Figure 1 As shown, the training method of the neural network strategy includes:

[0069] Step S201: Model the UAV navigation and obstacle avoidance problem as a Markov decision model in advance.

[0070] Specifically, the drone in this embodiment can be a drone model. The drone navigation and obstacle avoidance problem is modeled as a Markov decision process in advance, that is, a Markov decision model is obtained. The Markov decision process describes the process of interaction between the drone and the environment. In this process, the neural network strategy will make a decision based on the observation information of the current drone, select an action to execute, and the environment will give feedback based on the action. After the drone executes an action, it will cause a state transfer. Therefore, the drone will generate a large number of states, actions and feedback in the process of continuous interaction with the environment. After multiple rounds of interaction, the neural network strategy can make more correct decisions, that is, learn better strategies.

[0071] In one implementation, the elements of the Markov decision model include:

[0072] The state space reflects the set of states of the drone;

[0073] An action space, which reflects the action set of the drone and is used to sample the actions of the neural network strategy to obtain the simulated execution actions;

[0074] A state transfer equation is used to calculate the simulation state information of the drone at the next moment;

[0075] A reward function, used to calculate the instantaneous reward at the current moment;

[0076] Initial state distribution, used to sample the initial state information of the UAV;

[0077] A discount factor, reflecting the importance of future rewards, is used to calculate the discounted rewards.

[0078] Specifically, this embodiment represents the Markov decision process through six elements: state space, action space, state transfer equation, reward, initial state distribution, and discount coefficient, and models the strategy through a neural network. First, the drone samples the initial state from the initial state distribution, and then randomly samples actions from the action space at each time step through the strategy. The drone interacts with the environment with the sampled actions and transfers from the state of the current time step to the state of the next time step through the state transfer equation, and obtains timely rewards through the reward function. The ultimate goal of this embodiment is to obtain the optimal strategy by maximizing the discounted reward.

[0079] For example, the problem of high-speed navigation and obstacle avoidance of UAV in cluttered environment is modeled as a Markov decision process. ,in represents the state space, represents the action space, represents the state transfer equation, For reward, is the initial state distribution, is the discount factor. The strategy is modeled by a deep neural network as The drones are distributed from the initial state Initial state of sampling At each time step , the drone status is And through strategy Randomly sample actions Drones The state transition equation is used for the interaction between the action and the environment Transfer to state And get timely rewards The ultimate goal is to optimize the optimal strategy by maximizing the discounted reward , the formula is as follows:

[0080] .

[0081] Step S202: Acquire simulated observation information of the UAV through a pre-constructed simulation environment, wherein the complexity of the simulation environment may vary randomly.

[0082] Specifically, in order to improve the generalization ability of the neural network strategy in new scenarios, this embodiment uses a simulation environment with randomly changing complexity to perform strategy training.

[0083] Step S203: performing action sampling according to the simulation observation information through the neural network strategy to obtain the simulation execution action of the drone.

[0084] Specifically, the environmental parameters and drone states in the simulation environment can be randomly sampled for strategy training. The training goal is to enable the neural network strategy to learn the mapping relationship between different observation information and execution actions. In order to enable drones to better perform flight missions in complex environments, such as Figure 2 As shown, this embodiment designs a multi-dimensional observation space and action space for the neural network strategy. The observation space includes the position, posture, and speed of the drone, and the drone and future The relative position of the target point, the drone and the observation range The relative position of obstacles. , are all non-negative numbers. The action space includes the combined thrust and angular velocity of the drone. This embodiment comprehensively considers the position information of the target point and obstacles, so that the neural network strategy can plan the path and adjust the flight attitude more accurately, thereby improving the accuracy and precision of navigation, and helping the drone to take appropriate obstacle avoidance actions, thereby improving the safety and stability of the drone in complex environments.

[0085] For example, by training a deep neural network strategy, the drone moment, from observation Directly mapped to actions .in Defined as ,in , , , Respectively represent the position, attitude, linear velocity, and angular velocity of the drone; Indicates the current position of the drone and the area it is about to pass. The relative position of the target points; Indicates the nearest The relative position of obstacles. Action space is defined as , respectively represent the combined velocity and angular velocity of the UAV.

[0086] Step S204, calculating the simulation state information of the drone at the next moment and the timely reward at the current moment through the Markov decision model based on the simulation execution action and the simulation state information of the drone at the current moment, wherein the timely reward includes a path progress reward, a safety reward and a terminal reward; and calculating the discount reward based on the timely reward.

[0087] Specifically, in the Markov decision process, the drone will undergo state transition according to the execution action output by the strategy, and feedback, i.e., reward, will be given to the drone's execution action. The rewards designed for the Markov decision process in this embodiment include path progress rewards, safety rewards, and terminal rewards. Path progress rewards encourage drones to fly along the preset route, safety rewards encourage drones to stay away from obstacles, and terminal reward minimization encourages drones to shorten the task completion time, i.e., minimize the total journey time. These three rewards can better optimize the neural network strategy.

[0088] For example, the reward function set in this embodiment is divided into three parts: path progress reward , Safety Rewards , Terminal Rewards The path progress reward ensures that the drone flies to the designated location in the shortest time by maximizing the progress of the drone along the path; the safety reward ensures that the drone maintains a safe distance from obstacles to avoid collisions; the terminal reward is divided into two parts. The first part sets a collision penalty avoidance strategy to sample dangerous actions again, and the second part further shortens the time it takes for the drone to complete the task by minimizing the total journey time.

[0089] The total reward function is shown in the following formula:

[0090] .

[0091] In one implementation, the calculation method of the path progress reward includes:

[0092] Calculate the first distance value by using the coordinates of the target point at the current moment and the coordinates of the drone at the current moment;

[0093] Calculate the second distance value through the coordinates of the target point at the current moment and the coordinates of the drone at the previous moment;

[0094] The path progress reward is calculated using the first distance value, the second distance value, and the angular velocity of the drone at the current moment.

[0095] Specifically, the calculation formula for path progress rewards is as follows:

[0096] ;

[0097] in , Expressed as The coordinates of the target point and the coordinates of the drone at the moment, is the coefficient.

[0098] In one implementation, the calculation method of the safety reward includes:

[0099] The safety reward is calculated according to the distance value between the drone and the obstacle at the current moment.

[0100] Specifically, the calculation formula for safety rewards is as follows:

[0101] ;

[0102] in Indicated in Moment drone and obstacles The distance between The safety bonus factor is used to adjust the aggressiveness of the flight. The safety bonus can encourage the drone to stay away from obstacles.

[0103] In one implementation, the method for calculating the terminal reward includes:

[0104] Determine whether the drone has collided. If a collision occurs, generate a collision penalty value;

[0105] Determine whether the drone has completed the flight mission. If the flight mission is completed, generate a mission reward value;

[0106] The terminal reward is calculated according to the collision penalty value and the task reward value.

[0107] Specifically, terminal rewards (i.e., collision penalty value) is a very large negative value, which is triggered when a UAV collides, preventing the strategy from generating collision actions again. (i.e. the task reward value), is the preset coefficient, It is the mission completion time, which is triggered when the UAV completes the mission, and encourages the strategy to produce a strategy with a smaller total flight time.

[0108] In one implementation, a method for determining whether a drone collides with an obstacle includes:

[0109] According to each obstacle, determine the radius value of the spherical space corresponding to the obstacle;

[0110] Get the arm length of the drone, and calculate the safety distance value based on the radius value, arm length, and preset safety margin;

[0111] The actual distance between the obstacle and the drone is obtained. If the actual distance is less than the safety distance, it is determined that the drone has collided with the obstacle.

[0112] For example, in the process of interaction between the drone and the environment, this embodiment regards obstacles as multiple objects with a radius of When the distance between the drone and the obstacle Less than a safe distance When the two drones collide, it is considered that the drones have collided. Represents the arm length of the drone, For the safety margin:

[0113] .

[0114] Step S205: determine whether the discount reward reaches the expected goal, and if not, optimize the neural network strategy according to the discount reward.

[0115] Specifically, this embodiment uses the Proximal Policy Optimization (PPO) algorithm to train the neural network strategy. The advantage of the PPO algorithm is that it can handle continuous action space and has good convergence and stability. At each time step, the drone will receive a discounted reward when interacting with the environment, and the strategy is optimized by maximizing the discounted reward. In one implementation, the expected goal can be that the change value of the discounted reward corresponding to several adjacent time steps is less than a preset threshold, or the number of iterative optimization steps reaches a specified number of steps.

[0116] In one implementation, the simulation environment includes several parallelized simulation environments, wherein each of the simulation environments runs independently, and the environmental parameters and UAV parameters therein can be randomly sampled; the environmental parameters are used to reflect obstacle information and / or route information, and the UAV parameters are used to reflect the UAV status.

[0117] Specifically, in order to better train the neural network strategy, the present embodiment pre-establishes multiple parallel simulation environments, which can be in three-dimensional form for simulation and data collection. The multiple simulation environments are independent of each other, and the elements in the environment and the state of the drone are controllable and can be randomly sampled. For example, the elements in the environment may include the location of obstacles, the location of intermediate waypoints, the number of obstacles, and the density of obstacles. The state of the drone may include the initial posture, speed, and position of the drone, and the position of the obstacle may change randomly. This embodiment is based on the Markov decision process, collects data in a parallel environment, and trains the neural network strategy online through a reinforcement learning algorithm. The parameters of the drone and obstacles, as well as the complexity of the environment, are randomized during the training process, which can effectively enhance the robustness and generalization of the neural network strategy.

[0118] For example, this embodiment trains the neural network strategy online by collecting a large amount of data in a parallel environment. Specifically, 100 parallel simulation environments are established, each environment runs independently and each element in the environment is controllable. The starting point, end point and intermediate waypoints can be set at will in the simulation environment; the initial position and posture of the drone, and the position and posture of the obstacles can be set manually. This method can greatly improve the training efficiency and enrich the data diversity. Four levels of simulation environment density are set: Level 1, obstacle spacing greater than 10 meters; Level 2, obstacle spacing 5~10 meters; Level 3, obstacle spacing 3~5 meters; Level 4, obstacle spacing 1~3m.

[0119] In one implementation, after the neural network strategy is trained, the following steps are further included:

[0120] Build a new test environment to test the navigation and obstacle avoidance performance of drones.

[0121] Specifically, after the neural network strategy is trained, this embodiment will also construct one or more new cluttered environments as test environments for testing the navigation and obstacle avoidance performance of the drone. The test environment is constructed in the same way as the simulation environment used for training, and the elements in the environment and the drone state are controllable and can be randomly sampled.

[0122] For example, several new environments are created for testing, during which the initial position, attitude, linear speed and position of intermediate waypoints of the drone are randomized. Figure 3 , Figure 4 The effect diagrams of the present invention on two examples are respectively shown. Figure 3 The flight mission in the invention is to start from a starting point and pass through 7 waypoints in sequence, wherein the position of one waypoint is randomly changed. The present invention tests 50 environments, and the results show that the UAV can quickly pass through each waypoint and generate a collision-free trajectory. Figure 4 The flight mission is to start from a starting point, pass through 9 waypoints in sequence and arrive at the end point, wherein the starting point position, the initial posture and speed of the UAV and the end point position are randomly sampled. The present invention has tested 50 environments, and the results show that the method of the present invention can effectively and quickly fly over the target point without collision in the middle.

[0123] The advantages of the present invention are:

[0124] 1. The present invention iterates the trial-and-error process of the deep reinforcement learning method to generate the optimal time and collision-free strategy, which can achieve a response speed of several milliseconds, avoiding the coupling error problem between the planning process and the tracking process, and can effectively improve the high-speed navigation and obstacle avoidance functions of the UAV in a cluttered environment, and successfully realize the high-speed navigation and obstacle avoidance flight of the UAV in a cluttered environment.

[0125] 2. The present invention constructs a total reward function through route progress rewards, safety rewards and terminal rewards. The route progress rewards encourage the drone to move forward quickly according to the predetermined route; the safety rewards can effectively avoid potential collision risks by associating the distance between the drone and obstacles; the terminal rewards are triggered when the task is completed, and their values ​​decrease as the flight time increases. By punishing excessive flight time, the drone is prompted to complete the task more quickly and effectively. The rewards set by the present invention effectively improve the drone's obstacle avoidance ability in a cluttered environment while ensuring that the drone completes the task in the shortest time.

[0126] 3. The present invention randomizes the environment density during training and effectively improves the generalization ability of the strategy in new scenarios by training the neural network strategy in environments of different complexity.

[0127] 4. The present invention uses the relative positions of the current position of the drone and the multiple target points that it is about to pass, and the relative positions of the multiple obstacles closest to the drone's observation range as part of the strategy observation space. By comprehensively considering the position information of the target points and obstacles, the deep reinforcement learning strategy (DRL strategy) can plan the path and adjust the flight attitude more accurately, thereby improving the accuracy and precision of navigation, and helping the drone to take appropriate obstacle avoidance actions, improving the safety and stability of the drone in complex environments.

[0128] Based on the above embodiments, the present invention also provides a high-speed navigation and obstacle avoidance system for unmanned aerial vehicles, such as Figure 5 As shown, the system comprises:

[0129] Information acquisition module 01, used to obtain observation information of the drone;

[0130] The action generation module 02 is used to input the observation information into a pre-trained neural network strategy to obtain the execution action of the drone;

[0131] Strategy training module 03, used to pre-model the UAV navigation and obstacle avoidance problem as a Markov decision model;

[0132] Acquire simulated observation information of the UAV through a pre-constructed simulation environment, wherein the complexity of the simulation environment may vary randomly;

[0133] Performing action sampling according to the simulation observation information through the neural network strategy to obtain the simulation execution action of the drone;

[0134] Calculating the simulation state information of the drone at the next moment and the timely reward at the current moment through the Markov decision model based on the simulation execution action and the simulation state information of the drone at the current moment, wherein the timely reward includes a path progress reward, a safety reward, and a terminal reward; and calculating a discount reward based on the timely reward;

[0135] Determine whether the discounted reward reaches the desired goal, and if not, optimize the neural network strategy according to the discounted reward.

[0136] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be shown as follows: Figure 6 As shown. The terminal includes a processor, a memory, a network interface, and a display screen connected through a system bus. Among them, the processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for high-speed navigation and obstacle avoidance of a drone is implemented. The display screen of the terminal can be a liquid crystal display screen or an electronic ink display screen.

[0137] Those skilled in the art will understand that Figure 6 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0138] In one implementation, the terminal stores one or more programs in its memory, and is configured to be executed by one or more processors, wherein the one or more programs include instructions for performing a method for high-speed navigation and obstacle avoidance of a drone.

[0139] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0140] In summary, the present invention discloses a method, system, terminal and storage medium for providing high-speed navigation and obstacle avoidance of unmanned aerial vehicles. The method comprises: obtaining observation information of the unmanned aerial vehicle; inputting the observation information into a pre-trained neural network strategy to obtain the execution action of the unmanned aerial vehicle; the training method of the neural network strategy comprises: pre-modeling the navigation and obstacle avoidance problem of the unmanned aerial vehicle as a Markov decision model; obtaining the simulated observation information of the unmanned aerial vehicle through a pre-constructed simulation environment, wherein the complexity of the simulation environment can change randomly; sampling actions according to the simulated observation information through the neural network strategy to obtain the simulated execution action of the unmanned aerial vehicle; calculating the simulation state information of the unmanned aerial vehicle at the next moment and the timely reward at the current moment based on the simulated execution action and the simulation state information of the unmanned aerial vehicle at the current moment through the Markov decision model, wherein the timely reward includes a path progress reward, a safety reward and a terminal reward; calculating a discount reward according to the timely reward; judging whether the discount reward reaches the expected goal, if not, optimizing the neural network strategy according to the discount reward. The present invention encourages unmanned aerial vehicles to move quickly along the predetermined route through path progress rewards, safety rewards and terminal rewards, avoids potential collision risks, and completes tasks more quickly and efficiently. In addition, the complexity of the simulation environment used for neural network strategy training can change randomly, which effectively improves the generalization ability of neural network strategies in new scenarios.

[0141] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for high-speed navigation and obstacle avoidance of an unmanned aerial vehicle, characterized in that: The method comprises: Obtaining observation information of the drone; the observation information includes the drone position, drone attitude, drone linear velocity, drone angular velocity, the relative position of the drone's current position and several target points that the drone is about to pass, and the relative position of several obstacles within the drone's observation range and the nearest obstacles; Inputting the observation information into a pre-trained neural network strategy to obtain an execution action of the UAV; the execution action includes a combined thrust and an angular velocity of the UAV; The training method of the neural network strategy includes: Model the UAV navigation and obstacle avoidance problem as a Markov decision model in advance; Acquire simulated observation information of the UAV through a pre-built simulation environment, wherein the complexity of the simulation environment can vary randomly, and the elements in the environment include obstacle positions, intermediate waypoint positions, obstacle quantity, and obstacle density; Performing action sampling according to the simulation observation information through the neural network strategy to obtain the simulation execution action of the drone; Calculating the simulation state information of the drone at the next moment and the timely reward at the current moment through the Markov decision model based on the simulation execution action and the simulation state information of the drone at the current moment, wherein the timely reward includes a path progress reward, a safety reward, and a terminal reward; and calculating a discount reward based on the timely reward; Determining whether the discounted reward reaches the desired goal, and if not, optimizing the neural network strategy according to the discounted reward; The calculation formula for the path progress reward is: ; in , Expressed as The coordinates of the target point and the coordinates of the drone at the moment, is the coefficient; The calculation formula of the safety reward is: ; in Indicated in Moment drone and obstacles The distance between The safety bonus factor is used to adjust the aggressiveness of the flight. The safety bonus encourages the drone to stay away from obstacles. is the number of obstacles; The calculation method of the terminal reward includes: Collision Penalty A very large negative value is triggered when a UAV collides with the target, preventing the strategy from generating collision actions again. Mission Reward Value , is the preset coefficient, It is the mission completion time, which is triggered when the UAV completes the mission, and encourages the strategy to produce a strategy with a smaller total flight time.

2. The method for high-speed navigation and obstacle avoidance of an unmanned aerial vehicle according to claim 1, characterized in that: The elements of the Markov decision model include: The state space reflects the set of states of the drone; An action space, reflecting the action set of the drone, is used to sample the actions of the neural network strategy to obtain the simulation execution action; A state transfer equation is used to calculate the simulation state information of the drone at the next moment; A reward function, used to calculate the instantaneous reward at the current moment; Initial state distribution, used to sample the initial state information of the UAV; A discount factor, reflecting the importance of future rewards, is used to calculate the discounted rewards.

3. The method for high-speed navigation and obstacle avoidance of an unmanned aerial vehicle according to claim 1, characterized in that: The calculation method of the path progress reward includes: Calculate the first distance value by using the coordinates of the target point at the current moment and the coordinates of the drone at the current moment; Calculate the second distance value through the coordinates of the target point at the current moment and the coordinates of the drone at the previous moment; The path progress reward is calculated using the first distance value, the second distance value, and the angular velocity of the drone at the current moment.

4. The method for high-speed navigation and obstacle avoidance of an unmanned aerial vehicle according to claim 1, characterized in that: The calculation method of the safety reward includes: The safety reward is calculated according to the distance value between the drone and the obstacle at the current moment.

5. The method for high-speed navigation and obstacle avoidance of an unmanned aerial vehicle according to claim 1, characterized in that: The calculation method of the terminal reward includes: Determine whether the drone has collided. If a collision occurs, generate a collision penalty value; Determine whether the drone has completed the flight mission. If the flight mission is completed, generate a mission reward value; The terminal reward is calculated according to the collision penalty value and the task reward value.

6. The method for high-speed navigation and obstacle avoidance of an unmanned aerial vehicle according to claim 1, characterized in that: The simulation environment includes several parallelized simulation environments, wherein each simulation environment runs independently, and the environmental parameters and UAV parameters therein can be randomly sampled; the environmental parameters are used to reflect obstacle information and / or route information, and the UAV parameters are used to reflect the UAV status.

7. A high-speed navigation and obstacle avoidance system for unmanned aerial vehicles, characterized in that: The system comprises: An information acquisition module is used to acquire observation information of the drone; the observation information includes the drone position, drone attitude, drone linear velocity, drone angular velocity, the relative position of the drone's current position and several target points that are about to pass by, and the relative position of several obstacles within the drone's observation range and the nearest obstacles; An action generation module is used to input the observation information into a pre-trained neural network strategy to obtain an execution action of the UAV; the execution action includes the combined thrust and angular velocity of the UAV; A strategy training module, which is used to pre-model the UAV navigation and obstacle avoidance problem as a Markov decision model; Acquire simulated observation information of the UAV through a pre-built simulation environment, wherein the complexity of the simulation environment can vary randomly, and the elements in the environment include obstacle positions, intermediate waypoint positions, obstacle quantity, and obstacle density; Performing action sampling according to the simulation observation information through the neural network strategy to obtain the simulation execution action of the drone; Calculating the simulation state information of the drone at the next moment and the timely reward at the current moment through the Markov decision model based on the simulation execution action and the simulation state information of the drone at the current moment, wherein the timely reward includes a path progress reward, a safety reward, and a terminal reward; and calculating a discount reward based on the timely reward; Determining whether the discounted reward reaches the desired goal, and if not, optimizing the neural network strategy according to the discounted reward; The calculation formula for the path progress reward is: ; in , Expressed as The coordinates of the target point and the coordinates of the drone at the moment, is the coefficient; The calculation formula of the safety reward is: ; in Indicated in Moment drone and obstacles The distance between The safety bonus factor is used to adjust the aggressiveness of the flight. The safety bonus encourages the drone to stay away from obstacles. is the number of obstacles; The calculation method of the terminal reward includes: Collision Penalty A very large negative value is triggered when a UAV collides with the target, preventing the strategy from generating collision actions again. Mission Reward Value , is the preset coefficient, It is the mission completion time, which is triggered when the UAV completes the mission, and encourages the strategy to produce a strategy with a smaller total flight time.

8. A terminal, characterized in that: The terminal includes a memory and one or more processors; the memory stores one or more programs; the program contains instructions for executing the unmanned aerial vehicle high-speed navigation and obstacle avoidance method as described in any one of claims 1-6; and the processor is used to execute the program.

9. A computer-readable storage medium having a plurality of instructions stored thereon, characterized in that: The instructions are suitable for being loaded and executed by a processor to implement the steps of the drone high-speed navigation and obstacle avoidance method described in any one of claims 1-6 above.