Unmanned aerial vehicle flight path planning method and device, unmanned aerial vehicle and storage medium

Abstract

The invention relates to the technical field of unmanned aerial vehicle autonomous control, in particular to an unmanned aerial vehicle flight path planning method and device, an unmanned aerial vehicle and a storage medium. The method comprises the following steps: deploying a trained agent on an unmanned aerial vehicle; acquiring the current state of the unmanned aerial vehicle; based on the current state, determining a trajectory control point of the unmanned aerial vehicle through an intelligent agent; and updating the current state of the unmanned aerial vehicle based on the trajectory control point, and determining the trajectory control point of the unmanned aerial vehicle through an intelligent agent based on the updated current state until the trajectory duration is reached so as to generate the space-time trajectory of the unmanned aerial vehicle. According to the embodiment of the invention, deep learning and the multi-head attention mechanism are combined, and the complex relationship among the reference trajectory, the unmanned aerial vehicle trajectory and the environmental obstacle is captured through the multi-head attention mechanism, so that the flight attitude can be flexibly adjusted to pass through a narrow gap, thereby improving the autonomous flight capability of the unmanned aerial vehicle in a complex environment, and improving the flight experience of the unmanned aerial vehicle. And the safety of the unmanned aerial vehicle in aggressive flight can be ensured.

Description

Technical Field

[0001] The present application relates to the technical field of autonomous control of unmanned aerial vehicles, and in particular to a method and device for planning a flight path of an unmanned aerial vehicle, an unmanned aerial vehicle, and a storage medium. Background Art

[0002] With the rapid development of drone technology, quad-rotor drones have gradually become the preferred unmanned platform for performing tasks in complex environments due to their high flexibility and versatility. However, when current drones fly in complex environments, their trajectory planning often needs to stick closely to surrounding obstacles. How to ensure the safe flight of drones when close to obstacles has become an important issue. Existing methods can only plan obstacle avoidance routes according to the drone's level flight size when dealing with obstacles in narrow spaces, and cannot guarantee the safety of drones in aggressive flights.

[0003] Therefore the prior art still needs to be improved and enhanced. Summary of the invention

[0004] The technical problem to be solved by the present application is to provide a method, device, drone and storage medium for planning the flight path of a drone in view of the deficiencies in the prior art.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a method for planning a flight path of a drone, wherein the method for planning an aggressive flight path of a drone specifically includes: Deploy a trained agent on a drone, wherein the agent uses a hybrid discrete-continuous reinforcement learning architecture combined with an attention mechanism; Acquire the current state of the drone, wherein the current state includes a pre-built reference trajectory, a safe distance for gap collision, a preset number of trajectory control points, a distance between each trajectory control point, and a trajectory duration; Based on the current state, determining a trajectory control point of the drone through the agent; The current state of the drone is updated based on the trajectory control point, and based on the updated current state, the trajectory control point of the drone is determined by the agent until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone.

[0006] The flight path planning method of the UAV, wherein the space-time trajectory is described by using B-spline to describe the space-time trajectory of the UAV, and the trajectory control points are spline control points of the B-spline used to describe the space-time trajectory.

[0007] The method for planning the flight path of the drone, wherein updating the current state of the drone based on the trajectory control point specifically includes: Updating the safety distance of the gap collision based on the trajectory control point; The trajectory control point is used to replace the trajectory control point that is located at the front in time sequence among the preset number of trajectory control points, and the intervals between the trajectory control points are updated based on the updated preset number of trajectory control points.

[0008] The method for planning the flight path of the unmanned aerial vehicle, wherein the preset number is determined according to the order of the B-spline corresponding to the space-time trajectory.

[0009] The method for planning the flight path of the drone, wherein the updating of the safe distance for the gap collision based on the trajectory control point specifically includes: generating a spatiotemporal trajectory segment based on the trajectory control point and the preset number of trajectory control points; A new safety distance for gap collision is calculated based on the space-time trajectory segment, and the new safety distance for gap collision is used to replace the safety distance for gap collision in the current state, so as to update the safety distance for gap collision.

[0010] In the flight path planning method of the UAV, the calculation formula of the new safety distance of the gap collision is: , , in, Indicates the safe distance for new gap collisions, is the first segment on the time trajectory Points, is the first Points, It is Obstacle points, is the angle between the trajectory direction of the trajectory segment at the moment and the obstacle direction, Represents the trajectory direction vector.

[0011] The flight path planning method of the unmanned aerial vehicle, wherein the reward function used in the training process of the intelligent agent includes trajectory deviation reward, forward reward, acceleration penalty, roll angle reward and trajectory length reward.

[0012] A second aspect of the present application provides a flight path planning device for an unmanned aerial vehicle, wherein the flight path planning device for the unmanned aerial vehicle specifically includes: A deployment module, used to deploy trained agents on drones; A construction module, used to obtain the current state of the drone, wherein the current state includes a pre-constructed reference trajectory, a safety distance for gap collision, a preset number of trajectory control points, a distance between each trajectory control point, and a trajectory duration; A determination module, configured to determine a trajectory control point of the UAV through the agent based on the current state; A control module is used to update the current state of the drone based on the trajectory control point, and based on the updated current state, determine the trajectory control point of the drone through the intelligent agent until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone.

[0013] A third aspect of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the flight path planning method of a drone as described above.

[0014] A fourth aspect of the present application provides a drone, comprising: a processor and a memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in the flight path planning method of the unmanned aerial vehicle as described above are implemented.

[0015] Beneficial effects: 1) Combining deep learning with the multi-head attention mechanism, the multi-head attention mechanism captures the complex relationship between the reference trajectory, the UAV trajectory and environmental obstacles, enabling it to flexibly adjust its flight posture to pass through narrow gaps, thereby improving the UAV's autonomous flight capability in complex environments and ensuring the safety of the UAV in aggressive flight.

[0016] 2) Using onboard sensors to perceive environmental information, drones can fly autonomously in complex environments without prior information about the environment.

[0017] 3) By using B-spline to continuously describe the space-time trajectory, the smoothness and stability of the flight trajectory are ensured.

[0018] 4) By updating the safe distance for gap collision, the drone can maintain a sufficient safe distance from surrounding obstacles, thereby further ensuring the safety of the drone during aggressive flight. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flowchart of a method for planning the flight path of a drone provided in an embodiment of the present application.

[0021] Figure 2 The figure is a principle flow chart of a specific embodiment.

[0022] Figure 3 Schematic diagram of trajectory planning results in the simulation environment.

[0023] Figure 4 A functional block diagram of a flight path planning device for a drone provided in an embodiment of the present application.

[0024] Figure 5 A block diagram of the working principle of the drone provided in the embodiment of the present application. DETAILED DESCRIPTION

[0025] The embodiments of the present application provide a method, device, drone and storage medium for planning the flight path of a drone. To make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0026] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0027] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0028] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0029] After research, it was found that with the rapid development of drone technology, quadcopters have gradually become the preferred unmanned platform for performing tasks in complex environments due to their high flexibility and versatility. However, when drones fly in complex environments, their trajectory planning often needs to stick to surrounding obstacles. How to ensure the safe flight of drones when they are close to obstacles has become an important issue. When dealing with obstacles in narrow spaces, existing methods can only plan obstacle avoidance routes according to the drone's level flight size, and cannot guarantee the safety of drones in aggressive flights.

[0030] In order to solve the above problems, in an embodiment of the present application, a trained agent is deployed on a drone, wherein the agent adopts a hybrid discrete-continuous reinforcement learning architecture combined with an attention mechanism; obtains the current state of the drone; based on the current state, determines the trajectory control point of the drone through the agent; updates the current state of the drone based on the trajectory control point, and based on the updated current state, determines the trajectory control point of the drone through the agent until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone. The embodiment of the present application combines deep learning with a multi-head attention mechanism, and uses a multi-head attention mechanism to capture the complex relationship between the reference trajectory, the drone trajectory, and environmental obstacles, so that it can flexibly adjust the flight posture to pass through narrow gaps, thereby improving the autonomous flight capability of the drone in complex environments, and can ensure the safety of the drone in aggressive flight.

[0031] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0032] This embodiment provides a method for planning a flight path of a drone. Figure 1 As shown, the method includes: S10. Deploy the trained agent on the drone.

[0033] Specifically, the agent is trained to generate trajectory control points based on the current state of the drone, so as to generate a spatiotemporal trajectory based on the trajectory control points. Figure 2 As shown, the agent adopts a hybrid discrete-continuous reinforcement learning architecture combined with an attention mechanism. Hybrid discrete-continuous reinforcement learning means that the agent needs to select a discrete action and the value of one or more continuous action parameters corresponding to the discrete action at each step. In other words, the agent selects a discrete trajectory control point and then forms a continuous spatiotemporal trajectory based on the discrete trajectory control point.

[0034] like Figure 2 As shown in the figure, in the training process of the intelligent agent, a deep Q-value network and a discrete evaluation network are combined. Among them, the deep Q-value network processes the path information, error information and trajectory control point status by using the self-attention mechanism, and outputs the Q-value estimation (i.e., the value estimation of the discrete trajectory control point). Specifically, the deep Q-value network includes an input layer, a self-attention layer and an output layer. The self-attention layer is configured with a self-attention mechanism. The self-attention mechanism captures the complex relationship between the reference trajectory, the UAV trajectory and the environmental obstacles to determine the attention weight of the environmental information in each decision step, so that the UAV can adjust its behavior strategy according to the real-time changes of the environmental information. The discrete evaluation network adopts a dual Q network structure for value estimation in continuous action space (i.e., for value estimation of continuous moment trajectories).

[0035] Exemplarily, the expression for determining the attention weight through the self-attention mechanism in the deep Q-value network can be: , in, express Moment and The time between The attention weight calculated by each head, yes Moment The query vector of the head, yes Moment The key vector of the head, is the attention head dimension. Moment The query vector and Moment The key vectors of each head are determined based on the current state at each moment.

[0036] Furthermore, the process of constructing spatiotemporal trajectories based on intelligent agents can be modeled as a Markov decision process to facilitate trajectory generation and optimization through reinforcement learning. The Markov decision process can be composed of four tuples: Description, where represents the state space, represents the action space, represents the state transition probability, Represents the reward function. Specifically, the state space is used to reflect the agent's environmental information, the action space is used to reflect the actions that the agent can choose, the state transition probability is used to determine the Q value corresponding to each action, and the reward function is used to provide feedback on the agent's decision. Among them, the current state may include a reference trajectory, a safe distance for gap collision, a preset number of trajectory control points, the distance between each trajectory control point, and the trajectory duration. The action space includes a set of trajectory control points. Specifically, in order to make the trajectory control point easy to control, the trajectory control point can use an acceleration trajectory control point. State transition probability It can be expressed as: , in, express The current state at the moment, express The action of the moment, Represents the time step.

[0037] The reward function used includes trajectory deviation reward, forward reward, jerk penalty, roll angle reward and trajectory length reward. That is to say, during the training process of the agent, a reward function is constructed based on trajectory deviation reward, forward reward, jerk penalty, roll angle reward and trajectory length reward. For example, the sum of trajectory deviation reward, forward reward, jerk penalty, roll angle reward and trajectory length reward is used as the reward function, or the average of trajectory deviation reward, forward reward, jerk penalty, roll angle reward and trajectory length reward is used as the reward function, or the trajectory deviation reward, forward reward, jerk penalty, roll angle reward and trajectory length reward are weighted to determine the establishment function, etc.

[0038] In the embodiment of the present application, the sum of the trajectory deviation reward, the forward reward, the jerk penalty, the roll angle reward and the trajectory length reward is used as the reward function, that is, the reward function can be expressed as: , in, represents the reward function, represents the trajectory deviation reward, Indicates the forward reward, represents the acceleration penalty, represents the roll angle bonus, represents the trajectory length reward.

[0039] The trajectory deviation reward is used to encourage the flight trajectory to remain within the safety corridor to reduce the risk of collision. The expression of the trajectory deviation reward can be: , in, represents the trajectory deviation reward, Indicates the maximum allowable deviation ， Indicates the actual deviation, Indicates the deviation in the vertical direction, represents the time step, and is a constant, for example, .

[0040] The forward reward is used to encourage the drone to continue moving along the reference path to prevent stagnation. The forward reward can be expressed as: , in, Indicates the forward reward, represents the forward reward coefficient, represents the advance distance between adjacent time steps, is a constant, for example, .

[0041] The jerk penalty is used to suppress drastic jerk changes to ensure the smoothness of the flight trajectory, where the expression of the jerk penalty can be: , in, represents the acceleration penalty, and To represent the penalty coefficient, for example, , ; represents the horizontal acceleration, is the vertical acceleration, Indicates the maximum permissible value of the horizontal acceleration. is the maximum permissible value of the vertical acceleration.

[0042] The roll angle reward is used to ensure a smooth transition of the posture, where the expression of the roll angle reward can be: , in, represents the roll angle bonus, represents the sensitivity adjustment factor, represents the roll angle error, represents the roll angle variance reward, Represents the roll angle error penalty coefficient, which is a constant.

[0043] Further, the sensitivity adjustment may be determined according to the average value of the trajectory angle change, which may be expressed as , is the average value of the trajectory angle change. The roll angle error can be determined based on the actual roll angle and can be expressed as , Indicates the actual roll angle, Indicates the expected roll angle, the expected roll angle The Gaussian distribution model is adopted, that is, , represents the maximum roll angle, and Controls the center and width of the Gaussian distribution. The roll angle variance reward can be expressed as ,in is the roll angle bonus, Represents the scaling factor.

[0044] Of course, in actual applications, the roll angle reward, roll angle error and roll angle variance reward can also be determined in other ways. For example, directly setting a corresponding table of roll angle reward, roll angle error and roll angle variance reward, and then selecting from the corresponding relationships according to the average value of the trajectory angle change, the actual roll angle, the expected roll angle and the maximum roll angle, or setting other calculation methods for the roll angle reward, roll angle error and roll angle variance reward.

[0045] The trajectory length reward is used to encourage the generation of a complete traversal trajectory, where the expression of the trajectory length reward can be: , in, represents the length bonus coefficient, represents the actual track length, Indicates the minimum expected length.

[0046] S20: Obtain the current state of the drone.

[0047] Specifically, the current state is used to reflect the environmental information at the current moment, wherein the current state can be determined based on the airborne sensor of the drone sensing the environmental information. For example, the obstacle information is sensed by the airborne sensor, and then the safe distance of the gap collision is determined based on the sensed obstacle information, the reference trajectory is pre-constructed (for example, the reference trajectory is constructed by a heuristic method, etc.), the preset number of trajectory control points are the trajectory control points of the preset number of moments before the current moment (when the current state corresponds to the initial moment, the preset number of trajectory control points can be generated in advance for the initial moment), the distance between each trajectory control point is determined based on the preset number of trajectory control points, and the trajectory duration is pre-set. To this end, when obtaining the current state of the drone, the obstacle information can be sensed by the airborne sensor first, and the safe distance of the gap collision can be determined based on the sensed obstacle information, and then the preset number of trajectory control points before the current moment are obtained, and the distance between the trajectory control points is determined based on the obtained trajectory control points, and finally the reference trajectory, the safe distance of the gap collision, the preset number of trajectory control points, the distance between each trajectory control point and the trajectory duration are used as the current state of the drone.

[0048] Further, the preset number may be pre-set or determined according to the space-time trajectory. In the embodiment of the present application, the space-time trajectory is described by B-spline, and the preset number is determined based on the order of B-spline, for example, preset number=B-spline order-1.

[0049] S30. Based on the current state, determine the trajectory control point of the drone through the intelligent agent.

[0050] Specifically, after obtaining the current state, the current state is input into the agent so that the trajectory control point of the drone is output through the agent, where the trajectory control point is the trajectory control point at the moment corresponding to the current state. Figure 2 As shown, after obtaining the current state, the state data in the current state is vectorized to obtain the embedded vector of the state data in the current state, and then the embedded vectors of each state data are spliced ​​to obtain the input vector, and the input vector is used as the input data of the attention layer in the intelligent agent, and the attention weight is determined based on the input data by the attention layer, and finally the trajectory control point of the drone is determined based on the attention weight by the activation function layer.

[0051] S40, updating the current state of the drone based on the trajectory control point, and determining the trajectory control point of the drone through the agent based on the updated current state until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone.

[0052] Specifically, the space-time trajectory is a flight trajectory that is a correction of the reference trajectory, that is, the drone will be controlled to fly according to the space-time trajectory. The space-time trajectory uses B-spline to describe the space-time trajectory of the drone, and the trajectory control point is the spline control point of the B-spline used to describe the space-time trajectory. That is to say, after obtaining the trajectory control point, a continuous space-time trajectory segment can be generated based on the trajectory control point and its preceding trajectory control point, that is, a continuous space-time trajectory is formed using discrete trajectory control points.

[0053] For example: Assuming that the 4th-order B-spline is used to describe the space-time trajectory, the space-time trajectory can be expressed as: , in, represents the space-time trajectory, represents the trajectory control point, represents the number of trajectory control points, Represents the B-spline Level The basis functions are obtained by the following recursive calculation formula: , in, , , , , Both indicate time.

[0054] Furthermore, to ensure the continuity and stability of the trajectory, the derivative properties of the B-spline trajectory include velocity, acceleration, and jerk, where velocity, acceleration, and jerk are expressed as: , in, Indicates speed, represents the velocity trajectory control point, represents acceleration, represents the acceleration trajectory control point, represents the acceleration, Indicates the jerk trajectory control point.

[0055] This application determines the velocity, acceleration and jerk through a B-spline description based on the momentary trajectory, so that the trajectory control points of the drone can accurately define the shape of the trajectory, and then determine the expected dynamic behavior of the quadrotor drone when executing the current moment trajectory, laying the foundation for the subsequent trajectory optimization process.

[0056] Furthermore, after obtaining the trajectory control points, the preset number of trajectory control points and the spacing between each trajectory control point in the current state of the drone can be updated according to the trajectory control points to update the current state; the safety spacing of the gap collision, the preset number of trajectory control points and the spacing between each trajectory control point can also be updated according to the trajectory control points to update the current state. Among them, when updating the safety spacing of the gap collision, the safety spacing of the gap collision can be updated using the perceived obstacle information and the newly generated trajectory control points, or the safety spacing of the gap collision can be updated based on the new trajectory control point and the space-time trajectory segment formed by the preset number of its predecessor trajectory control points and the perceived obstacle information, or the safety spacing of the gap collision can be updated based on the new trajectory control point and the space-time trajectory segment formed by the preset number of its predecessor trajectory control points and the obstacle information collected by the drone at the trajectory control point.

[0057] In a specific implementation, after obtaining the trajectory control point, it can be determined whether the trajectory control point is located within the coverage of the existing obstacle information. When the trajectory control point is located within the coverage of the existing obstacle information, the existing obstacle information is directly used to update the safety distance of the gap collision. When the trajectory control point is not located within the coverage of the existing obstacle information, the drone is controlled to re-perceive the obstacle information. Of course, other methods can also be used to determine whether it is necessary to re-perceive the environmental information, such as the relationship between the distance between the trajectory control point and the coverage area of ​​the obstacle information and a preset threshold.

[0058] It should be noted that after obtaining the trajectory control point, the current state can be directly updated to obtain the next trajectory control point until the trajectory duration of the spatiotemporal trajectory generated based on all trajectory control points meets the trajectory duration in the current state, and then the generation of trajectory control points is stopped and the spatiotemporal trajectory of the trajectory is formed. Figure 3 Of course, in practical applications, after a trajectory control point is generated, a space-time trajectory segment is generated based on the trajectory control point and the drone is controlled to fly according to the space-time trajectory point, and the process of steps S10-S30 is performed during the flight so that the path planning during the flight reaches the preset trajectory duration.

[0059] Further, no matter which of the above methods is used to update the safe distance of the gap collision and whether the method of planning the path first and then flying or planning the path during the flight is used, the updating process of the current state can be: Updating the safety distance of the gap collision based on the trajectory control point; The trajectory control point is used to replace the trajectory control point that is located at the front in time sequence among the preset number of trajectory control points, and the intervals between the trajectory control points are updated based on the updated preset number of trajectory control points.

[0060] Specifically, updating the safety distance of the gap collision refers to updating the safe distance between the space-time trajectory and the obstacle, and after updating the safety distance of the gap collision, the safety distance of the gap collision before the update can be recorded, so that a safe flight corridor can be constructed according to the safety distances of all gap collisions, and real-time update of the safety distance between the trajectory point and the obstacle gap can be achieved.

[0061] In one implementation, updating the safe distance of the gap collision based on the trajectory control point specifically includes: generating a spatiotemporal trajectory segment based on the trajectory control point and the preset number of trajectory control points; A new safety distance for gap collision is calculated based on the space-time trajectory segment, and the new safety distance for gap collision is used to replace the safety distance for gap collision in the current state, so as to update the safety distance for gap collision.

[0062] Specifically, after the space-time trajectory segment is generated, the new gap collision safety distance can be calculated based on the space-time trajectory segment and the obstacle information perceived by the drone, and then the new gap collision safety distance is used to calculate the gap collision safety distance in the current state. The calculation formula of the new gap collision safety distance is: , , in, Indicates the safe distance for new gap collisions, is the first segment on the time trajectory Points, is the first Points, It is Obstacle points, is the angle between the trajectory direction of the trajectory segment at the moment and the obstacle direction, Represents the trajectory direction vector.

[0063] It should be noted that the obstacle point may be obstacle information sensed by an airborne sensor, and the obstacle information may be the existing obstacle information described above, or may be newly sensed obstacle information, and no specific limitation is made here.

[0064] In summary, this embodiment provides a flight path planning method based on the above-mentioned drone, the method comprising deploying a trained agent on the drone; obtaining the current state of the drone; determining the trajectory control point of the drone through the agent based on the current state; updating the current state of the drone based on the trajectory control point, and determining the trajectory control point of the drone through the agent based on the updated current state until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone. The embodiment of the present application combines deep learning with a multi-head attention mechanism to capture the complex relationship between the reference trajectory, the drone trajectory and environmental obstacles through a multi-head attention mechanism, so that it can flexibly adjust the flight posture to pass through narrow gaps, thereby improving the autonomous flight capability of the drone in a complex environment, and can ensure the safety of the drone in aggressive flight. At the same time, on the basis of ensuring flight safety and trajectory smoothness, the embodiment of the present application enables the drone to fly at high speed (such as 6m / s) and large inclination (such as 60 degrees) in a complex environment.

[0065] Based on the above-mentioned method for planning the flight path of a UAV, this embodiment provides a device for planning the flight path of a UAV, such as Figure 4 As shown, the flight path planning device of the UAV specifically includes: A deployment module 100, for deploying the trained agent on a drone; A construction module 200 is used to obtain the current state of the drone, wherein the current state includes a pre-constructed reference trajectory, a safety distance for gap collision, a preset number of trajectory control points, a distance between each trajectory control point, and a trajectory duration; A determination module 300, configured to determine a trajectory control point of the UAV through the agent based on the current state; The control module 400 is used to update the current state of the drone based on the trajectory control point, and based on the updated current state, determine the trajectory control point of the drone through the intelligent agent until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone.

[0066] Based on the above-mentioned UAV flight path planning method, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the UAV flight path planning method as described in the above-mentioned embodiment.

[0067] Based on the above-mentioned UAV flight path planning method, the present application also provides a UAV, such as Figure 5As shown, it includes at least one processor 20 and a memory 22, and may also include a communications interface 23 and a bus 21. The processor 20, the memory 22 and the communications interface 23 may communicate with each other through the bus 21. The communications interface 23 may transmit information. The processor 20 may call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0068] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0069] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0070] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the drone, etc. In addition, the specific process of the above storage medium and the multiple instruction processors in the drone loading and executing has been described in detail in the above method, and will not be described here one by one.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for planning a flight path of an unmanned aerial vehicle, characterized in that: The method for planning the aggressive flight path of the UAV specifically includes: Deploy a trained agent on a drone, wherein the agent uses a hybrid discrete-continuous reinforcement learning architecture combined with an attention mechanism; Acquire the current state of the drone, wherein the current state includes a pre-built reference trajectory, a safe distance for gap collision, a preset number of trajectory control points, a distance between each trajectory control point, and a trajectory duration; Based on the current state, determining a trajectory control point of the drone through the agent; The current state of the drone is updated based on the trajectory control point, and based on the updated current state, the trajectory control point of the drone is determined by the agent until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone.

2. The method for planning the flight path of an unmanned aerial vehicle according to claim 1, characterized in that: The space-time trajectory is a space-time trajectory of the UAV described by using B-spline, and the trajectory control points are spline control points of the B-spline used to describe the space-time trajectory.

3. The method for planning the flight path of an unmanned aerial vehicle according to claim 1, characterized in that: The updating of the current state of the drone based on the trajectory control point specifically includes: Updating the safety distance of the gap collision based on the trajectory control point; The trajectory control point is used to replace the trajectory control point that is located at the front in time sequence among the preset number of trajectory control points, and the intervals between the trajectory control points are updated based on the updated preset number of trajectory control points.

4. The method for planning the flight path of an unmanned aerial vehicle according to claim 3, characterized in that: The preset number is determined according to the order of the B-spline corresponding to the space-time trajectory.

5. The method for planning the flight path of an unmanned aerial vehicle according to claim 3, characterized in that: The updating of the safe distance of the gap collision based on the trajectory control point specifically includes: generating a spatiotemporal trajectory segment based on the trajectory control point and the preset number of trajectory control points; A new safety distance for gap collision is calculated based on the space-time trajectory segment, and the new safety distance for gap collision is used to replace the safety distance for gap collision in the current state, so as to update the safety distance for gap collision.

6. The method for planning the flight path of an unmanned aerial vehicle according to claim 4, characterized in that: The calculation formula for the new safety distance of gap collision is: ， ， in, Indicates the safe distance for new gap collisions, is the first segment on the time trajectory Points, is the first Points, It is Obstacle points, is the angle between the trajectory direction of the trajectory segment at the moment and the obstacle direction, Represents the trajectory direction vector.

7. The method for planning the flight path of an unmanned aerial vehicle according to claim 1, characterized in that: The reward function used in the training process of the agent includes trajectory deviation reward, forward reward, acceleration penalty, roll angle reward and trajectory length reward.

8. A flight path planning device for an unmanned aerial vehicle, characterized in that: The flight path planning device of the UAV specifically includes: A deployment module, used to deploy trained agents on drones; A construction module, used to obtain the current state of the drone, wherein the current state includes a pre-constructed reference trajectory, a safety distance for gap collision, a preset number of trajectory control points, a distance between each trajectory control point, and a trajectory duration; A determination module, configured to determine a trajectory control point of the UAV through the agent based on the current state; A control module is used to update the current state of the drone based on the trajectory control point, and based on the updated current state, determine the trajectory control point of the drone through the intelligent agent until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the flight path planning method of the drone as described in any one of claims 1-7.

10. A drone, characterized in that: include: Processor and memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in the flight path planning method of the drone as described in any one of claims 1-7 are implemented.

Images