UAV flight path planning method, device, UAV and storage medium

Abstract

The present application relates to the field of autonomous control technology for drones, and in particular to a method, device, drone, and storage medium for planning the flight path of a drone. The method includes deploying a trained intelligent agent on a drone; obtaining the current state of the drone; determining the trajectory control point of the drone through the intelligent agent based on the current state; updating the current state of the drone based on the trajectory control point, and determining the trajectory control point of the drone through the intelligent agent based on the updated current state until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone. The embodiment of the present application combines deep learning with a multi-head attention mechanism to capture the complex relationship between the reference trajectory, the drone trajectory, and environmental obstacles through the multi-head attention mechanism, so that the drone can flexibly adjust its flight posture to pass through narrow gaps, thereby improving the autonomous flight capability of the drone in complex environments and ensuring the safety of the drone in aggressive flight.

Description

Technical Field

[0001] The present application relates to the field of autonomous control technology for unmanned aerial vehicles (UAVs), and in particular to a method and device for planning a flight path for a UAV, a UAV, and a storage medium. Background Art

[0002] With the rapid development of drone technology, quadcopters, due to their high flexibility and versatility, have gradually become the preferred unmanned platform for performing missions in complex environments. However, when currently operating drones in complex environments, their trajectory planning often requires them to closely adhere to surrounding obstacles. Ensuring safe flight when approaching obstacles has become a critical issue. Existing methods can only plan obstacle avoidance routes based on the drone's level flight dimensions when dealing with obstacles in confined spaces, failing to guarantee the drone's safety during aggressive flight.

[0003] Therefore existing technology still needs to be improved and improved. Summary of the Invention

[0004] The technical problem to be solved by this application is to provide a method, device, drone and storage medium for planning the flight path of a drone in response to the shortcomings of the existing technology.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a method for planning a flight path of a drone, wherein the method for planning an aggressive flight path of a drone specifically includes:

[0006] Deploying a trained agent on a drone, wherein the agent uses a hybrid discrete-continuous reinforcement learning architecture combined with an attention mechanism;

[0007] Obtaining a current state of the drone, wherein the current state includes a pre-established reference trajectory, a safe distance for gap collision, a preset number of trajectory control points, a distance between each trajectory control point, and a trajectory duration;

[0008] Based on the current state, determining a trajectory control point of the UAV through the agent;

[0009] The current state of the drone is updated based on the trajectory control point, and based on the updated current state, the trajectory control point of the drone is determined by the intelligent agent until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone.

[0010] The flight path planning method of the UAV is described, wherein the space-time trajectory is described by using B-spline, and the trajectory control points are spline control points of the B-spline used to describe the space-time trajectory.

[0011] In the method for planning the flight path of a drone, updating the current state of the drone based on the trajectory control point specifically includes:

[0012] Updating the safety distance of the gap collision based on the trajectory control point;

[0013] The trajectory control point is used to replace the trajectory control point that is located at the front of the preset number of trajectory control points in time sequence, and the intervals between the trajectory control points are updated based on the updated preset number of trajectory control points.

[0014] The method for planning the flight path of a drone, wherein the preset number is determined according to the order of the B-spline corresponding to the space-time trajectory.

[0015] In the method for planning the flight path of the UAV, updating the safe distance for gap collision based on the trajectory control point specifically includes:

[0016] generating a spatiotemporal trajectory segment based on the trajectory control point and the preset number of trajectory control points;

[0017] A new gap collision safety distance is calculated based on the space-time trajectory segment, and the new gap collision safety distance is used to replace the gap collision safety distance in the current state to update the gap collision safety distance.

[0018] In the method for planning the flight path of the UAV, the calculation formula for the new gap collision safety distance is:

[0019] ,

[0020] ,

[0021] in, Indicates the safe distance for new gap collisions, is the first segment on the time trajectory Points, is the first Points, It is Obstacle points, is the angle between the trajectory direction of the moment segment and the obstacle direction, Represents the trajectory direction vector.

[0022] The flight path planning method of the UAV, wherein the reward function used in the training process of the intelligent agent includes trajectory deviation reward, forward reward, acceleration penalty, roll angle reward and trajectory length reward.

[0023] A second aspect of the present application provides a flight path planning device for a drone, wherein the flight path planning device for the drone specifically includes:

[0024] A deployment module for deploying trained agents on drones;

[0025] A construction module is used to obtain the current state of the UAV, wherein the current state includes a pre-constructed reference trajectory, a safe distance for gap collision, a preset number of trajectory control points, a distance between each trajectory control point, and a trajectory duration;

[0026] a determination module, configured to determine, based on the current state, a trajectory control point of the UAV through the agent;

[0027] A control module is configured to update the current state of the drone based on the trajectory control point, and determine the trajectory control point of the drone through the intelligent agent based on the updated current state until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone.

[0028] A third aspect of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the flight path planning method of a drone as described above.

[0029] A fourth aspect of the present application provides a drone, comprising: a processor and a memory;

[0030] The memory stores a computer-readable program executable by the processor;

[0031] When the processor executes the computer-readable program, the steps in any of the above-described methods for planning the flight path of a UAV are implemented.

[0032] Beneficial effects:

[0033] 1) Combining deep learning with a multi-head attention mechanism, the multi-head attention mechanism captures the complex relationship between the reference trajectory, the drone trajectory, and environmental obstacles, enabling it to flexibly adjust its flight posture to pass through narrow gaps, thereby improving the drone's autonomous flight capabilities in complex environments and ensuring the drone's safety during aggressive flight.

[0034] 2) Using onboard sensors to perceive environmental information, drones can fly autonomously in complex environments without prior information about the environment.

[0035] 3) By using B-spline to continuously describe the space-time trajectory, the smoothness and stability of the flight trajectory are guaranteed.

[0036] 4) By updating the safe distance for gap collision, the drone can maintain a sufficient safe distance from surrounding obstacles, thereby further ensuring the safety of the drone during aggressive flight. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0038] Figure 1 A flowchart of a method for planning the flight path of a drone provided in an embodiment of the present application.

[0039] Figure 2 The figure is a principle flow chart of a specific embodiment.

[0040] Figure 3 Schematic diagram of trajectory planning results in the simulation environment.

[0041] Figure 4 This is a block diagram of the principles of a flight path planning device for a drone according to an embodiment of the present application.

[0042] Figure 5 This is a block diagram of the principles of the drone provided in the embodiments of the present application. DETAILED DESCRIPTION

[0043] The embodiments of the present application provide a method, apparatus, drone, and storage medium for planning a flight path for a drone. To make the objectives, technical solutions, and effects of the present application more clear and explicit, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to explain the present application and are not intended to limit the present application.

[0044] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0045] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0046] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not imply the order of execution. The order of execution of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of this application.

[0047] Research has shown that with the rapid development of drone technology, quadcopters, due to their high flexibility and versatility, have gradually become the preferred unmanned platform for performing missions in complex environments. However, when currently operating drones in complex environments, their trajectory planning often requires them to closely follow surrounding obstacles. Ensuring safe flight when approaching obstacles has become a critical issue. Existing methods for dealing with obstacles in confined spaces can only plan obstacle avoidance routes based on the drone's horizontal flight dimensions, failing to guarantee safety during aggressive flight.

[0048] In order to solve the above problems, in an embodiment of the present application, a trained intelligent agent is deployed on a drone, wherein the intelligent agent adopts a hybrid discrete-continuous reinforcement learning architecture combined with an attention mechanism; obtains the current state of the drone; determines the trajectory control point of the drone based on the current state through the intelligent agent; updates the current state of the drone based on the trajectory control point, and determines the trajectory control point of the drone based on the updated current state through the intelligent agent until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone. The embodiment of the present application combines deep learning with a multi-head attention mechanism to capture the complex relationship between the reference trajectory, the drone trajectory and environmental obstacles through the multi-head attention mechanism, so that the drone can flexibly adjust its flight posture to pass through narrow gaps, thereby improving the autonomous flight capability of the drone in complex environments and ensuring the safety of the drone in aggressive flight.

[0049] The application content will be further explained below through description of embodiments in conjunction with the accompanying drawings.

[0050] This embodiment provides a method for planning the flight path of a drone. Figure 1 As shown, the method includes:

[0051] S10. Deploy the trained agent on the drone.

[0052] Specifically, the agent is trained to generate trajectory control points based on the current state of the drone, so as to generate a spatiotemporal trajectory based on the trajectory control points. Figure 2 As shown, the agent uses a hybrid discrete-continuous reinforcement learning architecture combined with an attention mechanism. Hybrid discrete-continuous reinforcement learning requires the agent to select a discrete action and the corresponding values of one or more continuous action parameters at each step. In other words, the agent selects a discrete trajectory control point and then forms a continuous spatiotemporal trajectory based on the discrete trajectory control points.

[0053] like Figure 2As shown in Figure 1, the training process of the intelligent agent combines a deep Q-value network and a discrete evaluation network. The deep Q-value network utilizes a self-attention mechanism to process path information, error information, and trajectory control point states, outputting Q-value estimates (i.e., discrete trajectory control point value estimates). Specifically, the deep Q-value network comprises an input layer, a self-attention layer, and an output layer. The self-attention layer is equipped with a self-attention mechanism. This mechanism captures the complex relationship between the reference trajectory, the drone trajectory, and environmental obstacles, determining the attention weight of environmental information in each decision step. This enables the drone to adjust its behavioral strategy based on real-time changes in environmental information. The discrete evaluation network employs a dual Q-network structure for value estimation in continuous action spaces (i.e., for continuous moment-to-moment trajectory value estimation).

[0054] For example, the expression for determining the attention weight through the self-attention mechanism in the deep Q-value network can be:

[0055] ,

[0056] in, express Moment and The first time The attention weights calculated by each head are: yes Moment The query vector of the head, yes Moment The key vector of the head, is the attention head dimension. Moment The query vector and Moment The key vectors of each head are determined based on the current state at each moment.

[0057] Furthermore, the process of constructing spatiotemporal trajectories based on intelligent agents can be modeled as a Markov decision process to facilitate trajectory generation and optimization through reinforcement learning, where the Markov decision process can be represented by the four-tuple Description, where represents the state space, represents the action space, represents the state transition probability, Represents the reward function. Specifically, the state space is used to reflect the environment information of the agent, the action space is used to reflect the actions that the agent can choose, the state transition probability is used to determine the Q value corresponding to each action, and the reward function is used to provide feedback on the agent's decision. Among them, the current state may include a reference trajectory, a safe distance for gap collision, a preset number of trajectory control points, the distance between each trajectory control point, and the trajectory duration. The action space includes a set of trajectory control points. Specifically, in order to make the trajectory control point easy to control, the trajectory control point can adopt an acceleration trajectory control point. State transition probability It can be expressed as:

[0058] ,

[0059] in, express The current state at the moment, express The action of the moment, Indicates the time step.

[0060] The reward function used includes a trajectory deviation reward, a forward movement reward, a jerk penalty, a roll angle reward, and a trajectory length reward. That is, during the training process of the agent, a reward function is constructed based on the trajectory deviation reward, forward movement reward, jerk penalty, roll angle reward, and trajectory length reward. For example, the sum of the trajectory deviation reward, forward movement reward, jerk penalty, roll angle reward, and trajectory length reward is used as the reward function, or the average of the trajectory deviation reward, forward movement reward, jerk penalty, roll angle reward, and trajectory length reward is used as the reward function, or the trajectory deviation reward, forward movement reward, jerk penalty, roll angle reward, and trajectory length reward are weighted to determine the established function.

[0061] In the embodiment of the present application, the sum of the trajectory deviation reward, the forward reward, the jerk penalty, the roll angle reward, and the trajectory length reward is used as the reward function, that is, the reward function can be expressed as:

[0062] ,

[0063] in, represents the reward function, represents the trajectory deviation reward, Indicates the forward reward, represents the jerk penalty, represents the roll angle bonus, represents the trajectory length reward.

[0064] The trajectory deviation reward is used to encourage the flight trajectory to remain within the safety corridor to reduce the risk of collision. The expression of the trajectory deviation reward can be:

[0065] ,

[0066] in, represents the trajectory deviation reward, Indicates the maximum allowable deviation ， Indicates the actual deviation, Indicates the deviation in the vertical direction, represents the time step, and is a constant, for example, .

[0067] The forward reward is used to encourage the drone to continue moving along the reference path to prevent stagnation. The forward reward can be expressed as:

[0068] ,

[0069] in, Indicates the forward reward, represents the forward reward coefficient, represents the forward distance between adjacent time steps, is a constant, for example, .

[0070] The jerk penalty is used to suppress drastic jerk changes to ensure the smoothness of the flight trajectory. The jerk penalty can be expressed as:

[0071] ,

[0072] in, represents the jerk penalty, and To express the penalty coefficient, for example, , ; represents the horizontal acceleration, represents the vertical acceleration, Indicates the maximum permissible value of horizontal acceleration, The maximum permissible value of the vertical acceleration.

[0073] The roll angle reward is used to ensure a smooth transition of the posture. The expression of the roll angle reward can be:

[0074] ,

[0075] in, represents the roll angle bonus, represents the sensitivity adjustment factor, represents the roll angle error, represents the roll angle variance reward, Represents the roll angle error penalty coefficient, which is a constant.

[0076] Furthermore, the sensitivity adjustment can be determined based on the average value of the trajectory angle change, which can be expressed as , is the average value of the trajectory angle change. The roll angle error can be determined based on the actual roll angle and can be expressed as , Indicates the actual roll angle, Indicates the expected roll angle, the expected roll angle The Gaussian distribution model is used, that is, , Indicates the maximum roll angle, and Controls the center and width of the Gaussian distribution. The roll angle variance reward can be expressed as ,in is the roll angle bonus, Represents the scaling factor.

[0077] Of course, in actual applications, the roll angle reward, roll angle error, and roll angle variance reward can also be determined in other ways. For example, a corresponding table of roll angle reward, roll angle error, and roll angle variance reward can be directly set, and then selected from the corresponding relationships based on the average value of the trajectory angle change, the actual roll angle, the expected roll angle, and the maximum roll angle. Alternatively, other calculation methods for the roll angle reward, roll angle error, and roll angle variance reward can be set.

[0078] The trajectory length reward is used to encourage the generation of complete traversal trajectories. The expression of the trajectory length reward can be:

[0079] ,

[0080] in, represents the length bonus coefficient, represents the actual track length, Indicates the minimum expected length.

[0081] S20: Obtain the current status of the drone.

[0082] Specifically, the current state reflects the current moment's environmental information. The current state can be determined based on the environmental information sensed by the drone's onboard sensors. For example, obstacle information is sensed by the onboard sensors, and then a safe gap collision distance is determined based on the sensed obstacle information. The reference trajectory is pre-constructed (e.g., using a heuristic method), the preset number of trajectory control points are trajectory control points from a preset number of moments before the current moment (when the current state corresponds to an initial moment, a preset number of trajectory control points can be pre-generated for the initial moment), the spacing between each trajectory control point is determined based on the preset number of trajectory control points, and the trajectory duration is pre-set. To this end, when obtaining the drone's current state, obstacle information can first be sensed by the onboard sensors, and a safe gap collision distance can be determined based on the sensed obstacle information. A preset number of trajectory control points can then be acquired before the current moment, and the spacing between each trajectory control point can be determined based on the acquired trajectory control points. Finally, the reference trajectory, the safe gap collision distance, the preset number of trajectory control points, the spacing between each trajectory control point, and the trajectory duration are used as the drone's current state.

[0083] Furthermore, the preset number can be pre-set or determined based on the spatiotemporal trajectory. In the embodiment of the present application, the spatiotemporal trajectory is described using B-spline, and the preset number is determined based on the order of the B-spline, for example, the preset number = B-spline order - 1.

[0084] S30. Based on the current state, determine the trajectory control point of the UAV through the intelligent agent.

[0085] Specifically, after obtaining the current state, the current state is input into the agent to output the trajectory control point of the UAV through the agent, wherein the trajectory control point is the trajectory control point at the moment corresponding to the current state. Figure 2 As shown, after obtaining the current state, the state data in the current state is vectorized to obtain the embedding vector of the state data in the current state, and then the embedding vectors of each state data are spliced to obtain the input vector, and the input vector is used as the input data of the attention layer in the intelligent agent. The attention weight is determined based on the input data through the attention layer, and finally the trajectory control point of the drone is determined based on the attention weight through the activation function layer.

[0086] S40: updating the current state of the drone based on the trajectory control point, and determining the trajectory control point of the drone through the intelligent agent based on the updated current state until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone.

[0087] Specifically, the spatiotemporal trajectory is a modified flight trajectory of a reference trajectory. In other words, the drone is controlled to fly according to the spatiotemporal trajectory. The spatiotemporal trajectory is described using a B-spline, and the trajectory control points are the spline control points of the B-spline used to describe the spatiotemporal trajectory. In other words, after obtaining a trajectory control point, a continuous spatiotemporal trajectory segment can be generated based on the trajectory control point and its predecessors, i.e., a continuous spatiotemporal trajectory is formed using discrete trajectory control points.

[0088] For example: Assuming that a 4th-order B-spline is used to describe the space-time trajectory, the space-time trajectory can be expressed as:

[0089] ,

[0090] in, represents the space-time trajectory, represents the trajectory control point, Indicates the number of trajectory control points, Represents B-spline Level The basis functions are obtained by the following recursive calculation formula:

[0091] ,

[0092] in, , , , , Both indicate time.

[0093] Furthermore, to ensure the continuity and stability of the trajectory, the derivative properties of the B-spline trajectory include velocity, acceleration, and jerk, where velocity, acceleration, and jerk are expressed as:

[0094] ,

[0095] in, Indicates speed, represents the velocity trajectory control point, represents acceleration, represents the acceleration trajectory control point, represents the acceleration, Indicates the jerk trajectory control point.

[0096] This application determines the velocity, acceleration and jerk through B-spline description based on the momentary trajectory, so that the trajectory control points of the drone can accurately define the shape of the trajectory, and then determine the expected dynamic behavior of the quadrotor drone when executing the current moment trajectory, laying the foundation for the subsequent trajectory optimization process.

[0097] Furthermore, after obtaining the trajectory control points, the preset number of trajectory control points and the spacing between each trajectory control point in the current state of the drone can be updated according to the trajectory control points to update the current state; the safety distance for gap collision, the preset number of trajectory control points and the spacing between each trajectory control point can also be updated according to the trajectory control points to update the current state. Specifically, when updating the safety distance for gap collision, the already perceived obstacle information and the newly generated trajectory control points can be used to update the safety distance for gap collision, or the safety distance for gap collision can be updated based on the new trajectory control point and the spatiotemporal trajectory segment formed by the preset number of its predecessor trajectory control points and the already perceived obstacle information, or the safety distance for gap collision can be updated based on the new trajectory control point and the spatiotemporal trajectory segment formed by the preset number of its predecessor trajectory control points and the obstacle information collected by the drone at the trajectory control point.

[0098] In one specific implementation, after acquiring a trajectory control point, it is possible to determine whether the trajectory control point is within the coverage of existing obstacle information. If the trajectory control point is within the coverage of existing obstacle information, the existing obstacle information is directly used to update the safe distance for gap collisions. If the trajectory control point is not within the coverage of existing obstacle information, the drone is controlled to re-perceive obstacle information. Of course, other methods can also be used to determine whether re-perceiving environmental information is necessary, such as the relationship between the distance between the trajectory control point and the area covered by the obstacle information and a preset threshold.

[0099] It should be noted that after obtaining a trajectory control point, the current state can be directly updated to obtain the next trajectory control point, until the trajectory duration of the spatiotemporal trajectory generated based on all trajectory control points meets the trajectory duration in the current state, then the generation of trajectory control points is stopped and the spatiotemporal trajectory of the trajectory is formed. For example, Figure 3 Of course, in actual applications, after a trajectory control point is generated, a spatiotemporal trajectory segment is generated based on the trajectory control point and the drone is controlled to fly according to the spatiotemporal trajectory point, and the process of steps S10-S30 is performed during the flight to ensure that the path planning during the flight reaches the preset trajectory duration.

[0100] Furthermore, regardless of which of the above methods is used to update the safe distance for gap collision and whether the method of planning the path first and then flying or planning the path during flight is used, the updating process of the current state can be:

[0101] Updating the safety distance of the gap collision based on the trajectory control point;

[0102] The trajectory control point is used to replace the trajectory control point that is located at the front of the preset number of trajectory control points in time sequence, and the intervals between the trajectory control points are updated based on the updated preset number of trajectory control points.

[0103] Specifically, updating the safe distance of the gap collision refers to updating the safe distance between the space-time trajectory and the obstacle, and after updating the safe distance of the gap collision, the safe distance of the gap collision before the update can be recorded, so that a safe flight corridor can be constructed according to the safe distances of all gap collisions, and the real-time update of the safe distance between the trajectory point and the obstacle gap can be achieved.

[0104] In one implementation, updating the safe distance for gap collision based on the trajectory control point specifically includes:

[0105] generating a spatiotemporal trajectory segment based on the trajectory control point and the preset number of trajectory control points;

[0106] A new gap collision safety distance is calculated based on the space-time trajectory segment, and the new gap collision safety distance is used to replace the gap collision safety distance in the current state to update the gap collision safety distance.

[0107] Specifically, after the spatiotemporal trajectory segment is generated, a new gap collision safety distance can be calculated based on the spatiotemporal trajectory segment and the obstacle information perceived by the drone, and then the new gap collision safety distance is used to calculate the gap collision safety distance in the current state. The calculation formula for the new gap collision safety distance is:

[0108] ,

[0109] ,

[0110] in, Indicates the safe distance for new gap collisions, is the first segment on the time trajectory Points, is the first Points, It is Obstacle points, is the angle between the trajectory direction of the moment segment and the obstacle direction, Represents the trajectory direction vector.

[0111] It should be noted that the obstacle point may be obstacle information sensed by an onboard sensor, and the obstacle information may be the existing obstacle information described above, or may be newly sensed obstacle information, and no specific limitation is imposed here.

[0112] In summary, this embodiment provides a flight path planning method for the aforementioned drone. The method includes deploying a trained agent on the drone; obtaining the drone's current state; determining the drone's trajectory control points via the agent based on the current state; updating the drone's current state based on the trajectory control points, and determining the drone's trajectory control points via the agent based on the updated current state until the trajectory duration is reached, thereby generating the drone's spatiotemporal trajectory. This embodiment combines deep learning with a multi-head attention mechanism to capture the complex relationship between the reference trajectory, the drone's trajectory, and environmental obstacles. This allows the drone to flexibly adjust its flight attitude to navigate narrow gaps, thereby improving the drone's autonomous flight capabilities in complex environments and ensuring the drone's safety during aggressive flight. Furthermore, while ensuring flight safety and trajectory smoothness, this embodiment enables the drone to fly at high speeds (e.g., 6 m / s) and at large inclination angles (e.g., 60 degrees) in complex environments.

[0113] Based on the above-mentioned UAV flight path planning method, this embodiment provides a UAV flight path planning device, such as Figure 4 As shown, the flight path planning device of the UAV specifically includes:

[0114] A deployment module 100 for deploying the trained agent on a UAV;

[0115] A construction module 200 is configured to obtain a current state of the drone, wherein the current state includes a pre-constructed reference trajectory, a safety distance for gap collision, a preset number of trajectory control points, a distance between each trajectory control point, and a trajectory duration;

[0116] A determination module 300 is configured to determine, based on the current state, a trajectory control point of the UAV through the agent;

[0117] The control module 400 is used to update the current state of the drone based on the trajectory control point, and based on the updated current state, determine the trajectory control point of the drone through the intelligent agent until the trajectory duration is reached to generate the spatiotemporal trajectory of the drone.

[0118] Based on the above-mentioned method for planning the flight path of a drone, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the method for planning the flight path of a drone as described in the above-mentioned embodiment.

[0119] Based on the above-mentioned UAV flight path planning method, the present application also provides a UAV, such as Figure 5 As shown, it includes at least one processor 20 and memory 22, and may also include a communications interface 23 and a bus 21. The processor 20, memory 22, and communications interface 23 can communicate with each other via the bus 21. The communications interface 23 can transmit information. The processor 20 can call logic instructions in the memory 22 to execute the method in the above embodiment.

[0120] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0121] The memory 22, as a computer-readable storage medium, can be configured to store software programs or computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes the software programs, instructions, or modules stored in the memory 22 to perform functional applications and data processing, thereby implementing the methods in the above embodiments.

[0122] The memory 22 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated during the use of the drone. Furthermore, the specific process of loading and executing the aforementioned storage medium and the multiple instructions in the drone's processor has been described in detail in the aforementioned method and will not be further detailed here.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for planning a flight path of an unmanned aerial vehicle, characterized in that: The flight path planning method of the UAV specifically includes: Deploy a trained intelligent agent on a UAV, wherein the intelligent agent adopts a hybrid discrete-continuous reinforcement learning architecture combined with an attention mechanism. The intelligent agent selects a discrete action and the value of one or more continuous action parameters corresponding to the discrete action at each step. The reward function used in the training process of the intelligent agent includes a trajectory deviation reward, a forward reward, an acceleration penalty, a roll angle reward, and a trajectory length reward. Obtaining a current state of the drone, wherein the current state includes a pre-established reference trajectory, a safe distance for gap collision, a preset number of trajectory control points, a distance between each trajectory control point, and a trajectory duration; Based on the current state, determining a trajectory control point of the UAV through the agent; updating a current state of the drone based on the trajectory control point, and determining a trajectory control point of the drone through the agent based on the updated current state until a trajectory duration is reached to generate a spatiotemporal trajectory of the drone; Wherein, updating the current state of the drone based on the trajectory control point specifically includes: Updating the safety distance of the gap collision based on the trajectory control point; Using the trajectory control point to replace the trajectory control point that is located at the front of the preset number of trajectory control points in chronological order, and updating the spacing between the trajectory control points based on the updated preset number of trajectory control points; The roll angle bonus is: ， ， ， ， in, represents the roll angle bonus, represents the sensitivity adjustment factor, represents the roll angle error, represents the roll angle variance reward, represents the roll angle error penalty coefficient, is the average value of trajectory angle change, Indicates the actual roll angle, represents the desired roll angle, Indicates the maximum roll angle, and Controls the center and width of the Gaussian distribution.

2. The method for planning the flight path of an unmanned aerial vehicle according to claim 1, wherein: The space-time trajectory is a B-spline used to describe the space-time trajectory of the UAV, and the trajectory control points are spline control points of the B-spline used to describe the space-time trajectory.

3. The method for planning the flight path of an unmanned aerial vehicle according to claim 1, wherein: The preset number is determined according to the order of the B-spline corresponding to the space-time trajectory.

4. The method for planning the flight path of an unmanned aerial vehicle according to claim 1, wherein: The updating of the safe distance of the gap collision based on the trajectory control point specifically includes: generating a spatiotemporal trajectory segment based on the trajectory control point and the preset number of trajectory control points; A new gap collision safety distance is calculated based on the space-time trajectory segment, and the new gap collision safety distance is used to replace the gap collision safety distance in the current state to update the gap collision safety distance.

5. The method for planning the flight path of an unmanned aerial vehicle according to claim 4, wherein: The calculation formula for the new safety distance of gap collision is: ， ， in, Indicates the safe distance for new gap collisions, is the first segment on the time trajectory Points, is the first Points, It is Obstacle points, is the angle between the trajectory direction of the moment segment and the obstacle direction, Represents the trajectory direction vector.

6. A flight path planning device for an unmanned aerial vehicle, characterized in that: The flight path planning device of the UAV specifically includes: A deployment module is used to deploy a trained intelligent agent on a UAV. The intelligent agent adopts a hybrid discrete-continuous reinforcement learning architecture combined with an attention mechanism. At each step, the intelligent agent needs to select a discrete action and the value of one or more continuous action parameters corresponding to the discrete action. The reward function used in the training process of the intelligent agent includes trajectory deviation reward, forward reward, acceleration penalty, roll angle reward, and trajectory length reward. A construction module is used to obtain the current state of the UAV, wherein the current state includes a pre-constructed reference trajectory, a safe distance for gap collision, a preset number of trajectory control points, a distance between each trajectory control point, and a trajectory duration; a determination module, configured to determine, based on the current state, a trajectory control point of the UAV through the agent; a control module, configured to update a current state of the drone based on the trajectory control point, and determine the trajectory control point of the drone through the agent based on the updated current state until the trajectory duration is reached to generate a spatiotemporal trajectory of the drone; Wherein, updating the current state of the drone based on the trajectory control point specifically includes: Updating the safety distance of the gap collision based on the trajectory control point; Using the trajectory control point to replace the trajectory control point that is located at the front of the preset number of trajectory control points in chronological order, and updating the spacing between the trajectory control points based on the updated preset number of trajectory control points; The roll angle bonus is: ， ， ， ， in, represents the roll angle bonus, represents the sensitivity adjustment factor, represents the roll angle error, represents the roll angle variance reward, represents the roll angle error penalty coefficient, is the average value of trajectory angle change, Indicates the actual roll angle, represents the desired roll angle, Indicates the maximum roll angle, and Controls the center and width of the Gaussian distribution.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the flight path planning method of the drone as described in any one of claims 1 to 5.

8. A drone, characterized in that: include: processor and memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, it implements the steps in the flight path planning method of the drone as described in any one of claims 1 to 5.

Images