Autonomous Environmental Perception, Path Planning and Dynamic Landing Methods and Systems for Unmanned Aerial Vehicles

By obtaining environmental information in real time and combining local and boundary-aware path optimization algorithms, selecting appropriate path planning methods, and using deep reinforcement learning algorithms to dynamically land drones, solving the challenges of drones' perception and path planning in unknown scenarios. Improved path optimization efficiency and landing accuracy.

CN118672284BActive Publication Date: 2024-11-19CHINA JILIANG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411158190.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-11-19
Estimated Expiration
2044-08-22

AI Technical Summary

Technical Problem

In unknown large scenarios, autonomous environment perception and path planning face accuracy, real-time and security challenges. The existing methods are prone to falling into local optimal solutions, resulting in path redundancy and waste of resources, and landing control depends on expert experience and Repeat training.

Method used

By obtaining environmental depth information and two-dimensional plane information in real time, generating three-dimensional environmental information, combining local and boundary-aware path optimization algorithms, selecting appropriate path planning methods, and dynamically landing using deep reinforcement learning algorithms after reaching the global endpoint.

Benefits of technology

It realizes that the drone chooses the appropriate path planning method in different situations, avoids path redundancy and local optimal solutions, improves path optimization efficiency, and realizes accurate dynamic landing, enhances the efficiency of neural network updates, and solves the small sample space and processing. The problem of slow data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118672284B_ABST
    Figure CN118672284B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of unmanned aerial vehicles, and discloses a method and system for autonomous environmental perception, path planning and dynamic landing of unmanned aerial vehicles, the method comprising: obtaining three-dimensional environmental information in real time; determining a global starting point and a global end point, and generating an initial path; optimizing the initial path based on a local path optimization algorithm to obtain a first optimized path; when the perception threshold of the current position of the unmanned aerial vehicle is greater than a preset threshold, optimizing the initial path based on a path optimization method based on boundary perception to obtain a second optimized path and a local end point; when the unmanned aerial vehicle advances to the local end point, switching to optimizing the initial path in real time based on a local path optimization algorithm; when the unmanned aerial vehicle reaches the global end point, performing dynamic landing based on a deep reinforcement learning algorithm. The present invention selects different path planning methods for different situations, avoids path redundancy or falling into a local optimal solution, improves path optimization efficiency, and realizes precise dynamic landing of the unmanned aerial vehicle.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a method and system for autonomous environmental perception, path planning and dynamic landing of unmanned aerial vehicles. Background Art

[0002] In modern society, the rapid development of drone technology and its wide application in various fields, such as military, agriculture, express delivery and disaster relief, have greatly promoted the development of social productivity. However, in unknown large scenarios (such as complex urban environments, unmapped natural terrain or post-disaster ruins, etc.), drone autonomous environmental perception and path planning face many challenges. These challenges mainly include the accuracy and real-time performance of environmental perception, the efficiency and safety of path planning, etc.

[0003] In an unknown large scene, if you want to get the global optimal path, you have to know the environmental information in advance, but this is unrealistic in actual applications. In order to better apply to the actual environment, the existing drone path planning algorithms are all designed as local planning methods, targeting small obstacles in the environment. In such an environment, in order to ensure real-time operation, drones also perceive local information, which will have limitations in space. There is no memory of the perceived environment, and it is easy to continuously perform meaningless re-planning at the same location, wasting resources. Most of the current existing drone landing control methods are based on traditional control strategies, which require a lot of expert experience in design and debugging, and learning methods require multiple repetitive training and learning in a specific environment, so they put forward strict requirements on the specific environment.

[0004] Therefore, there is an urgent need for autonomous environmental perception, path planning and dynamic landing methods and systems for drones, which can select different path planning methods for different situations, avoid path redundancy or falling into local optimal solutions, improve path optimization efficiency, and realize precise dynamic landing of drones. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method and system for autonomous environmental perception, path planning and dynamic landing of UAVs, which can select different path planning methods for different situations, avoid path redundancy or falling into local optimal solutions, improve path optimization efficiency, and realize precise dynamic landing of UAVs.

[0006] The present invention provides a method for autonomous environment perception, path planning and dynamic landing of a drone, comprising the following steps:

[0007] S1. The drone obtains the depth information and two-dimensional plane information of the environment in real time, and generates three-dimensional environment information in space through a three-dimensional reconstruction algorithm based on the depth information and the two-dimensional plane information;

[0008] S2, determine the global starting point and the global end point, and generate an initial path according to the global starting point and the global end point;

[0009] S3, based on the local path optimization algorithm, the initial path is optimized in real time to obtain the first optimized path, and at the same time, it is determined in real time whether the perception threshold of the current position of the drone is greater than the preset threshold; if the perception threshold is greater than the preset threshold, enter S4, otherwise continue to execute S3 until the global end point is reached, and enter S7;

[0010] S4, sensing the boundary of the drone's field of view, and storing the sensed boundary in the form of spatial points, recorded as boundary points, performing distance optimization processing on the boundary points, and obtaining a boundary spatial point set P;

[0011] S5. According to the boundary space point set P, the initial path is optimized in real time based on the boundary-aware path optimization method to obtain a second optimized path and a local end point;

[0012] S6. When the UAV advances along the second optimized path to the local end point, it switches to optimizing the initial path in real time based on the local path optimization algorithm;

[0013] S7. When the drone reaches the global destination, it performs dynamic landing based on the deep reinforcement learning algorithm.

[0014] Further, in S3, the initial path is optimized in real time based on the local path optimization algorithm to obtain a first optimized path, and at the same time, it is determined in real time whether the perception threshold is greater than a preset threshold, wherein the calculation formula of the perception threshold is as follows:

[0015] ;

[0016] Among them, τ per represents the perception threshold, represents the initial path within the UAV’s field of view, represents the first optimized path within the UAV’s field of view, τ represents the preset threshold, and when τ per >τ, enter S4.

[0017] Furthermore, in S4, the distance optimization process is performed on the boundary points to obtain the boundary space point set P including:

[0018] With the boundary point as the center point, a cube area with a side length of 2m is constructed, and the cube area is divided into 8 sub-cube areas with a side length of m. The vertices of each sub-cube area are used as collision judgment points to determine whether the distance between each collision judgment point and the target obstacle is less than the preset collision distance; if there is at least one collision judgment point with a distance less than the preset collision distance between the target obstacle, the boundary point is deleted;

[0019] Repeat the above steps to traverse all boundary points, and construct a boundary space point set P based on the boundary points finally retained.

[0020] Further, in S5, according to the boundary space point set P, the initial path is optimized in real time based on the boundary-aware path optimization method, and the second optimized path and the local end point are obtained, including:

[0021] S51, determining a local endpoint;

[0022] According to the UAV's field of view, select the point on the initial path that is farthest from the UAV's current position as the initial local end point. If the initial local end point is within the obstacle range, double the UAV's field of view distance to select the second local end point on the basis of the initial local end point until the selected local end point is outside the obstacle range.

[0023] S52, determining decision indicators, and calculating a decision function according to the decision indicators; wherein the decision indicators include: adjacent point constraints, distance constraints, and direction constraints;

[0024] The adjacent point constraint expression is as follows:

[0025] ;

[0026] ;

[0027] Among them, d A represents the distance between adjacent points, f A represents the neighboring point cost function, p i represents the i-th boundary point in the boundary space point set P, p c Indicates the current position of the drone. represents the Euclidean distance operation, d min Indicates the minimum value of the selected point range within the UAV’s field of view, d max Indicates the maximum value of the selected point range within the drone's field of view;

[0028] The distance constraint expression is as follows:

[0029] ;

[0030] ;

[0031] Among them, d L Indicates the current position of the drone p c With local endpoint p g The distance between L represents the distance cost function, p g represents the local endpoint, d LminRepresents all boundary points in the boundary space point set P to the local end point p g The minimum distance, d Lmax Represents all boundary points in the boundary space point set P to the local end point p g The maximum value of the distance;

[0032] The direction constraint expression is as follows:

[0033] ;

[0034] ;

[0035] Among them, d D Indicates the specific value of the deviation from the current speed direction, f D represents the direction cost function, v c Indicates the current speed of the drone, d Dmin Indicates the minimum value of deviation from the current speed direction, d Dmax Indicates the maximum value of deviation from the current speed direction;

[0036] The calculation formula of the decision function is as follows:

[0037] ;

[0038] Among them, λ A , L , D They represent the weight of the adjacent point constraint, the weight of the distance constraint, and the weight of the direction constraint, respectively. b represents the optimal replanning point;

[0039] S53, according to the current position of the drone c and the optimal replanning point p b A second optimized path is generated.

[0040] Furthermore, in S6, when the UAV advances along the second optimized path to the local end point, switching to optimizing the initial path in real time based on the local path optimization algorithm includes:

[0041] S61: When the UAV moves along the second optimized path, S5 is re-executed every time a preset time threshold or a preset distance threshold is passed, and the optimal re-planning point p is updated. b and a second optimization path;

[0042] S62: Record the path length L that the UAV flies along the second optimized path from the current position. thresh , when the path length L thresh When it is greater than the preset path length, reselect the local end point p g , according to the new local endpoint, re-execute S5 and update the best replanning point pb and a second optimization path;

[0043] Among them, reselect the local end point p g Specifically:

[0044] Based on the current local endpoint, double the drone's field of view to select a new local endpoint;

[0045] S63, when the drone reaches the local endpoint p g When the local path optimization algorithm is used, the initial path is optimized in real time.

[0046] Further, when the initial path is optimized in real time based on the local path optimization algorithm to obtain the first optimized path, or the initial path is optimized in real time based on the boundary-aware path optimization method to obtain the second optimized path, it also includes:

[0047] According to the current position of the UAV and the local end point, the intermediate point is inserted to generate the first optimized path or the second optimized path;

[0048] The midpoint is calculated as follows:

[0049] ;

[0050] Among them, p m Indicates the middle point, a max Indicates the maximum angular velocity of the drone.

[0051] Furthermore, in S7, when the drone reaches the global endpoint, dynamic landing based on the deep reinforcement learning algorithm includes:

[0052] S71. Construct a landing control agent based on a deep deterministic policy gradient algorithm.

[0053] S72, obtaining observation values ​​of the UAV and the landing platform;

[0054] S73, defining a reward function; wherein the reward function includes a horizontal direction reward function and a vertical direction reward function;

[0055] S74, judging whether the drone meets the landing conditions according to the observation values ​​of the drone and the landing platform and the reward function;

[0056] S75. The landing control agent controls the UAV according to the observation values ​​of the UAV and the landing platform, the reward function, and whether the UAV meets the landing conditions until the UAV completes the dynamic landing.

[0057] The present invention also provides a UAV autonomous environment perception, path planning and dynamic landing system, which is used to execute any of the above-mentioned UAV autonomous environment perception, path planning and dynamic landing methods, and the system includes the following modules:

[0058] A data acquisition module is used to acquire the depth information and two-dimensional plane information of the environment in real time through the UAV, and generate three-dimensional environment information in space through a three-dimensional reconstruction algorithm based on the depth information and the two-dimensional plane information;

[0059] An initial path generation module, connected to the data acquisition module, is used to determine a global starting point and a global end point, and generate an initial path according to the global starting point and the global end point;

[0060] The optimization path generation module is connected with the data acquisition module and the initial path generation module, and is used to optimize the initial path in real time based on the local path optimization algorithm to obtain the first optimized path, and at the same time, judge in real time whether the perception threshold of the current position of the drone is greater than the preset threshold; if the perception threshold is greater than the preset threshold, the boundary of the drone's field of view is perceived, and the perceived boundary is stored in the form of space points, recorded as boundary points, and the boundary points are distance-optimized to obtain a boundary space point set P; according to the boundary space point set P, the initial path is optimized in real time based on the path optimization method of boundary perception to obtain a second optimized path and a local end point; when the drone moves along the second optimized path, the distance between the current position of the drone and the local end point is judged in real time, and when the distance is less than the preset distance, it is switched to optimize the initial path in real time based on the local path optimization algorithm; otherwise, the local path optimization algorithm is continued to be used to optimize the initial path in real time until the global end point is reached;

[0061] The dynamic landing module is connected to the optimized path generation module and is used to perform dynamic landing based on the deep reinforcement learning algorithm when the drone reaches the global destination.

[0062] The embodiments of the present invention have the following technical effects:

[0063] The present invention determines whether the current UAV path optimization strategy is based on a local path optimization algorithm or a path optimization method based on boundary perception by calculating the size of the perception threshold and the preset threshold. Different path planning methods can be selected for different situations to avoid path redundancy or falling into a local optimal solution, thereby improving the efficiency of path optimization. By designing reward functions for the horizontal and vertical directions, a deep reinforcement learning algorithm is used to control the landing of the UAV, thereby increasing the updating efficiency of the neural network, improving the problems of small sample space and slow data processing, and realizing accurate dynamic landing of the UAV.

[0064] In addition, by inserting intermediate points during path optimization, it is possible to avoid the situation where the optimization trajectory deviates too much and causes no solution to the optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 is a flow chart of a method for autonomous environment perception, path planning and dynamic landing of a drone provided by an embodiment of the present invention;

[0067] Figure 2 It is a logic diagram of the autonomous environment perception, path planning and dynamic landing method of the UAV provided by the embodiment of the present invention;

[0068] Figure 3 It is a schematic diagram of a perception threshold less than or equal to a preset threshold provided by an embodiment of the present invention;

[0069] Figure 4 is a schematic diagram of a perception threshold greater than a preset threshold provided by an embodiment of the present invention;

[0070] Figure 5 It is a schematic diagram of obtaining a boundary space point set by sensing the boundary of the field of view of a drone provided by an embodiment of the present invention;

[0071] Figure 6 It is a logic diagram of dynamic landing based on deep reinforcement learning algorithm provided by an embodiment of the present invention;

[0072] Figure 7 is a schematic diagram of the flight action of the UAV in space provided by an embodiment of the present invention;

[0073] Figure 8 It is a structural schematic diagram of the autonomous environment perception, path planning and dynamic landing system of the UAV provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0075] The present invention proposes a method for autonomous environment perception, path planning and dynamic landing of a UAV. Figure 1is a flow chart of a method for autonomous environment perception, path planning and dynamic landing of a drone provided by an embodiment of the present invention. Figure 2 is a logic diagram of the autonomous environment perception, path planning and dynamic landing method of the UAV provided by the embodiment of the present invention, see Figure 1 and Figure 2 , specifically including:

[0076] S1. The drone obtains the depth information and two-dimensional plane information of the environment in real time, and generates three-dimensional environment information in space through a three-dimensional reconstruction algorithm based on the depth information and the two-dimensional plane information.

[0077] Specifically, the drone obtains the depth information and two-dimensional plane information of the environment in real time through a binocular depth camera, generates a three-dimensional environment in space through a three-dimensional reconstruction algorithm, and performs obstacle avoidance path planning based on the three-dimensional environment.

[0078] S2. Determine a global starting point and a global end point, and generate an initial path according to the global starting point and the global end point.

[0079] Specifically, since the current scene is an unknown scene, the global start point and the global end point are set arbitrarily, that is, the method is applicable to various unknown scenes and has strong adaptability to any unknown scene.

[0080] S3. Based on the local path optimization algorithm, the initial path is optimized in real time to obtain the first optimized path, and at the same time, it is determined in real time whether the perception threshold of the current position of the drone is greater than the preset threshold.

[0081] Specifically, Figure 3 is a schematic diagram of a perception threshold less than or equal to a preset threshold provided by an embodiment of the present invention, Figure 4 is a schematic diagram of a perception threshold greater than a preset threshold provided by an embodiment of the present invention, see Figure 3 and Figure 4 , the calculation formula of the perception threshold is as follows:

[0082] ;

[0083] Among them, τ per represents the perception threshold, represents the initial path within the UAV’s field of view, represents the first optimized path within the UAV’s field of view, and τ represents the preset threshold. If the perception threshold is greater than the preset threshold, that is, when τ per >τ, enter S4, otherwise continue to execute S3 until reaching the global end point and enter S7.

[0084] Specifically, the first optimized path is a collision-free path obtained based on the initial path through a local path optimization algorithm. The initial path is a straight line path from the global starting point to the global end point (it may pass through obstacles). Therefore, the length of the first optimized path must be greater than or equal to the length of the initial path. Then, when the volume of the obstacle is larger, in order to complete the obstacle avoidance task, the optimized path will also be longer. Therefore, when τ per >τ, it means that a larger obstacle is encountered. In order to avoid path redundancy or falling into a local optimal solution, the path optimization method based on boundary perception is switched to at this time.

[0085] S4. Sense the boundary of the drone's field of view, store the sensed boundary in the form of spatial points, record them as boundary points, perform distance optimization on the boundary points, and obtain a boundary spatial point set P.

[0086] Specifically, Figure 5 FIG. 1 is a schematic diagram of obtaining a boundary space point set by sensing the boundary of the field of view of a drone according to an embodiment of the present invention, see Figure 5 , perceive the boundary of the drone's field of view. The area within the drone's field of view is a known area, and the area outside the drone's field of view is an unknown area. The boundary between the known area and the unknown area is the boundary. The perceived boundary is stored in the form of a spatial point, recorded as a boundary point. With the boundary point as the center point, a cube area with a side length of 2m is constructed, and the cube area is divided into 8 sub-cube areas with a side length of m. The vertex of each sub-cube area is used as a collision judgment point. There are 26 collision judgment points around a boundary point. Determine whether the distance between each collision judgment point and the target obstacle is less than the preset collision distance. If there is at least one collision judgment point whose distance with the target obstacle is less than the preset collision distance, it means that when the drone reaches the position of the boundary point, it may collide with the obstacle, so delete the boundary point. Repeat the above steps, traverse all boundary points, and form a boundary space point set P based on the boundary points finally retained. Among them, Figure 5 The green points in the figure are the boundary points that constitute the boundary space point set P, and the red points are the boundary points that may collide with obstacles and therefore need to be deleted.

[0087] S5. According to the boundary space point set P, the initial path is optimized in real time based on the boundary-aware path optimization method to obtain a second optimized path and a local end point.

[0088] S51. Determine a local endpoint.

[0089] Specifically, according to the UAV's field of view, the point farthest from the current position of the UAV on the initial path is selected as the initial local end point. If the initial local end point is within the obstacle range, the second local end point is selected by doubling the UAV's field of view distance on the basis of the initial local end point, until the selected local end point is outside the obstacle range. It should be noted that the selected local end points are all on the initial path.

[0090] S52. Determine a decision indicator, and calculate a decision function based on the decision indicator.

[0091] Among them, the decision indicators include: adjacent point constraints, distance constraints, and direction constraints.

[0092] Specifically, the adjacent point constraint expression is as follows:

[0093] ;

[0094] ;

[0095] Among them, d A represents the distance between adjacent points, f A represents the neighboring point cost function, p i represents the i-th boundary point in the boundary space point set P, p c Indicates the current position of the drone. represents the Euclidean distance operation, d min Indicates the minimum value of the selected point range within the UAV’s field of view, d max Indicates the maximum value of the selected point range within the drone's field of view.

[0096] The distance constraint expression is as follows:

[0097] ;

[0098] ;

[0099] Among them, d L Indicates the current position of the drone p c With local endpoint p g The distance between L represents the distance cost function, p g represents the local endpoint, d Lmin Represents all boundary points in the boundary space point set P to the local end point p g The minimum distance, d Lmax Represents all boundary points in the boundary space point set P to the local end point p g The maximum value of the distance.

[0100] The direction constraint expression is as follows:

[0101] ;

[0102] ;

[0103] Among them, d D Indicates the specific value of the deviation from the current speed direction, f D represents the direction cost function, v c Indicates the current speed of the drone, d Dmin Indicates the minimum value of deviation from the current speed direction, d Dmax Indicates the maximum deviation from the current speed direction.

[0104] The calculation formula of the decision function is as follows:

[0105] ;

[0106] Among them, λ A , L , D They represent the weight of the adjacent point constraint, the weight of the distance constraint, and the weight of the direction constraint, respectively. b Indicates the optimal replanning point.

[0107] By minimizing the decision function, the best replanning point p can be selected b .

[0108] S53, according to the current position of the drone c and the optimal replanning point p b A second optimized path is generated.

[0109] Furthermore, when generating the current position p of the drone c and replanning point p b When the path between b It is constantly changing, and there may be two replanning points p that appear one after the other. b The difference in distance is large, resulting in the situation where the optimized path has no solution. Therefore, in the process of optimizing the initial path to the optimized path, it is necessary to c and replanning point p b An intermediate point is added in between so that the final optimization trajectory will not deviate too much and cause an optimization problem with no solution.

[0110] Furthermore, when the initial path is optimized in real time based on the local path optimization algorithm to obtain the first optimized path, there is also a local end point. The local end point in the local path optimization algorithm is the point farthest from the current position of the drone within the field of view of the drone.

[0111] Therefore, when optimizing the initial path in real time based on the local path optimization algorithm to obtain the first optimized path, or optimizing the initial path in real time based on the boundary-aware path optimization method to obtain the second optimized path, it also includes:

[0112] According to the current position of the UAV and the local end point, the intermediate point is inserted to generate the first optimized path or the second optimized path;

[0113] The midpoint is calculated as follows:

[0114] ;

[0115] Among them, p m Indicates the middle point, a max Indicates the maximum angular velocity of the drone.

[0116] S6. When the UAV advances along the second optimized path to the local end point, it switches to optimizing the initial path in real time based on the local path optimization algorithm.

[0117] S61: When the UAV moves along the second optimized path, S5 is re-executed every time a preset time threshold or a preset distance threshold is passed, and the optimal re-planning point p is updated. b and the second optimization path.

[0118] For example, the preset time threshold may be set to 0.5s, and the preset distance threshold may be set to 1m. That is, when the UAV moves along the second optimized path, the optimal replanning point p is updated every time the running exceeds 0.5s or the running distance exceeds 1m. b and the second optimization path.

[0119] S62: Record the path length L that the UAV flies along the second optimized path from the current position. thresh , when the path length L thresh When it is greater than the preset path length, reselect the local end point p g , according to the new local endpoint, re-execute S5 and update the best replanning point p b and the second optimization path.

[0120] Among them, reselect the local end point p g Specifically: based on the current local endpoint, double the drone's field of view distance to select a new local endpoint.

[0121] For example, the preset path length can be set to 5 times the field of view distance of the drone. thresh If the drone's field of view distance is greater than 5 times and still does not reach the local endpoint, it means that the currently selected local endpoint is inappropriate, which may cause the path to deviate and the local endpoint needs to be reselected.

[0122] S63, when the drone reaches the local endpoint p g When the local path optimization algorithm is used, the initial path is optimized in real time.

[0123] Specifically, when the UAV reaches the local endpoint p g When , it indicates that the UAV has bypassed a large obstacle. At this time, switching to the local path optimization algorithm to optimize the initial path in real time can further improve the efficiency of path optimization.

[0124] S7. When the drone reaches the global destination, it performs dynamic landing based on the deep reinforcement learning algorithm.

[0125] Specifically, Figure 6 is a logic diagram of dynamic landing based on deep reinforcement learning algorithm provided by an embodiment of the present invention, see Figure 6 When the drone reaches the global destination and completes the mission, it needs to land on the ground or on a mobile carrier. At this time, the dynamic landing function is needed.

[0126] S71. Construct a landing control agent based on the deep deterministic policy gradient algorithm.

[0127] Specifically, first, based on the theory of stochastic processes, the landing control problem of the UAV is transformed into a Markov decision process. Secondly, the reward function that considers the horizontal and vertical control processes of the UAV is designed, and the landing control problem is transferred to the reinforcement learning framework. Then, the deep deterministic policy gradient (DDPG) algorithm is used to solve the reinforcement learning problem, and the landing control agent is obtained through a large amount of training. The DDPG algorithm is a method that combines neural networks and Q-learning. It integrates the function approximation and representation capabilities of deep learning (DL) into reinforcement learning (RL). This algorithm is very similar to Q-learning. On the basis of Q-learning, a target Q value network is added, and an experience replay mechanism is adopted.

[0128] The landing control agent includes a horizontal module, a horizontal agent, a vertical module, and a vertical agent. The horizontal module and the vertical module are used to calculate the reward function based on the observation values ​​of the drone and the landing platform, and whether the drone meets the landing conditions; the horizontal agent and the vertical agent are used to control the drone based on the observation values ​​of the drone and the landing platform, the reward function, and whether the drone meets the landing conditions.

[0129] Figure 5The delayed memory in represents the state of the drone at the last moment (i.e., the state of the drone after the landing control agent controlled the drone at the last moment), and the current observation value of the drone is determined based on the state of the drone at the last moment.

[0130] S72. Obtain observation values ​​of the UAV and the landing platform.

[0131] Specifically, the observation values ​​of the UAV and the landing platform are obtained, including: the position of the UAV, the position of the landing platform, the speed of the UAV, the flight action of the UAV, etc.; wherein the landing platform may include a fixed platform, a small car, etc.

[0132] S73. Define a reward function.

[0133] Among them, the reward function includes a horizontal reward function and a vertical reward function.

[0134] Furthermore, Figure 7 is a schematic diagram of the flight action of the UAV in space provided by an embodiment of the present invention, see Figure 7 , the optional flight actions of the designed drone in space include: forward movement A1, backward movement A2, left movement A3, right movement A4, upward movement A5, downward movement A6 and hovering A7, a total of 7 types. In the ground coordinate system, forward movement means moving along the positive direction of the X axis, and backward movement is the opposite; left movement means moving along the positive direction of the Y axis, and right movement is the opposite; upward movement means moving along the positive direction of the Z axis, and downward movement is the opposite; hovering means that the drone is stationary in the air.

[0135] Specifically, the horizontal reward function r level Includes the positive reward r for the drone to reach the landing condition horizontally l1 , Flying Action Reward l2 , Reward Shaping l3 ;

[0136] For example, the positive reward r for the drone to reach the landing condition horizontally is l1 The expression is as follows:

[0137] ;

[0138] in, Represents the horizontal distance error between the drone and the landing platform, Represents the horizontal relative speed between the drone and the landing platform. The above formula indicates: first determine whether the horizontal distance error between the drone and the landing platform is within 1m, if it is satisfied, reward +1, then determine whether the horizontal relative speed between the drone and the landing platform is within 0.4m / s, if it is satisfied, reward +1 again, otherwise no reward is given; wherein, the threshold of the horizontal distance error and the threshold of the horizontal relative speed can be set according to the actual situation, and are not limited to 1m and 0.4m / s mentioned in this embodiment.

[0139] Flying Action Reward l2 The expression is as follows:

[0140] ;

[0141] Among them, θ represents the angle between the projection of the drone on the horizontal plane XOY and the X-axis, a t Represents the flight action of the drone. The above formula means: if the drone moves close to the landing platform, a positive reward is given, otherwise a negative reward is given.

[0142] Reward Shaping l3 The expression is as follows:

[0143] ;

[0144] Among them, z q,t Indicates the vertical distance of the drone on the horizontal plane XOY. The above formula indicates: first determine whether the horizontal distance error between the drone and the landing platform is within 0.5m, then determine whether the height of the drone is within 0.2m. If it is satisfied, a reward is given, otherwise no reward is given; wherein, the threshold of the horizontal distance error and the height threshold of the drone can be set according to the actual situation, and are not limited to 0.5m and 0.2m mentioned in this embodiment.

[0145] Horizontal reward function r level The expression is as follows:

[0146] r level =r l1 +r l2 +r l3 .

[0147] Specifically, the vertical reward function r vertical The comprehensive discrimination reward r including the drone height and action command v1 , hover action reward r v2 ;

[0148] For example, the comprehensive discrimination reward r of the drone height and action command is v1 The expression is as follows:

[0149] ;

[0150] The above formula means: first determine whether the height of the drone is greater than 0m, then determine whether the drone is descending. If the condition is met, a positive reward is given, otherwise a negative reward is given.

[0151] Hover Action Reward v2 The expression is as follows:

[0152]

[0153] The above formula indicates that if the height of the drone is less than 0.2m or the drone is not in a hovering state, no reward is given; if the height of the drone is greater than or equal to 0.2m and the drone is in a hovering state, a negative reward is given; the height threshold of the drone can be set according to actual conditions and is not limited to 0.2m mentioned in this embodiment.

[0154] Vertical reward function r vertical The expression is as follows:

[0155] r vertical =r v1 +r v2 .

[0156] S74. Determine whether the UAV meets the landing conditions based on the observation values ​​of the UAV and the landing platform and the reward function.

[0157] S75. The landing control agent controls the UAV according to the observation values ​​of the UAV and the landing platform, the reward function, and whether the UAV meets the landing conditions until the UAV completes the dynamic landing.

[0158] Specifically, when the horizontal reward function r level And the vertical reward function r vertical When all converge, the drone lands successfully. level And the vertical reward function r vertical They are stabilized at a constant value, which means the horizontal reward function r level And the vertical reward function r vertical convergence.

[0159] The present invention determines whether the current UAV path optimization strategy is based on a local path optimization algorithm or a path optimization method based on boundary perception by calculating the size of the perception threshold and the preset threshold. Different path planning methods can be selected for different situations to avoid path redundancy or falling into a local optimal solution, thereby improving the efficiency of path optimization. By designing reward functions for the horizontal and vertical directions, a deep reinforcement learning algorithm is used to control the landing of the UAV, thereby increasing the updating efficiency of the neural network, improving the problems of small sample space and slow data processing, and realizing accurate dynamic landing of the UAV.

[0160] In addition, by inserting intermediate points during path optimization, it is possible to avoid the situation where the optimization trajectory deviates too much and causes no solution to the optimization.

[0161] Figure 8 is a schematic diagram of the structure of the UAV autonomous environment perception, path planning and dynamic landing system provided by an embodiment of the present invention, which is used to execute the UAV autonomous environment perception, path planning and dynamic landing method described in the above embodiment, such as Figure 8 As shown, the system includes the following modules:

[0162] A data acquisition module is used to acquire the depth information and two-dimensional plane information of the environment in real time through the UAV, and generate three-dimensional environment information in space through a three-dimensional reconstruction algorithm based on the depth information and the two-dimensional plane information;

[0163] An initial path generation module, connected to the data acquisition module, is used to determine a global starting point and a global end point, and generate an initial path according to the global starting point and the global end point;

[0164] The optimization path generation module is connected with the data acquisition module and the initial path generation module, and is used to optimize the initial path in real time based on the local path optimization algorithm to obtain the first optimized path, and at the same time, judge in real time whether the perception threshold of the current position of the drone is greater than the preset threshold; if the perception threshold is greater than the preset threshold, the boundary of the drone's field of view is perceived, and the perceived boundary is stored in the form of space points, recorded as boundary points, and the boundary points are distance-optimized to obtain a boundary space point set P; according to the boundary space point set P, the initial path is optimized in real time based on the path optimization method of boundary perception to obtain a second optimized path and a local end point; when the drone moves along the second optimized path, the distance between the current position of the drone and the local end point is judged in real time, and when the distance is less than the preset distance, it is switched to optimize the initial path in real time based on the local path optimization algorithm; otherwise, the local path optimization algorithm is continued to be used to optimize the initial path in real time until the global end point is reached;

[0165] The dynamic landing module is connected to the optimized path generation module and is used to perform dynamic landing based on the deep reinforcement learning algorithm when the drone reaches the global destination.

[0166] It should be noted that the terms used in the present invention are only for describing specific embodiments, rather than limiting the scope of the present application. As shown in the present specification, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular, but may also include the plural. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of more restrictions, the elements defined by the sentence "include one..." do not exclude the presence of other identical elements in the process, method or device including the elements.

[0167] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. Autonomous environment perception, path planning and dynamic landing methods of drones, It is characterized in that The steps include: S1. The drone obtains depth information and two-dimensional plane information of the environment in real time, and generates three-dimensional environment information in space through a three-dimensional reconstruction algorithm according to the depth information and the two-dimensional plane information; S2, determining a global starting point and a global end point, and generating an initial path according to the global starting point and the global end point; S3, optimizing the initial path in real time based on the local path optimization algorithm to obtain a first optimized path, and determining in real time whether the perception threshold of the current position of the drone is greater than a preset threshold; if the perception threshold is greater than the preset threshold, entering S4, otherwise continuing to execute S3 until reaching the global end point, entering S7; The calculation formula of the perception threshold is as follows: ; Among them, τ per represents the perception threshold, represents the initial path within the UAV’s field of view, represents the first optimized path within the UAV’s field of view, τ represents the preset threshold, and when τ per >τ, enter S4; S4, sensing the boundary of the drone's field of view, and storing the sensed boundary in the form of spatial points, recorded as boundary points, performing distance optimization processing on the boundary points, and obtaining a boundary spatial point set P; S5. According to the boundary space point set P, the initial path is optimized in real time based on a boundary-aware path optimization method to obtain a second optimized path and a local end point; S6. When the UAV advances along the second optimized path to the local end point, switching to optimizing the initial path in real time based on the local path optimization algorithm; S7. When the UAV reaches the global endpoint, it performs dynamic landing based on a deep reinforcement learning algorithm.

2. The method for autonomous environment perception, path planning and dynamic landing of a drone according to claim 1, It is characterized in that In S4, the boundary points are subjected to distance optimization processing to obtain a boundary space point set P including: Taking the boundary point as the center point, construct a cube area with a side length of 2m, divide the cube area into 8 sub-cube areas with a side length of m, use the vertex of each sub-cube area as a collision judgment point, and judge whether the distance between each collision judgment point and the target obstacle is less than the preset collision distance; if there is at least one collision judgment point with a distance less than the preset collision distance between the target obstacle, delete the boundary point; Repeat the above steps to traverse all boundary points, and construct a boundary space point set P based on the boundary points finally retained.

3. The method for autonomous environment perception, path planning and dynamic landing of a UAV according to claim 2, It is characterized in that In S5, according to the boundary space point set P, the initial path is optimized in real time based on the boundary-aware path optimization method to obtain the second optimized path and the local end point, including: S51, determining a local endpoint; According to the field of vision of the drone, the point on the initial path that is farthest from the current position of the drone is selected as the initial local end point. If the initial local end point is within the obstacle range, the field of vision of the drone is doubled based on the initial local end point to select the second local end point until the selected local end point is outside the obstacle range. S52, determining a decision indicator, and calculating a decision function according to the decision indicator; wherein the decision indicator includes: adjacent point constraint, distance constraint, and direction constraint; The adjacent point constraint expression is as follows: ; ; Among them, d A represents the distance between adjacent points, f A represents the neighboring point cost function, p i represents the i-th boundary point in the boundary space point set P, p c Indicates the current position of the drone. represents the Euclidean distance operation, d min Indicates the minimum value of the selected point range within the UAV’s field of view, d max Indicates the maximum value of the selected point range within the drone's field of view; The distance constraint expression is as follows: ; ; Among them, d L Indicates the current position of the drone p c With local endpoint p g The distance between L represents the distance cost function, p g represents the local endpoint, d Lmin Represents all boundary points in the boundary space point set P to the local end point p g The minimum distance, d Lmax Represents all boundary points in the boundary space point set P to the local end point p g The maximum value of the distance; The direction constraint expression is as follows: ; ; Among them, d D Indicates the specific value of the deviation from the current speed direction, f D represents the direction cost function, v c Indicates the current speed of the drone, d Dmin Indicates the minimum value of deviation from the current speed direction, d Dmax Indicates the maximum value of deviation from the current speed direction; The calculation formula of the decision function is as follows: ; Among them, λ A , L , D They represent the weight of the adjacent point constraint, the weight of the distance constraint, and the weight of the direction constraint, respectively. b represents the optimal replanning point; S53, according to the current position of the drone c and the optimal replanning point p b The second optimized path is generated.

4. The method for autonomous environment perception, path planning and dynamic landing of a UAV according to claim 3, It is characterized in that In S6, when the drone advances along the second optimized path to the local end point, switching to optimizing the initial path in real time based on the local path optimization algorithm includes: S61: When the UAV moves along the second optimized path, S5 is re-executed every time a preset time threshold or a preset distance threshold is passed, and the optimal re-planning point p is updated. b and a second optimization path; S62: Record the path length L that the UAV flies along the second optimized path from the current position. thresh , when the path length L thresh When it is greater than the preset path length, reselect the local end point p g , according to the new local endpoint, re-execute S5 and update the best replanning point p b and a second optimization path; Among them, reselect the local end point p g Specifically: Based on the current local endpoint, double the drone's field of view to select a new local endpoint; S63, when the UAV reaches the local end point p g , switching to optimize the initial path in real time based on the local path optimization algorithm.

5. The method for autonomous environment perception, path planning and dynamic landing of a drone according to claim 4, It is characterized in that When the initial path is optimized in real time based on a local path optimization algorithm to obtain a first optimized path, or when the initial path is optimized in real time based on a boundary-aware path optimization method to obtain a second optimized path, the method further includes: Inserting intermediate points according to the current position of the UAV and the local end point to generate the first optimized path or the second optimized path; The midpoint is calculated as follows: ; Among them, p m Indicates the middle point, a max Indicates the maximum angular velocity of the drone.

6. The method for autonomous environment perception, path planning and dynamic landing of a UAV according to claim 1, It is characterized in that In S7, when the drone reaches the global endpoint, dynamic landing based on the deep reinforcement learning algorithm includes: S71. Construct a landing control agent based on a deep deterministic policy gradient algorithm. S72, obtaining observation values ​​of the UAV and the landing platform; S73, defining a reward function; wherein the reward function includes a horizontal direction reward function and a vertical direction reward function; S74, judging whether the drone meets the landing conditions according to the observation values ​​of the drone and the landing platform and the reward function; S75. The landing control agent controls the drone according to the observation values ​​of the drone and the landing platform, the reward function, and whether the drone meets the landing conditions until the drone completes the dynamic landing.

7. An autonomous environment perception, path planning and dynamic landing system for unmanned aerial vehicles, used to implement the autonomous environment perception, path planning and dynamic landing method for unmanned aerial vehicles as described in any one of claims 1 to 6 above, It is characterized in that The system includes the following modules: A data acquisition module is used to acquire depth information and two-dimensional plane information of the environment in real time through the drone, and generate three-dimensional environment information in space through a three-dimensional reconstruction algorithm according to the depth information and the two-dimensional plane information; An initial path generation module, connected to the data acquisition module, for determining a global starting point and a global end point, and generating an initial path according to the global starting point and the global end point; The optimized path generation module is connected to the data acquisition module and the initial path generation module, and is used to optimize the initial path in real time based on the local path optimization algorithm to obtain a first optimized path, and at the same time, determine in real time whether the perception threshold of the current position of the drone is greater than a preset threshold; the calculation formula of the perception threshold is as follows: ; Among them, τ per represents the perception threshold, represents the initial path within the UAV’s field of view, represents the first optimized path within the field of view of the drone, and τ represents a preset threshold value; if the perception threshold value is greater than the preset threshold value, the boundary of the field of view of the drone is perceived, and the perceived boundary is stored in the form of a spatial point, recorded as a boundary point, and the boundary point is distance-optimized to obtain a boundary space point set P; according to the boundary space point set P, the initial path is optimized in real time based on the path optimization method based on boundary perception to obtain a second optimized path and a local end point; when the drone moves along the second optimized path, the distance between the current position of the drone and the local end point is determined in real time, and when the distance is less than the preset distance, the initial path is switched to be optimized in real time based on the local path optimization algorithm; otherwise, the initial path is optimized in real time based on the local path optimization algorithm until the global end point is reached; The dynamic landing module is connected to the optimized path generation module and is used to perform dynamic landing based on a deep reinforcement learning algorithm when the drone reaches the global endpoint.