Unmanned aerial vehicle obstacle avoidance control method and system based on deep reinforcement learning

Abstract

This application discloses a drone obstacle avoidance control method and system based on deep reinforcement learning, relating to the field of drone flight control technology. The method includes: collecting environmental images, and if airflow generators are present in the environmental images, setting an obstacle area; planning a detour path and calculating the detour time required for the detour path; planning a temporary path during the exploration phase; collecting environmental data, and if there are dangerous areas, using the DDPG algorithm to generate key parameters for the exploration path and calculate the exploration time required for the exploration path; if the detour time is less than or equal to the exploration time, the detour path is designated as a low-time path; if the detour time is greater than the exploration time, the exploration path is designated as a low-time path. This application combines the curiosity mechanism with deep reinforcement learning technology to effectively improve the accuracy and efficiency of drone obstacle avoidance when passing through airflow.

Description

Technical Field

[0001] The present application relates to the field of drone flight control technology, and in particular to a drone obstacle avoidance control method and system based on deep reinforcement learning. Background Art

[0002] Drones are currently common electronic devices in both civilian and industrial production. They can perform various agricultural and industrial production tasks at low altitudes, such as sowing and fertilizing crops and inspecting industrial areas. With their low cost, ease of use, and excellent performance, drones have become an indispensable device in daily life and production.

[0003] Regardless of the mission, drones must fly to their destination. During flight, due to their low altitude, they often encounter various obstacles. Drones use various sensors, such as cameras and radar, to detect these obstacles and perform corresponding avoidance maneuvers. Currently, the vast majority of obstacles are visible in visible light, such as indoor pipes, doors, and windows. Camera-generated images alone can achieve relatively accurate and timely obstacle avoidance. However, in addition to these visible targets, some obstacles are invisible in visible light, such as the airflow generated by the inlets and outlets of airflow generators like air conditioners and fans. These invisible obstacles are obviously not captured by cameras. While these obstacles can be detected using other means, such as infrared cameras and millimeter-wave radar, airflow, unlike other obstacles, lacks clearly defined boundaries, making existing obstacle avoidance methods ineffective against them. Summary of the Invention

[0004] The embodiments of the present application provide a drone obstacle avoidance control method and system based on deep reinforcement learning, which is used to solve the problem that the obstacle avoidance methods for visible obstacles in the prior art are not suitable for invisible obstacles with unclear boundaries such as airflow.

[0005] On the one hand, the embodiments of the present application provide a drone obstacle avoidance control method based on deep reinforcement learning, including:

[0006] Get flight routes;

[0007] During the flight along the flight route, real-time environmental images are collected to identify whether there is an airflow generator in the environmental image. If so, a set area on the entrance and exit of the airflow generator is set as an obstacle area;

[0008] Extract multiple waypoints from the flight route, and use the two waypoints that are outside the obstacle area and closest to the obstacle area as the route start and end points, respectively. Plan a detour path between the route start and end points, and calculate the detour time required for the detour path. The detour path bypasses the obstacle area.

[0009] Determine the distance between the drone and the obstacle area. If the distance is less than or equal to the distance threshold, enter the exploration phase. In the exploration phase, exploration points are generated around the obstacle area and a temporary path is planned for the drone to move from its current position to the exploration points.

[0010] Move along the temporary path to the exploration point, collect environmental data at the current location, and determine whether there is a dangerous area based on the environmental data;

[0011] If there is a dangerous area, the DDPG (Deep Deterministic Policy Gradient) algorithm is used to generate the key parameters of the exploration path, calculate the exploration time required for the exploration path, and ensure that the end point of the exploration path coincides with the end point of the route;

[0012] Compare the detour time and exploration time. If the detour time is less than or equal to the exploration time, select the detour path as the low-time path. If the detour time is greater than the exploration time, select the exploration path as the low-time path.

[0013] Move along low-time paths to avoid obstruction areas;

[0014] In the process of using the DDPG algorithm to generate the key parameters of the exploration path, the range of the obstacle area is corrected according to the dangerous area to obtain the current range. The corresponding intrinsic reward is determined based on the difference between the current range and the historical range in the experience pool. The policy network is used to generate the action parameters of the drone. The value network is used to combine the extrinsic rewards and intrinsic rewards of the action parameters to evaluate the action parameters, and the action parameters with the largest value are used as the key parameters of the exploration path.

[0015] In a possible implementation, a target recognition model based on a CNN (convolutional neural network) is used to identify whether an airflow generator exists in an environment image.

[0016] In a possible implementation, after the obstacle area is set, the shape and size of the obstacle area are modified according to the environmental data.

[0017] In one possible implementation, during the exploration phase, a caution area is set outside the obstacle area with a distance threshold as its width, the exploration points and the temporary path are all located in the caution area, and multiple exploration points are at the same distance from the inner edge of the caution area.

[0018] In a possible implementation, the environmental data includes real-time wind speed. If the real-time wind speed exceeds a wind speed threshold, it is considered that a dangerous area exists.

[0019] In one possible implementation, the environmental data also includes an IMU (Inertial Measurement Unit) attitude angle, which is calculated according to the following formula during exploration:

[0020]

[0021] in, For exploration time, To explore the length of the path, is the maximum flight speed of the UAV, is the airflow disturbance coefficient, which is determined according to the IMU attitude angle.

[0022] In one possible implementation, while collecting environmental data, the noise of the environmental data is also analyzed. If the noise of the environmental data is greater than a noise threshold, the exploration path is selected only when the detour time is greater than the product of the exploration time and the safety factor.

[0023] In one possible implementation, after selecting a detour path or an exploration path, the detour path or the exploration path is bound to the position of the airflow generator. When the drone passes by the airflow generator again, the selected detour path or the exploration path is directly adopted.

[0024] On the other hand, an embodiment of the present application also provides a drone obstacle avoidance control system based on deep reinforcement learning, the system comprising:

[0025] Route acquisition module, used to obtain the flight route;

[0026] An area setting module is used to collect real-time environmental images during flight along the flight route, identify whether there is an airflow generator in the environmental image, and if so, set a set area on the airflow generator entrance and exit as an obstacle area;

[0027] A first path planning module is configured to extract multiple waypoints from the flight route, select the two waypoints that are outside the obstacle area and closest to the obstacle area as the route start point and route end point, respectively, plan a detour path between the route start point and the route end point, calculate the detour time required for the detour path, and bypass the obstacle area.

[0028] The second path planning module is used to determine the distance between the UAV and the obstacle area. If the distance is less than or equal to the distance threshold, the exploration phase begins. In the exploration phase, exploration points are generated around the obstacle area and a temporary path is planned for the UAV to move from the current position to the exploration points.

[0029] The hazard identification module is used to move to the exploration point along the temporary path, collect environmental data at the current location, and determine whether there is a dangerous area based on the environmental data;

[0030] The third path planning module is used to generate key parameters of the exploration path using the DDPG algorithm if there is a dangerous area, calculate the exploration time required for the exploration path, and ensure that the end point of the exploration path coincides with the end point of the route;

[0031] The path selection module is used to compare the detour time and the exploration time. If the detour time is less than or equal to the exploration time, the detour path is selected as the low-time path. If the detour time is greater than the exploration time, the exploration path is selected as the low-time path.

[0032] A movement control module for moving along a low-time path to avoid obstacle areas;

[0033] In the process of generating key parameters of the exploration path using the DDPG algorithm, the third path planning module corrects the range of the obstacle area according to the dangerous area to obtain the current range. The corresponding intrinsic reward is determined based on the difference between the current range and the historical range in the experience pool. The policy network is used to generate the action parameters of the drone. The value network is used to combine the external rewards and intrinsic rewards of the action parameters to evaluate the action parameters, and the action parameters with the largest value are used as the key parameters of the exploration path.

[0034] In one possible implementation, the third path planning module is deployed on an edge computing device.

[0035] The drone obstacle avoidance control method and system based on deep reinforcement learning in this application have the following advantages:

[0036] Combining the curiosity mechanism with deep reinforcement learning technology effectively improves the accuracy and efficiency of drones' obstacle avoidance when passing through air currents. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0038] Figure 1 Flowchart of the drone obstacle avoidance control method based on deep reinforcement learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0040] Figure 1 Flowchart of a drone obstacle avoidance control method based on deep reinforcement learning provided in an embodiment of the present application. This embodiment of the present application provides a drone obstacle avoidance control method based on deep reinforcement learning, including:

[0041] S100, obtaining a flight route.

[0042] S110 , collecting real-time environmental images during the flight along the flight route, identifying whether there is an airflow generator in the environmental image, and if so, setting a set area on the entrance and exit of the airflow generator as an obstacle area.

[0043] For example, the drone is preferably a multi-rotor drone, which facilitates hovering at a desired location and rapid directional adjustment. The drone is integrated with a camera, millimeter-wave radar, and IMU sensor, which are used to collect environmental images, real-time wind speed, and IMU attitude angle, respectively.

[0044] Furthermore, a CNN-based target recognition model is used to identify whether there are airflow generators in the environmental image. When a drone flies along a branching route, if an airflow generator appears in front of it and is located near the vertical center axis of the environmental image, the drone is likely to pass through the entrance and exit of the airflow generator. In this case, not only is it necessary to identify the airflow generator, but it is also necessary to continuously collect and analyze the environmental image to determine whether the airflow generator is still located near the vertical center axis of the environmental image when the distance between the drone and the airflow generator reaches a certain value. If so, it is considered that the drone will enter the obstacle area if it continues to fly. Due to the large amount of airflow at the entrance and exit of the airflow generator, if the drone enters, it will cause an accident due to the sudden change in wind speed. Therefore, the area where the airflow is located will be considered an obstacle area.

[0045] Furthermore, after the obstacle area is set, the shape and size of the obstacle area are modified according to the environmental data.

[0046] Specifically, when there is no wind, the obstacle zone can be roughly considered a cylindrical area with the same shape and size as the airflow generator's inlet and outlet. This cylindrical area is relatively high, and the drone cannot pass over it. However, in reality, the airflow generator's environment will have some wind. When there is wind, the obstacle zone will no longer be a cylindrical area, and its shape will change. The degree of change is directly related to the wind speed, with the greater the wind speed, the greater the change. Therefore, it is necessary to use the real-time wind speed in the environmental data to correct the shape and size of the obstacle zone to make the corrected obstacle zone more consistent with the actual situation.

[0047] S120: Extract multiple waypoints of the flight route, use the two waypoints that are outside the obstacle area and closest to the obstacle area as the route starting point and route end point, respectively, plan a detour path between the route starting point and the route end point, calculate the detour time required for the detour path, and bypass the obstacle area from the outside.

[0048] For example, before planning a detour path, the obstacle area needs to be expanded. In the embodiment of the present application, the safety width outside the obstacle area is expanded to, for example, 5 meters, to form a safety area surrounding the obstacle area. The detour path can then be planned in this safety area, so that the drone will neither enter the obstacle area nor make the detour path too long.

[0049] Specifically, when planning a detour path, since the route starting point, route end point and obstacle area are all known, the A* algorithm or Dubins curve can be used to generate a detour path within the safe area.

[0050] S130, determining the distance between the UAV and the obstacle area. If the distance is less than or equal to the distance threshold, entering the exploration phase, in which exploration points are generated around the obstacle area, and a temporary path is planned for the UAV to move from the current position to the exploration points.

[0051] Exemplarily, in the exploration phase, a cautious area with a distance threshold as width is set outside the obstacle area, the exploration points and the temporary path are all located in the cautious area, and the distances between multiple exploration points and the inner edge of the cautious area are the same.

[0052] Millimeter-wave radar can be used to directly detect the distance between the airflow generator and the drone, using this distance as the distance between the drone and the obstacle zone. A caution zone can be set to a certain width outside the obstacle zone, such as a 2-meter surround. Within this caution zone, the ambient wind speed may already be affected by the airflow, and because the airflow range can change slightly at any time, the drone needs to enter this area for further exploration to determine the exact extent of the obstacle zone.

[0053] When generating exploration points, five exploration points are evenly spaced within a range of ±30° horizontally and ±10 meters vertically, centered on the drone's current position. After planning a temporary path, the temporary time required for the drone to move from its current position to each exploration point is calculated.

[0054] Furthermore, Dijkstra or Rapidly Expanding Random Tree (RRT) can be used to generate a temporary path.

[0055] S140 , moving to the exploration point along the temporary path, collecting environmental data of the current location, and determining whether there is a dangerous area based on the environmental data.

[0056] For example, the drone needs to move from the current position to the first exploration point. After completing the collection of environmental data at the first exploration point, the drone will return to the current position and then move to the second exploration point, and repeat this process until the environmental data of all exploration points are collected.

[0057] In an embodiment of the present application, the environmental data includes real-time wind speed. If the real-time wind speed exceeds a wind speed threshold, it is considered that a dangerous area exists.

[0058] Specifically, the wind speed thresholds are 40°C and 8m / s. Since the embodiment of the present application collects environmental data at multiple exploration points, as long as the environmental data at any exploration point exceeds the corresponding threshold, the location of the exploration point is considered to be the edge of the danger zone. After determining whether there is a danger zone for each exploration point, the exploration point with a danger zone is merged into the obstacle area. At the same time, the areas directly above and below the exploration point are also considered to be danger zones and merged into the obstacle area, forming a more accurate obstacle area in the current state.

[0059] S150: If there is a dangerous area, the DDPG algorithm is used to generate key parameters of the exploration path, and the exploration time required for the exploration path is calculated. The end point of the exploration path coincides with the end point of the route.

[0060] For example, the environmental data also includes the IMU attitude angle, which is calculated according to the following formula during exploration:

[0061]

[0062] in, For exploration time, To explore the length of the path, is the maximum flight speed of the UAV, is the airflow disturbance coefficient, which is determined according to the IMU attitude angle.

[0063] Specifically, the IMU attitude angle is directly proportional to the airflow disturbance coefficient, and a corresponding relationship can be formed according to a preset range of the IMU attitude angle and the airflow disturbance coefficient, for example, 0 to 0.5. According to this corresponding relationship, each IMU attitude angle will correspond to an airflow disturbance coefficient.

[0064] Furthermore, the detour time t _ bypass The length of the detour path can be L r_bypass and the maximum flight speed of the drone v _ max Direct confirmation.

[0065] In the process of using the DDPG algorithm to generate the key parameters of the exploration path, the range of the obstacle area is corrected according to the dangerous area to obtain the current range. The corresponding intrinsic reward is determined based on the difference between the current range and the historical range in the experience pool. The policy network is used to generate the action parameters of the drone. The value network is used to combine the extrinsic rewards and intrinsic rewards of the action parameters to evaluate the action parameters, and the action parameters with the largest value are used as the key parameters of the exploration path.

[0066] The current range is the area obtained by merging the danger area at each exploration point with the obstacle area. Each time the current range is generated, it will be placed in the experience pool D for subsequent use.

[0067] In the above process of using the DDPG algorithm to generate key parameters for the exploration path, the intrinsic reward will first be determined based on the curiosity mechanism. The curiosity mechanism will first measure the novelty of the current range and the historical range in the experience pool. The novelty is determined based on the following formula:

[0068]

[0069] in, For the current range s t The novelty, is the number of historical ranges in the experience pool D, s i For the i A historical range, and are the current ranges extracted by the CNN networks t Hedi i Historical range s i The characteristic data, is the L2 norm, is the standard deviation of wind speed.

[0070] Based on the novelty above, the intrinsic reward can be determined according to the following formula:

[0071]

[0072] in, For intrinsic rewards, is the weight coefficient of novelty.

[0073] After determining intrinsic rewards based on the curiosity mechanism, the DDPG algorithm uses a policy network (Actor) and a value network (Critic) to plan the exploration path. Specifically, the policy network's input data consists of a 64-dimensional wind speed feature vector extracted from real-time wind speed and a 3-dimensional attitude feature vector extracted from the IMU attitude angle. The policy network's output is an action vector that includes changes in horizontal steering angle and vertical speed. The horizontal steering angle range is [-30°, 30°], and the vertical speed range is [-2m / s, 2m / s].

[0074] The input of the value network is the state vector of the drone and the action vector output by the policy network, and the output is a scalar value Q. The Q value is used to evaluate the long-term reward of the action vector output by the policy network in the current state. The long-term reward can be expressed as:

[0075]

[0076] in, For long-term rewards, For external rewards, is the weight coefficient of the intrinsic reward. The extrinsic reward can be determined based on the collision with the obstacle zone, whether the destination is reached, and the path time. For example, the extrinsic reward is +100 for successful obstacle avoidance, -500 for collision with an obstacle, and +200 for reaching the destination. The path time penalty is -0.1×path time.

[0077] S160 , comparing the detour time and the exploration time. If the detour time is less than or equal to the exploration time, selecting the detour path as the low-time path; if the detour time is greater than the exploration time, selecting the exploration path as the low-time path.

[0078] For example, while collecting environmental data, the noise of the environmental data is also analyzed. If the noise of the environmental data is greater than a noise threshold, the exploration path is selected only when the detour time is greater than the product of the exploration time and the safety factor.

[0079] In the embodiment of the present application, the safety factor β The safety factor is set to 1.2. The purpose of setting the safety factor is to prevent the drone from entering unknown dangers due to blind exploration attempts.

[0080] S170: Move along a low-time path to avoid an obstacle area.

[0081] Furthermore, after selecting the detour path or the exploration path, the detour path or the exploration path is bound to the position of the airflow generator. When the UAV passes by the airflow generator again, the low-time path is directly adopted.

[0082] The present application also provides a drone obstacle avoidance control system based on deep reinforcement learning, which includes:

[0083] Route acquisition module, used to obtain the flight route;

[0084] An area setting module is used to collect real-time environmental images during flight along the flight route, identify whether there is an airflow generator in the environmental image, and if so, set a set area on the airflow generator entrance and exit as an obstacle area;

[0085] A first path planning module is configured to extract multiple waypoints from the flight route, select the two waypoints that are outside the obstacle area and closest to the obstacle area as the route start point and route end point, respectively, plan a detour path between the route start point and the route end point, calculate the detour time required for the detour path, and bypass the obstacle area.

[0086] The second path planning module is used to determine the distance between the UAV and the obstacle area. If the distance is less than or equal to the distance threshold, the exploration phase begins. In the exploration phase, exploration points are generated around the obstacle area and a temporary path is planned for the UAV to move from the current position to the exploration points.

[0087] The hazard identification module is used to move to the exploration point along the temporary path, collect environmental data at the current location, and determine whether there is a dangerous area based on the environmental data;

[0088] The third path planning module is used to generate key parameters of the exploration path using the DDPG algorithm if there is a dangerous area, calculate the exploration time required for the exploration path, and ensure that the end point of the exploration path coincides with the end point of the route;

[0089] The path selection module is used to compare the detour time and the exploration time. If the detour time is less than or equal to the exploration time, the detour path is selected as the low-time path. If the detour time is greater than the exploration time, the exploration path is selected as the low-time path.

[0090] A movement control module for moving along a low-time path to avoid obstacle areas;

[0091] In the process of generating key parameters of the exploration path using the DDPG algorithm, the third path planning module corrects the range of the obstacle area according to the dangerous area to obtain the current range. The corresponding intrinsic reward is determined based on the difference between the current range and the historical range in the experience pool. The policy network is used to generate the action parameters of the drone. The value network is used to combine the external rewards and intrinsic rewards of the action parameters to evaluate the action parameters, and the action parameters with the largest value are used as the key parameters of the exploration path.

[0092] For example, the third path planning module is deployed on an edge computing device. During deployment, the trained DDPG algorithm model is quantized to FP16 (half-precision floating point) precision and then deployed on an NVIDIA Jetson Nano device to ensure a single forward inference time of less than 20ms.

[0093] Simulation experiment

[0094] Scene setup: Contains three airflow generators, each of which randomly generates airflow above it. The airflow speed is 5m / s~10m / s, and all are vertical upward airflows.

[0095] Sensor model: millimeter-wave radar with a detection range of 80 m, a velocity accuracy of ±0.5 m / s, and a frame rate of 5 Hz.

[0096] Key parameters: as shown in Table 1.

[0097] Table 1 Key simulation parameters

[0098]

[0099] After simulation, after adopting the obstacle avoidance control method of this application, in the scenario of stable airflow distribution, the time taken to explore the path is reduced by an average of 15% compared with the time taken to bypass the path. In the scenario of strong airflow disturbance, that is, wind speed greater than 8m / s, the collision rate is less than 5%, which is better than the bypass path.

[0100] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0101] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. The obstacle avoidance control method of UAV based on deep reinforcement learning is characterized by: include: Get flight routes; collecting a real-time environmental image during flight along the flight route, identifying whether an airflow generator exists in the environmental image, and if so, setting a set area on the entrance and exit of the airflow generator as an obstacle area; extracting a plurality of waypoints from the flight route, using two of the waypoints that are outside the obstacle area and closest to the obstacle area as the route starting point and route end point, respectively, planning a detour path between the route starting point and the route end point, and calculating a detour time required for the detour path, wherein the detour path detours outside the obstacle area; determining a distance between the UAV and the obstacle area, and if the distance is less than or equal to a distance threshold, entering an exploration phase, generating exploration points around the obstacle area, and planning a temporary path for the UAV to move from a current position to the exploration points; Moving along the temporary path to the exploration point, collecting environmental data of the current location, and determining whether there is a dangerous area based on the environmental data; If there is a dangerous area, the DDPG algorithm is used to generate key parameters of the exploration path, and the exploration time required for the exploration path is calculated. The end point of the exploration path coincides with the end point of the route; Comparing the detour time with the exploration time, if the detour time is less than or equal to the exploration time, selecting the detour path as the low-time path; if the detour time is greater than the exploration time, selecting the exploration path as the low-time path; moving along the low-time path to avoid the obstacle area; In the process of generating the key parameters of the exploration path using the DDPG algorithm, the range of the obstacle area is corrected according to the dangerous area to obtain the current range, and the corresponding intrinsic reward is determined according to the difference between the current range and the historical range in the experience pool. The policy network is used to generate the action parameters of the drone, and the value network is used to combine the extrinsic reward and the intrinsic reward of the action parameters to evaluate the action parameters, and the action parameters with the largest value are used as the key parameters of the exploration path.

2. The UAV obstacle avoidance control method based on deep reinforcement learning according to claim 1 is characterized in that: A target recognition model based on CNN is used to identify whether an airflow generator exists in the environmental image.

3. The UAV obstacle avoidance control method based on deep reinforcement learning according to claim 1 is characterized in that: After the obstacle area is set, the shape and size of the obstacle area are modified according to the environmental data.

4. The UAV obstacle avoidance control method based on deep reinforcement learning according to claim 1 is characterized in that: During the exploration phase, a caution area is set outside the obstacle area with the distance threshold as the width, the exploration point and the temporary path are both located in the caution area, and the multiple exploration points are at the same distance from the inner edge of the caution area.

5. The UAV obstacle avoidance control method based on deep reinforcement learning according to claim 1 is characterized in that: The environmental data includes real-time wind speed. If the real-time wind speed exceeds a wind speed threshold, it is considered that the dangerous area exists.

6. The UAV obstacle avoidance control method based on deep reinforcement learning according to claim 5 is characterized in that: The environmental data also includes the IMU attitude angle, and the exploration time is calculated according to the following formula: in, For the time spent on the exploration, is the length of the exploration path, is the maximum flight speed of the UAV, is the airflow disturbance coefficient, which is determined according to the IMU attitude angle.

7. The UAV obstacle avoidance control method based on deep reinforcement learning according to claim 5 is characterized in that: While collecting the environmental data, the noise of the environmental data is also analyzed. If the noise of the environmental data is greater than a noise threshold, the exploration path is selected only when the detour time is greater than the product of the exploration time and the safety factor.

8. The UAV obstacle avoidance control method based on deep reinforcement learning according to claim 1 is characterized in that: After selecting the low-time path, the low-time path is bound to the position of the airflow generator. When the drone passes by the airflow generator again, the low-time path is directly used.

9. A drone obstacle avoidance control system based on deep reinforcement learning, wherein the system applies the method according to any one of claims 1 to 8, characterized in that: The system comprises: Route acquisition module, used to obtain flight routes; an area setting module, configured to collect a real-time environmental image during flight along the flight route, identify whether an airflow generator exists in the environmental image, and if so, set a set area on the entrance and exit of the airflow generator as an obstacle area; a first path planning module, configured to extract a plurality of waypoints from the flight route, select two of the waypoints that are outside the obstacle area and closest to the obstacle area as a route start point and a route end point, respectively, plan a detour path between the route start point and the route end point, and calculate a detour time required for the detour path, wherein the detour path detours outside the obstacle area; a second path planning module, configured to determine a distance between the UAV and the obstacle area, and if the distance is less than or equal to a distance threshold, enter an exploration phase, in which exploration points are generated around the obstacle area, and a temporary path is planned for the UAV to move from its current position to the exploration points; a hazard identification module, configured to move along the temporary path to the exploration point, collect environmental data of the current location, and determine whether a dangerous area exists based on the environmental data; A third path planning module is configured to generate key parameters of an exploration path using a DDPG algorithm if a dangerous area exists, calculate the exploration time required for the exploration path, and ensure that the end point of the exploration path coincides with the end point of the route; a path selection module, configured to compare the detour time with the exploration time, and select the detour path as the low-time path if the detour time is less than or equal to the exploration time; and select the exploration path as the low-time path if the detour time is greater than the exploration time; a movement control module, configured to move along the low-time path to avoid the obstacle area; In the process of the third path planning module using the DDPG algorithm to generate the key parameters of the exploration path, the range of the obstacle area is corrected according to the dangerous area to obtain the current range, and the corresponding intrinsic reward is determined according to the difference between the current range and the historical range in the experience pool. The policy network is used to generate the action parameters of the drone, and the value network is used to combine the external reward of the action parameters and the intrinsic reward to evaluate the action parameters, and the action parameters with the largest value are used as the key parameters of the exploration path.

10. The UAV obstacle avoidance control system based on deep reinforcement learning according to claim 9, characterized in that: The third path planning module is deployed on the edge computing device.

Images