An Adaptive Management and Control System and Method for Low-Altitude UAVs Based on Reinforcement Learning

Abstract

The present invention relates to the technical field of unmanned aerial vehicle (UAV) control, and in particular to a low-altitude UAV adaptive control system and method based on reinforcement learning. The system comprises an environmental detection module, a flight simulation module, a data fitting module, an analysis module, a learning module, and a control module. The system determines whether the operating parameters of the UAV are qualified based on a driving deviation parameter. When it is determined that the UAV is driving abnormally, the operating parameters of the UAV are adjusted based on the driving difference. The operating parameters of the UAV are adjusted in real time according to environmental changes and flight status, thereby improving the driving safety of the UAV in low-altitude complex environments.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) control, and in particular to a low-altitude UAV adaptive control system and method based on reinforcement learning. Background Art

[0002] Research on drone control technology is currently a hot topic among universities and research institutions both domestically and internationally. In recent years, drone aerial operations have become increasingly common and widely used in a variety of situations. They effectively overcome the shortcomings of manned aircraft in aerial operations, reducing maintenance and procurement costs while increasing operational flexibility and adaptability. However, drones often face safety threats during flight from tangible obstacles such as mountains, buildings, trees, and power lines, as well as intangible obstacles such as no-fly zones and danger zones. Therefore, obstacle avoidance is crucial for drones, a prerequisite for them to perform complex, versatile, and challenging maneuvers.

[0003] Chinese patent publication number: CN108062109A, discloses a drone obstacle avoidance method, which belongs to the field of drone navigation and includes the following steps: S1: installing a laser and a satellite positioning receiver on the drone; S2: during the flight of the drone, using the laser to collect obstacle information around the drone; S3: importing the three-dimensional map library into the information processing module of the drone, performing coordinate conversion on the collected obstacle information, and searching for a three-dimensional map of the target area in the three-dimensional map library; S4: replanning the flight route based on the three-dimensional map of the target area; It can be seen that the above technical solution has the following problems: it does not take into account the real-time adjustment of the drone's operating parameters according to environmental changes and flight status, which affects the safety of the drone in low-altitude complex environments. Summary of the Invention

[0004] To this end, the present invention provides a low-altitude UAV adaptive control system and method based on reinforcement learning, which is used to overcome the problem that the existing technology does not take into account the real-time adjustment of the UAV's operating parameters according to environmental changes and flight status, affecting the UAV's driving safety in low-altitude complex environments.

[0005] In one aspect, the present invention provides a low-altitude UAV adaptive control system based on reinforcement learning, comprising:

[0006] An environmental detection module, which includes a wind speed sensor installed on the surface of the drone to detect wind speed, several radar transmitters to obtain the actual distance between the drone and obstacles, and a camera to obtain image information;

[0007] The flight simulation module is used to simulate and calculate the estimated distance between the drone and obstacles at each time point based on the drone's flight angle and flight speed;

[0008] a data fitting module, connected to the flight simulation module and the environment detection module, respectively, for drawing an actual distance time domain curve based on the actual distance and drawing an estimated distance time domain curve based on the estimated distance;

[0009] an analysis module, connected to the environment detection module, the flight simulation module, and the data fitting module, respectively, for periodically determining whether the operating parameters of the UAV are qualified based on the driving deviation parameter, and, when determining that the UAV is driving abnormally, adjusting the operating parameters of the UAV based on the driving deviation, or determining that the operating parameters of the UAV are qualified and controlling the UAV to operate with the current operating parameters;

[0010] a learning module, connected to the environment detection module and the analysis module, respectively, for periodically determining whether to store each obstacle outline based on the number of obstacles marked as abnormal by the analysis module, and, if the overlap between the outline information identified based on the image information and the obstacle outline is greater than a preset overlap, determining to adjust the flight speed of the UAV to a corresponding value based on the overlap;

[0011] A control module is connected to the analysis module and the learning module respectively, and is used to control the flight speed, flight angle or flight altitude of the UAV based on the determination results of the analysis module and the learning module.

[0012] Furthermore, the analysis module is used to determine whether the operating parameters of the drone are qualified based on the driving deviation parameter when the operating time of the drone reaches a preset monitoring time, including:

[0013] The actual distance time domain curve and the estimated distance time domain curve are overlapped based on the time axis, and the area enclosed by the two overlapped curves within the preset monitoring time is calculated to obtain the driving deviation parameter;

[0014] If the driving deviation parameter is less than or equal to the first preset driving deviation parameter, the operating parameters of the UAV are determined to be qualified, and the UAV is controlled to operate with the current operating parameters;

[0015] If the driving deviation parameter is less than or equal to the second preset driving deviation parameter and greater than the first preset driving deviation parameter, determining whether the operating parameters of the UAV are qualified based on the driving floating parameter;

[0016] If the driving deviation parameter is greater than the second preset driving deviation parameter, it is determined that the driving of the UAV is abnormal, and the operating parameters of the UAV are adjusted based on the driving difference.

[0017] Furthermore, the analysis module is used to determine whether the operating parameters of the UAV are qualified based on the driving floating parameters, including:

[0018] Solve the variance of the driving deviation parameter within each preset monitoring time to obtain the driving floating parameter;

[0019] If the driving floating parameter is less than or equal to the preset driving floating parameter, it is determined that the flight speed of the UAV is adjusted to the corresponding value based on the average value of the driving deviation parameters within each preset monitoring time;

[0020] If the driving floating parameter is greater than the preset driving floating parameter, it is determined that the driving of the UAV is abnormal, and the operating parameters of the UAV are adjusted based on the driving difference.

[0021] Furthermore, the control module is used to adjust the flight speed of the UAV to a corresponding value based on the average value of the driving deviation parameter within each preset monitoring time, wherein,

[0022] The average value of the driving deviation parameter within each preset monitoring time is recorded as the average deviation parameter;

[0023] The reduction in flight speed is proportional to the mean deviation parameter.

[0024] Furthermore, the analysis module adjusts the operating parameters of the drone based on the driving difference when determining that the drone is driving abnormally, including:

[0025] Calculating a difference between the driving deviation parameter and a second preset driving deviation parameter to obtain a driving difference value;

[0026] If the driving difference is less than or equal to the first preset driving difference, it is determined that the driving altitude of the UAV is adjusted to a corresponding value based on the acquired approach speed of the obstacle;

[0027] If the driving difference is less than or equal to the second preset driving difference and greater than the first preset driving difference, adjusting the operating parameters of the UAV based on the acquired wind speed;

[0028] If the driving difference is greater than a second preset driving difference, the obstacle is marked as an abnormal obstacle.

[0029] Furthermore, the control module is used to adjust the driving altitude of the UAV to a corresponding value based on the acquired obstacle approach speed, wherein:

[0030] The increase in ride height is proportional to the approach speed.

[0031] Furthermore, the analysis module is used to adjust the operating parameters of the UAV based on the acquired wind speed, including:

[0032] If the wind speed is less than or equal to the preset wind speed, the drone's altitude is adjusted to a corresponding value based on the obtained obstacle approach speed;

[0033] If the wind speed is greater than the preset wind speed, the flight angle of the drone will be adjusted to the corresponding value based on the wind speed.

[0034] Furthermore, the learning module is used to determine whether to store each obstacle profile for each abnormal obstacle based on the number of obstacles marked as abnormal obstacles obtained within the preset learning time when the operating time of the drone reaches the preset learning time;

[0035] If the number of abnormal obstacles is less than or equal to the preset number of obstacles, the drone will be controlled to continue operating using the current operating parameters;

[0036] If the number of abnormal obstacles is greater than the preset number of obstacles, the obstacle contours of each abnormal obstacle are obtained based on the image information of each abnormal obstacle, and each obstacle contour is stored;

[0037] The learning module determines to adjust the flight speed of the UAV to a corresponding value based on the overlap, under the condition that the overlap between the contour information identified based on the image information and the obstacle contour is greater than the preset overlap between the obstacle contour and the obstacle contour.

[0038] Furthermore, the control module is used to adjust the flight speed of the UAV to a corresponding value based on the overlap, wherein:

[0039] The reduction in flight speed is proportional to the degree of overlap.

[0040] On the other hand, the present invention also provides a low-altitude UAV adaptive control method using the above-mentioned low-altitude UAV adaptive control system, comprising:

[0041] S1, based on the flight angle and speed of the UAV, simulates and calculates the estimated distance between the UAV and the obstacle at each time point;

[0042] S2, obtain the actual distance between the drone and the obstacle;

[0043] S3, drawing an actual distance time domain curve based on the actual distance, drawing an estimated distance time domain curve based on the estimated distance, and determining a driving deviation parameter based on the two curves;

[0044] S4, determining whether the operating parameters of the UAV are qualified based on the driving deviation parameter, including determining whether the UAV is driving abnormally and adjusting the operating parameters of the UAV based on the driving deviation parameter, wherein adjusting the operating parameters of the UAV includes marking obstacles and adjusting the flight speed, flight angle, or flight altitude of the UAV;

[0045] Or, determine that the operating parameters of the UAV are qualified and control the UAV to operate with the current operating parameters;

[0046] S5: Determine whether to store each obstacle profile based on the acquired number of obstacles marked as abnormal.

[0047] Compared with the existing technology, the beneficial effect of the present invention lies in that, by setting an environmental detection module, a flight simulation module, a data fitting module, an analysis module, a learning module and a control module, whether the operating parameters of the UAV are qualified are determined based on the driving deviation parameters. When the driving of the UAV is judged to be abnormal, the operating parameters of the UAV are adjusted based on the driving difference. The operating parameters of the UAV are adjusted in real time according to environmental changes and flight status, thereby improving the driving safety of the UAV in low-altitude complex environments.

[0048] Furthermore, the actual distance time-domain curve represents the change in the actual distance between the drone and the obstacle over time during flight, while the estimated distance time-domain curve represents the change in the theoretical distance obtained based on flight simulation. The area enclosed by the two curves is used to determine the driving deviation parameter, which intuitively represents the degree of deviation between the drone's actual flight performance and the estimated performance. This allows for a comprehensive and objective assessment of the drone's operational status. When the driving deviation parameter is less than or equal to a first preset driving deviation parameter, the driving deviation parameter is small, indicating that the drone's actual flight performance is close to the estimated performance, and operating parameters are acceptable. The current flight status can be maintained, reducing unnecessary parameter adjustments and improving flight efficiency. When the driving deviation parameter is greater than a second preset driving deviation parameter, the drone's driving deviation is large, indicating an abnormal flight status. Further adjustments to the drone's operational parameters based on the driving deviation parameter effectively ensure flight safety.

[0049] Furthermore, the drone's operating parameters are determined to be qualified based on the driving float parameter. The driving float parameter is obtained by calculating the variance of the driving deviation parameter. The driving float parameter reflects the fluctuation of the driving deviation parameter over different preset monitoring periods. When the driving float parameter is less than or equal to the preset driving float parameter, the fluctuation is small, indicating that the drone's flight state is relatively stable. In this case, the drone experiences regular abnormal deviations, and the flight speed is fine-tuned to correct the drone's operating parameters. When the driving float parameter is greater than the preset driving float parameter, the fluctuation is large, indicating that the drone's flight state is unstable and may pose a potential risk. Further analysis and adjustment of the drone's operating parameters are required. The introduction of the driving float parameter provides a more in-depth assessment of the stability of the drone's flight state. In case of instability, timely adjustment measures are taken, effectively improving the drone's operational stability.

[0050] Furthermore, the operating parameters of the drone are adjusted based on the driving difference. The driving difference characterizes abnormal flight conditions. When the driving difference is less than or equal to the first preset driving difference, the driving difference is small. In this case, the moving obstacles caused by falling obstacles affect the smooth driving of the drone. In this case, the driving height is adjusted to avoid the obstacles; when the driving difference is less than or equal to the second preset driving difference and greater than the first preset driving difference, there are moving obstacles caused by wind force. In this case, adjustment is made based on wind speed. Wind speed is an important environmental factor affecting the flight of the drone. When the wind speed is less than or equal to the preset wind speed, the wind speed is small, and the approaching speed of the obstacle poses a more prominent threat to the flight safety of the drone. At this time, adjusting the driving height can effectively avoid obstacles and ensure flight safety. When the wind speed is greater than the preset wind speed, the wind speed is large. In this case, the excessive wind speed has a greater impact on the flight posture and stability of the drone. By adjusting the flight angle to better utilize the wind force, the drone can maintain stable flight. Different adjustment strategies are adopted according to the wind speed conditions to enable the drone to better adapt to different meteorological conditions. When the wind speed is small, it focuses on dealing with obstacle threats; when the wind speed is large, the flight angle is adjusted in time to deal with the impact of wind force, thereby improving the flight safety and stability of the drone in different environments.

[0051] Furthermore, the number of unusual obstacles reflects the complexity of the drone's flight environment. When the number of unusual obstacles is small, the flight environment is relatively stable, requiring no special handling for these obstacles and allowing the current operating parameters to continue. However, when the number of unusual obstacles is large, the flight environment is complex, necessitating the learning and storage of their characteristics—namely, their profiles—to facilitate identification during future flights. Profile overlap measures the degree of similarity between the currently identified obstacle and the stored unusual obstacles. The higher the overlap, the more likely the obstacle is a previously encountered dangerous obstacle. When the overlap exceeds a preset threshold, the flight speed is reduced to increase response time and ensure flight safety. Regular deduplication reduces storage redundancy, improving both storage and recognition efficiency. This enables the drone system to possess learning and adaptive capabilities. By learning and storing unusual obstacles, dangerous obstacles can be identified and appropriate measures can be taken in advance during future flights, reducing the probability of accidents. Adjusting the flight speed based on the overlap allows for more precise speed adjustments, ensuring flight safety and further improving flight efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a module block diagram of a low-altitude UAV adaptive control system based on reinforcement learning according to an embodiment of the present invention;

[0053] Figure 2This is a step flow of a low-altitude UAV adaptive control method based on reinforcement learning according to an embodiment of the present invention;

[0054] Figure 3 This is a logic decision diagram for determining whether the operating parameters of the drone are qualified based on the driving deviation parameters by the analysis module of an embodiment of the present invention;

[0055] Figure 4 This is a logical decision diagram for the analysis module of an embodiment of the present invention to determine whether the operating parameters of the drone are qualified based on the driving floating parameters. DETAILED DESCRIPTION

[0056] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0057] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0058] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside", and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0059] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0060] See also Figure 1 As shown in FIG, it is a module block diagram of a low-altitude UAV adaptive control system based on reinforcement learning according to an embodiment of the present invention. The low-altitude UAV adaptive control system according to the present invention includes:

[0061] An environmental detection module, which includes a wind speed sensor installed on the surface of the drone to detect wind speed, several radar transmitters to obtain the actual distance between the drone and obstacles, and a camera to obtain image information;

[0062] The flight simulation module is used to simulate and calculate the estimated distance between the drone and obstacles at each time point based on the drone's flight angle and flight speed;

[0063] a data fitting module, connected to the flight simulation module and the environment detection module, respectively, for drawing an actual distance time domain curve based on the actual distance and drawing an estimated distance time domain curve based on the estimated distance;

[0064] an analysis module, connected to the environment detection module, the flight simulation module, and the data fitting module, respectively, for periodically determining whether the operating parameters of the UAV are qualified based on the driving deviation parameter, and, when determining that the UAV is driving abnormally, adjusting the operating parameters of the UAV based on the driving deviation, or determining that the operating parameters of the UAV are qualified and controlling the UAV to operate with the current operating parameters;

[0065] a learning module, connected to the environment detection module and the analysis module, respectively, for periodically determining whether to store each obstacle outline based on the number of obstacles marked as abnormal by the analysis module, and, if the overlap between the outline information identified based on the image information and the obstacle outline is greater than a preset overlap, determining to adjust the flight speed of the UAV to a corresponding value based on the overlap;

[0066] A control module is connected to the analysis module and the learning module respectively, and is used to control the flight speed, flight angle or flight altitude of the UAV based on the determination results of the analysis module and the learning module.

[0067] Specifically, by setting up an environmental detection module, a flight simulation module, a data fitting module, an analysis module, a learning module and a control module, the operating parameters of the UAV are determined to be qualified based on the driving deviation parameters. When the driving of the UAV is judged to be abnormal, the operating parameters of the UAV are adjusted based on the driving difference. The operating parameters of the UAV are adjusted in real time according to environmental changes and flight status, thereby improving the driving safety of the UAV in low-altitude complex environments.

[0068] Specifically, a high-precision wind speed sensor and ultrasonic anemometer are used, with a measurement accuracy of up to ±0.1 m / s, which can accurately obtain the current wind speed in real time.

[0069] Specifically, a dynamic model is used to simulate and calculate the estimated distance between the drone and obstacles at each time point based on the drone's flight angle and speed, combined with the current wind speed. The flight angle is the angle between the drone's direction of travel and true north on a horizontal reference plane, with a measurement accuracy of ±0.1°. The flight speed is measured with an accuracy of ±0.1 m / s.

[0070] Specifically, the data fitting module plots the actual distance time-domain curve based on the actual distance data acquired by the radar transmitter, with time as the horizontal axis and actual distance as the vertical axis. Based on the estimated distance data calculated by the flight simulation module, the estimated distance time-domain curve is also plotted, with time as the horizontal axis and estimated distance as the vertical axis.

[0071] See also Figure 2 As shown in FIG, it is a step flow of a low-altitude UAV adaptive control method based on reinforcement learning according to an embodiment of the present invention. The low-altitude UAV adaptive control method according to the present invention includes:

[0072] S1, based on the flight angle and speed of the UAV, simulates and calculates the estimated distance between the UAV and the obstacle at each time point;

[0073] S2, obtain the actual distance between the drone and the obstacle;

[0074] S3, drawing an actual distance time domain curve based on the actual distance, drawing an estimated distance time domain curve based on the estimated distance, and determining a driving deviation parameter based on the two curves;

[0075] S4, determining whether the operating parameters of the UAV are qualified based on the driving deviation parameter, including determining whether the UAV is driving abnormally and adjusting the operating parameters of the UAV based on the driving deviation parameter, wherein adjusting the operating parameters of the UAV includes marking obstacles and adjusting the flight speed, flight angle, or flight altitude of the UAV;

[0076] Or, determine that the operating parameters of the UAV are qualified and control the UAV to operate with the current operating parameters;

[0077] S5: Determine whether to store each obstacle profile based on the acquired number of obstacles marked as abnormal.

[0078] See also Figure 3 As shown in FIG. , it is a logic determination diagram for determining whether the operating parameters of a drone are qualified based on the driving deviation parameter by an analysis module according to an embodiment of the present invention. The analysis module according to the present invention is used to determine whether the operating parameters of a drone are qualified based on the driving deviation parameter when the operating time of the drone reaches a preset monitoring time, including:

[0079] The actual distance time domain curve and the estimated distance time domain curve are overlapped based on the time axis, and the area enclosed by the two overlapped curves within the preset monitoring time is calculated to obtain the driving deviation parameter;

[0080] If the driving deviation parameter is less than or equal to the first preset driving deviation parameter, the operating parameters of the UAV are determined to be qualified, and the UAV is controlled to operate with the current operating parameters;

[0081] If the driving deviation parameter is less than or equal to the second preset driving deviation parameter and greater than the first preset driving deviation parameter, determining whether the operating parameters of the UAV are qualified based on the driving floating parameter;

[0082] If the driving deviation parameter is greater than the second preset driving deviation parameter, it is determined that the driving of the UAV is abnormal, and the operating parameters of the UAV are adjusted based on the driving difference.

[0083] Specifically, the actual distance time-domain curve represents the change in the actual distance between the drone and obstacles over time during flight, while the estimated distance time-domain curve represents the change in the theoretical distance obtained based on flight simulation. The area enclosed by the two curves is used to determine the driving deviation parameter, which intuitively represents the degree of deviation between the drone's actual flight performance and the estimated performance. This allows for a comprehensive and objective assessment of the drone's operational status. When the driving deviation parameter is less than or equal to a first preset driving deviation parameter, the driving deviation parameter is small, indicating that the drone's actual flight performance is close to the estimated performance, and operating parameters are acceptable. The current flight status can be maintained, reducing unnecessary parameter adjustments and improving flight efficiency. When the driving deviation parameter is greater than a second preset driving deviation parameter, the drone's driving deviation is large, indicating an abnormal flight status. Further adjustments to the drone's operational parameters based on the driving deviation parameter effectively ensure flight safety.

[0084] Specifically, the first preset driving deviation parameter L1 is selected within the interval [0.25, 0.3], and the second preset driving deviation parameter L2 is selected within the interval [0.6, 0.7], and the unit is m 2 .

[0085] See also Figure 4 As shown in FIG. , it is a logic determination diagram for determining whether the operating parameters of a drone are qualified based on driving floating parameters by an analysis module according to an embodiment of the present invention. The analysis module according to the present invention is used to determine whether the operating parameters of a drone are qualified based on driving floating parameters, including:

[0086] Solve the variance of the driving deviation parameter within each preset monitoring time to obtain the driving floating parameter;

[0087] If the driving floating parameter is less than or equal to the preset driving floating parameter, it is determined that the flight speed of the UAV is adjusted to the corresponding value based on the average value of the driving deviation parameters within each preset monitoring time;

[0088] If the driving floating parameter is greater than the preset driving floating parameter, it is determined that the driving of the UAV is abnormal, and the operating parameters of the UAV are adjusted based on the driving difference.

[0089] Specifically, the drone's operating parameters are determined to be qualified based on the driving floating parameter. The driving floating parameter is obtained by calculating the variance of the driving deviation parameter. The driving floating parameter reflects the fluctuation of the driving deviation parameter within different preset monitoring periods. When the driving floating parameter is less than or equal to the preset driving floating parameter, the fluctuation is small, indicating that the drone's flight state is relatively stable. In this case, the drone experiences regular abnormal deviations, and the flight speed is fine-tuned to correct the drone's operating parameters. When the driving floating parameter is greater than the preset driving floating parameter, the fluctuation is large, indicating that the drone's flight state is unstable and may pose potential risks. Further analysis and adjustment of the drone's operating parameters are required. The introduction of the driving floating parameter provides a more in-depth assessment of the stability of the drone's flight state. In case of instability, timely adjustment measures are taken, effectively improving the drone's operational stability.

[0090] Specifically, the preset driving floating parameter is selected within the interval [0.1, 0.3].

[0091] Specifically, there is no limit on the number of preset monitoring periods selected to determine the driving floating parameters. It can be understood that in order to obtain the historical driving conditions of the drone, the number of preset monitoring periods selected should be no less than 20.

[0092] Specifically, the control module is used to adjust the flight speed of the UAV to a corresponding value based on the average value of the driving deviation parameter within each preset monitoring time, wherein:

[0093] The average value of the driving deviation parameter within each preset monitoring time is recorded as the average deviation parameter;

[0094] The reduction in flight speed is proportional to the mean deviation parameter.

[0095] In this embodiment, optionally,

[0096] comparing the average deviation parameter with a first preset average comparison threshold and a second preset average comparison threshold;

[0097] If the average deviation parameter is less than or equal to the first preset average comparison threshold, the flight speed of the UAV is adjusted to 0.92 times the initial flight speed;

[0098] If the average deviation parameter is less than or equal to the second preset average comparison threshold and greater than the first preset average comparison threshold, the flight speed of the UAV is adjusted to 0.82 times the initial flight speed;

[0099] If the average deviation parameter is greater than the second preset average comparison threshold, the flight speed of the UAV is adjusted to 0.73 times the initial flight speed;

[0100] The first preset average comparison threshold is 1.2L1, and the second preset average comparison threshold is 1.5L1.

[0101] Specifically, the analysis module adjusts the operating parameters of the drone based on the driving difference when determining that the drone is driving abnormally, including:

[0102] Calculating a difference between the driving deviation parameter and a second preset driving deviation parameter to obtain a driving difference value;

[0103] If the driving difference is less than or equal to the first preset driving difference, it is determined that the driving altitude of the UAV is adjusted to a corresponding value based on the acquired approach speed of the obstacle;

[0104] If the driving difference is less than or equal to the second preset driving difference and greater than the first preset driving difference, adjusting the operating parameters of the UAV based on the acquired wind speed;

[0105] If the driving difference is greater than a second preset driving difference, the obstacle is marked as an abnormal obstacle.

[0106] Specifically, under the condition that the analysis module marks the obstacle as an abnormal obstacle, the judgment control module controls the drone to adjust the flight angle to 1.3 times the initial flight angle, and adjusts the flight speed of the drone to 1.3 times the initial flight speed.

[0107] Specifically, the first preset driving difference amount is selected within the interval [0.08, 0.11], and the second preset driving difference amount is selected within the interval [0.17, 0.23].

[0108] Specifically, the flight angle is the angle between the drone's travel direction and the true north direction on the horizontal reference plane.

[0109] Specifically, there is no limitation on the method for determining the approach speed. It can be determined by a radar transmitter installed on the surface of the UAV. The radar transmitter works by emitting electromagnetic waves and receiving reflected waves. According to the Doppler effect, when there is relative motion between the obstacle and the radar, the frequency of the reflected wave will change. By measuring the frequency change, the radial velocity of the obstacle, that is, the approach speed of the obstacle relative to the UAV, can be calculated.

[0110] Specifically, the control module is used to adjust the driving altitude of the UAV to a corresponding value based on the acquired obstacle approach speed, wherein:

[0111] The increase in ride height is proportional to the approach speed.

[0112] In this embodiment, optionally,

[0113] comparing the approach speed with a first preset approach speed and a second preset approach speed;

[0114] If the approach speed is less than or equal to the first preset approach speed, the flying altitude of the UAV is adjusted to 1.11 times the initial flying altitude;

[0115] If the approach speed is less than or equal to the second preset approach speed and greater than the first preset approach speed, the flying altitude of the UAV is adjusted to 1.21 times the initial flying altitude;

[0116] If the approach speed is greater than the second preset approach speed, the flying altitude of the UAV is adjusted to 1.27 times the initial flying altitude;

[0117] The first preset approach speed is 0.9 m / s, and the second preset approach speed is 1.7 m / s.

[0118] Specifically, the analysis module is used to adjust the operating parameters of the drone based on the acquired wind speed, including:

[0119] If the wind speed is less than or equal to the preset wind speed, the drone's altitude is adjusted to a corresponding value based on the obtained obstacle approach speed;

[0120] If the wind speed is greater than the preset wind speed, the flight angle of the drone will be adjusted to the corresponding value based on the wind speed.

[0121] Specifically, the preset wind speed is selected within the range [3.4m / s, 4.8m / s].

[0122] Specifically, the operating parameters of the drone are adjusted based on the driving difference. The driving difference characterizes the abnormal flight situation. When the driving difference is less than or equal to the first preset driving difference, the driving difference is small. In this case, the moving obstacles caused by falling obstacles affect the smooth driving of the drone. In this case, the driving height is adjusted to avoid the obstacles; when the driving difference is less than or equal to the second preset driving difference and greater than the first preset driving difference, there are moving obstacles caused by wind force. In this case, adjustment is made based on wind speed. Wind speed is an important environmental factor affecting the flight of the drone. When the wind speed is less than or equal to the preset wind speed, the wind speed is small, and the approaching speed of the obstacle poses a more prominent threat to the flight safety of the drone. At this time, adjusting the driving height can effectively avoid obstacles and ensure flight safety. When the wind speed is greater than the preset wind speed, the wind speed is large. In this case, the excessive wind speed has a greater impact on the flight posture and stability of the drone. By adjusting the flight angle to better utilize the wind force, the drone can maintain stable flight. Different adjustment strategies are adopted according to the wind speed conditions to enable the drone to better adapt to different meteorological conditions. When the wind speed is small, it focuses on dealing with obstacle threats; when the wind speed is large, the flight angle is adjusted in time to deal with the impact of wind force, thereby improving the flight safety and stability of the drone in different environments.

[0123] Specifically, the learning module is used to determine whether to store each obstacle profile for each abnormal obstacle based on the number of obstacles marked as abnormal obstacles obtained within the preset learning time when the operating time of the drone reaches the preset learning time;

[0124] If the number of abnormal obstacles is less than or equal to the preset number of obstacles, the drone will be controlled to continue operating using the current operating parameters;

[0125] If the number of abnormal obstacles is greater than the preset number of obstacles, the obstacle contours of each abnormal obstacle are obtained based on the image information of each abnormal obstacle, and each obstacle contour is stored;

[0126] The learning module determines to adjust the flight speed of the UAV to a corresponding value based on the overlap, under the condition that the overlap between the contour information identified based on the image information and the obstacle contour is greater than the preset overlap between the obstacle contour and the obstacle contour.

[0127] Specifically, the preset number of obstacles is selected within the interval [5, 8].

[0128] Specifically, there is no limitation on the process of obtaining the obstacle contour based on image information. The obstacle contour can be extracted by image preprocessing and using an edge detection algorithm. This is an existing technology and will not be described in detail.

[0129] Specifically, the degree of overlap between the contour information and the obstacle contour is the ratio of the area of the overlapped region between the contour information and the obstacle contour to the sum of the overlapped areas between the contour information and the obstacle contour.

[0130] Specifically, for the stored obstacle contours, the learning module regularly performs duplicate deletion, that is, only one obstacle contour is retained if the overlap between two obstacle contours is greater than 0.99.

[0131] Specifically, the number of unusual obstacles reflects the complexity of the drone's flight environment. When the number of unusual obstacles is small, the flight environment is relatively stable, requiring no special handling for these obstacles and allowing continued use of current operating parameters. However, when the number of unusual obstacles is large, the flight environment is complex, necessitating the learning and storage of their characteristics—namely, their profiles—to facilitate identification during future flights. Profile overlap measures the degree of similarity between the currently identified obstacle and the stored unusual obstacles. The higher the overlap, the more likely the currently encountered obstacle is a previously encountered dangerous obstacle. When the overlap exceeds a preset threshold, the flight speed is reduced to increase response time and ensure flight safety. Regular deduplication reduces storage redundancy, improving both storage and recognition efficiency. This provides the drone system with learning and adaptive capabilities. By learning and storing unusual obstacles, dangerous obstacles can be identified and appropriate measures can be taken in advance during future flights, reducing the probability of accidents. Adjusting the flight speed based on the overlap allows for more precise speed adjustments, ensuring flight safety and further improving flight efficiency.

[0132] Specifically, the control module is used to adjust the flight speed of the UAV to a corresponding value based on the overlap, wherein:

[0133] The reduction in flight speed is proportional to the degree of overlap.

[0134] In this embodiment, optionally,

[0135] comparing the degree of overlap with a first preset degree of overlap and a second preset degree of overlap;

[0136] If the overlap is less than or equal to the first preset overlap, the flight speed is adjusted to 0.95 times the initial flight speed;

[0137] If the overlap is less than or equal to the second preset overlap and greater than the first preset overlap, the flight speed is adjusted to 0.87 times the initial flight speed;

[0138] If the overlap is greater than the second preset overlap, the flight speed is adjusted to 0.81 times the initial flight speed;

[0139] The first preset overlap is 0.86, and the second preset overlap is 0.95.

[0140] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

[0141] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A low-altitude UAV adaptive control system based on reinforcement learning, characterized by: include: An environmental detection module, which includes several radar transmitters installed on the surface of the drone to obtain the actual distance between the drone and obstacles, and a camera installed on the surface of the drone to obtain image information; The flight simulation module is used to simulate and calculate the estimated distance between the drone and obstacles at each time point based on the drone's flight angle and flight speed; a data fitting module, connected to the flight simulation module and the environment detection module, respectively, for drawing an actual distance time domain curve based on the actual distance and drawing an estimated distance time domain curve based on the estimated distance; an analysis module, connected to the environment detection module, the flight simulation module, and the data fitting module, respectively, and configured to periodically determine a driving deviation parameter based on the actual distance time-domain curve and the estimated distance time-domain curve, determine whether the operating parameters of the UAV are qualified based on the driving deviation parameter, and, when determining that the UAV is driving abnormally, adjust the operating parameters of the UAV based on a driving difference amount, and, when the driving difference amount is greater than a second preset driving difference amount, mark the obstacle as an abnormal obstacle, wherein the driving difference amount is obtained by solving the difference between the driving deviation parameter and the second preset driving deviation parameter; a learning module, connected to the environment detection module and the analysis module, respectively, for periodically determining whether to store each obstacle outline based on the number of obstacles marked as abnormal by the analysis module, and, if the degree of overlap between the outline information identified based on the image information and the obstacle outline is greater than a preset overlap, determining to adjust the flight speed of the UAV to a corresponding value based on the overlap; A control module is connected to the analysis module and the learning module respectively, and is used to control the flight speed, flight angle or flight altitude of the UAV based on the determination results of the analysis module and the learning module.

2. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 1 is characterized in that: The analysis module is used to determine whether the operating parameters of the drone are qualified based on the driving deviation parameter when the operating time of the drone reaches a preset monitoring time, including: The actual distance time domain curve and the estimated distance time domain curve are overlapped based on the time axis, and the area enclosed by the two overlapped curves within the preset monitoring time is calculated to obtain the driving deviation parameter; If the driving deviation parameter is less than or equal to the first preset driving deviation parameter, the operating parameters of the UAV are determined to be qualified, and the UAV is controlled to operate with the current operating parameters; If the driving deviation parameter is less than or equal to the second preset driving deviation parameter and greater than the first preset driving deviation parameter, determining whether the operating parameters of the UAV are qualified based on the driving floating parameter; If the driving deviation parameter is greater than the second preset driving deviation parameter, it is determined that the driving of the UAV is abnormal, and the operating parameters of the UAV are adjusted based on the driving difference.

3. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 2 is characterized in that: The analysis module is used to determine whether the operating parameters of the UAV are qualified based on the driving floating parameters, including: Solve the variance of the driving deviation parameter within each preset monitoring time to obtain the driving floating parameter; If the driving floating parameter is less than or equal to the preset driving floating parameter, it is determined that the flight speed of the UAV is adjusted to the corresponding value based on the average value of the driving deviation parameters within each preset monitoring time; If the driving floating parameter is greater than the preset driving floating parameter, it is determined that the driving of the UAV is abnormal, and the operating parameters of the UAV are adjusted based on the driving difference.

4. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 3 is characterized in that: The control module is used to adjust the flight speed of the UAV to a corresponding value based on the average value of the driving deviation parameter within each preset monitoring time, wherein, The average value of the driving deviation parameter within each preset monitoring time is recorded as the average deviation parameter; The reduction in flight speed is proportional to the mean deviation parameter.

5. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 4 is characterized in that: The environmental detection module further includes a wind speed sensor disposed on the surface of the drone for detecting wind speed; The analysis module adjusts the operating parameters of the drone based on the driving difference when determining that the drone is driving abnormally, including: If the driving difference is less than or equal to the first preset driving difference, it is determined that the driving altitude of the UAV is adjusted to a corresponding value based on the acquired approach speed of the obstacle; If the driving difference is less than or equal to the second preset driving difference and greater than the first preset driving difference, the operating parameters of the UAV are adjusted based on the acquired wind speed.

6. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 5 is characterized in that: The control module is used to adjust the driving height of the UAV to a corresponding value based on the acquired obstacle approach speed, wherein: The increase in ride height is proportional to the approach speed.

7. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 6 is characterized in that: The analysis module is used to adjust the operating parameters of the UAV based on the acquired wind speed, including: If the wind speed is less than or equal to the preset wind speed, the drone's altitude is adjusted to a corresponding value based on the obtained obstacle approach speed; If the wind speed is greater than the preset wind speed, the flight angle of the drone will be adjusted to the corresponding value based on the wind speed.

8. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 7 is characterized in that: The learning module is used to determine whether to store each obstacle profile for each abnormal obstacle based on the number of obstacles marked as abnormal obstacles obtained within the preset learning time when the operating time of the drone reaches the preset learning time; If the number of abnormal obstacles is less than or equal to the preset number of obstacles, the drone will be controlled to continue operating using the current operating parameters; If the number of abnormal obstacles is greater than the preset number of obstacles, the obstacle contours of each abnormal obstacle are obtained based on the image information of each abnormal obstacle, and each obstacle contour is stored; The learning module determines to adjust the flight speed of the UAV to a corresponding value based on the overlap when the overlap between the contour information identified based on the image information and the obstacle contour is greater than a preset overlap.

9. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 8 is characterized in that: The control module is used to adjust the flight speed of the UAV to a corresponding value based on the overlap, wherein: The reduction in flight speed is proportional to the degree of overlap.

10. A low-altitude UAV adaptive control method using the low-altitude UAV adaptive control system according to any one of claims 1 to 9, characterized in that: include: S1, based on the flight angle and speed of the UAV, simulates and calculates the estimated distance between the UAV and the obstacle at each time point; S2, obtain the actual distance between the drone and the obstacle; S3, drawing an actual distance time domain curve based on the actual distance, drawing an estimated distance time domain curve based on the estimated distance, and determining a driving deviation parameter based on the two curves; S4, determining whether the operating parameters of the UAV are qualified based on the driving deviation parameter, including determining whether the UAV is driving abnormally and adjusting the operating parameters of the UAV based on the driving deviation parameter, wherein adjusting the operating parameters of the UAV includes marking obstacles and adjusting the flight speed, flight angle, or flight altitude of the UAV; Or, determine that the operating parameters of the UAV are qualified and control the UAV to operate with the current operating parameters; S5: Determine whether to store each obstacle profile based on the acquired number of obstacles marked as abnormal.

Images