Low-altitude unmanned aerial vehicle adaptive management and control system and method based on reinforcement learning

Abstract

The invention relates to the technical field of unmanned aerial vehicle management and control, in particular to a reinforcement learning-based low-altitude unmanned aerial vehicle adaptive management and control system and method, and the system comprises an environment detection module, a flight simulation module, a data fitting module, an analysis module, a learning module and a control module. Whether the operation parameters of the unmanned aerial vehicle are qualified or not is determined based on the driving deviation parameters, the operation parameters of the unmanned aerial vehicle are adjusted based on the driving difference when it is determined that the driving of the unmanned aerial vehicle is abnormal, the operation parameters of the unmanned aerial vehicle are adjusted in real time according to the environment change and the flight state, and the driving safety of the unmanned aerial vehicle in the low-altitude complex environment is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) control, and particularly to an adaptive control system and method for low-altitude UAVs based on reinforcement learning. Background Art

[0002] The research on UAV control technology is one of the hot topics concerned by universities and research institutions at home and abroad. In recent years, UAV aerial operations have become increasingly common and have been widely applied in various scenarios. It effectively overcomes various deficiencies of manned aircraft during aerial operations, reduces maintenance and procurement costs, and increases the flexibility and adaptability of operations. However, during flight, UAVs often face safety threats from tangible obstacles such as mountains, buildings, trees, and transmission lines, as well as constraints from intangible obstacles such as no-fly zones and dangerous areas. Therefore, obstacle avoidance flight is of great significance to UAVs and is a prerequisite for UAVs to complete complex and multi-functional high-difficulty actions.

[0003] Chinese Patent Publication No.: CN108062109A discloses a UAV obstacle avoidance method in the field of UAV navigation, including the following steps: S1: Install a laser and a satellite positioning receiver on the UAV; S2: During the flight of the UAV, collect obstacle information around the UAV through the laser; S3: Import a three-dimensional map library into the information processing module of the UAV, perform coordinate conversion on the collected obstacle information, and search for the three-dimensional map of the target area in the three-dimensional map library; S4: Re-plan the flight route according to the three-dimensional map of the target area. It can be seen that the above technical solution has the following problems: It does not consider adjusting the operating parameters of the UAV in real time according to environmental changes and flight states, which affects the driving safety of the UAV in a low-altitude complex environment. Summary of the Invention

[0004] To this end, the present invention provides an adaptive control system and method for low-altitude UAVs based on reinforcement learning to overcome the problem in the prior art that the operating parameters of the UAV are not adjusted in real time according to environmental changes and flight states, which affects the driving safety of the UAV in a low-altitude complex environment.

[0005] On the one hand, the present invention provides an adaptive control system for low-altitude UAVs based on reinforcement learning, including: An environment detection module, which includes a wind speed sensor arranged on the surface of the UAV for detecting wind speed, a plurality of radar transmitters for obtaining the actual distance between the UAV and obstacles, and a camera for obtaining image information; A flight simulation module, which is used to simulate and calculate the estimated distance between the UAV and obstacles at each time node based on the flight angle and flight speed of the UAV; A data fitting module, which is respectively connected to the flight simulation module and the environment detection module, and is used to draw an actual distance time-domain curve based on the actual distance and draw an estimated distance time-domain curve based on the estimated distance; An analysis module, which is respectively connected to the environment detection module, the flight simulation module and the data fitting module, and is used to periodically determine whether the operating parameters of the drone are qualified based on the driving deviation parameters, and when it is determined that the driving of the drone is abnormal, adjust the operating parameters of the drone based on the driving difference amount, or determine that the operating parameters of the drone are qualified and control the drone to operate with the current operating parameters; A learning module, which is respectively connected to the environment detection module and the analysis module, and is used to periodically determine whether to store the contour of each obstacle based on the number of obstacles marked as abnormal obstacles by the analysis module obtained, and under the condition that the coincidence degree between the contour information recognized based on the image information and the obstacle contour is greater than the preset coincidence degree, determine to adjust the flight speed of the drone to the corresponding value based on the coincidence degree; A control module, which is respectively connected to the analysis module and the learning module, and is used to control the flight speed, flight angle or flight height of the drone based on the determination results of the analysis module and the learning module.

[0006] Further, the analysis module is used to determine whether the operating parameters of the drone are qualified based on the driving deviation parameters under the condition that the operating duration of the drone reaches the preset monitoring duration, including: Coincide the actual distance time-domain curve and the estimated distance time-domain curve based on the time axis, and calculate the area enclosed by the two coincided curves within the preset monitoring duration to obtain the driving deviation parameter; If the driving deviation parameter is less than or equal to the first preset driving deviation parameter, it is determined that the operating parameters of the drone are qualified, and the drone is controlled to operate with the current operating parameters; If the driving deviation parameter is less than or equal to the second preset driving deviation parameter and greater than the first preset driving deviation parameter, determine whether the operating parameters of the drone are qualified based on the driving floating parameter; If the driving deviation parameter is greater than the second preset driving deviation parameter, it is determined that the driving of the drone is abnormal, and the operating parameters of the drone are adjusted based on the driving difference amount.

[0007] Further, the analysis module is used to determine whether the operating parameters of the drone are qualified based on the driving floating parameter, including: Solve the variance of the driving deviation parameters within each preset monitoring duration to obtain the driving floating parameter; If the driving floating parameter is less than or equal to the preset driving floating parameter, it is determined to adjust the flight speed of the drone to the corresponding value based on the average value of the driving deviation parameters within each preset monitoring duration; If the driving floating parameter is greater than the preset driving floating parameter, it is determined that the driving of the drone is abnormal, and the operating parameters of the drone are adjusted based on the driving difference amount.

[0008] Further, the control module is used to adjust the flight speed of the drone to a corresponding value based on the average value of the driving deviation parameters within each preset monitoring duration, where The average value of the driving deviation parameters within each preset monitoring duration is denoted as the average deviation parameter; The reduction amplitude of the flight speed is proportional to the average deviation parameter.

[0009] Further, when the analysis module determines that the driving of the drone is abnormal, it adjusts the operating parameters of the drone based on the driving difference amount, including: Solve the difference between the driving deviation parameter and the second preset driving deviation parameter to obtain the driving difference amount; If the driving difference amount is less than or equal to the first preset driving difference amount, it is determined that the driving height of the drone is adjusted to a corresponding value based on the approaching speed of the obtained obstacle; If the driving difference amount is less than or equal to the second preset driving difference amount and greater than the first preset driving difference amount, the operating parameters of the drone are adjusted based on the obtained wind speed; If the driving difference amount is greater than the second preset driving difference amount, the obstacle is marked as an abnormal obstacle.

[0010] Further, the control module is used to adjust the driving height of the drone to a corresponding value based on the approaching speed of the obtained obstacle, where The increase amplitude of the driving height is proportional to the approaching speed.

[0011] Further, the analysis module is used to adjust the operating parameters of the drone based on the obtained wind speed, including: If the wind speed is less than or equal to the preset wind speed, it is determined that the driving height of the drone is adjusted to a corresponding value based on the approaching speed of the obtained obstacle; If the wind speed is greater than the preset wind speed, the flight angle of the drone is adjusted to a corresponding value based on the wind speed.

[0012] Further, the learning module is used to determine whether to store the obstacle contours of each abnormal obstacle based on the number of abnormal obstacles marked within the preset learning duration when the operating duration of the drone reaches the preset learning duration; If the number of abnormal obstacles is less than or equal to the preset number of obstacles, control the drone to continue running with the current operating parameters; If the number of abnormal obstacles is greater than the preset number of obstacles, based on the obtained image information of each abnormal obstacle, obtain the obstacle contours of each abnormal obstacle, and store each obstacle contour; When the degree of overlap between the contour information recognized based on the image information and the obstacle contour is greater than the preset degree of overlap, the learning module determines to adjust the flight speed of the UAV to the corresponding value based on the degree of overlap.

[0013] Furthermore, the control module is used to adjust the flight speed of the UAV to the corresponding value based on the degree of overlap, where the reduction amplitude of the flight speed is proportional to the degree of overlap.

[0014] On the other hand, the present invention also provides a method for adaptively controlling a low-altitude UAV using the above-mentioned low-altitude UAV adaptive control system, including: S1, simulating and calculating the estimated distance between the UAV and the obstacle at each time node based on the flight angle and flight speed of the UAV; S2, obtaining the actual distance between the UAV and the obstacle; S3, drawing an actual distance time-domain curve based on the actual distance, drawing an estimated distance time-domain curve based on the estimated distance, and determining a driving deviation parameter based on the two curves; S4, determining whether the operating parameters of the UAV are qualified based on the driving deviation parameter, including determining that the UAV is driving abnormally, and adjusting the operating parameters of the UAV based on the driving difference amount, where adjusting the operating parameters of the UAV includes marking the obstacle, adjusting the flight speed, flight angle or flight height of the UAV; Or, determining that the operating parameters of the UAV are qualified, and controlling the UAV to operate with the current operating parameters; S5, determining whether to store each obstacle contour based on the number of obstacles marked as abnormal obstacles obtained.

[0015] Compared with the prior art, the beneficial effect of the present invention is that by setting an environment detection module, a flight simulation module, a data fitting module, an analysis module, a learning module and a control module, it is determined whether the operating parameters of the UAV are qualified based on the driving deviation parameter. When it is determined that the UAV is driving abnormally, the operating parameters of the UAV are adjusted based on the driving difference amount, and the operating parameters of the UAV are adjusted in real time according to the environmental changes and flight states, improving the driving safety of the UAV in a low-altitude complex environment.

[0016] Further, the actual distance time-domain curve characterizes the change in the actual distance between the UAV and the obstacle during flight over time, and the estimated distance time-domain curve characterizes the theoretical distance change obtained based on flight simulation. The driving deviation parameter is determined by calculating the area enclosed by the two curves. The driving deviation parameter intuitively characterizes the deviation degree between the actual flight situation and the estimated situation of the UAV, comprehensively and objectively evaluating the operating state of the UAV. When the driving deviation parameter is less than or equal to the first preset driving deviation parameter, the driving deviation parameter is small. In this case, the actual flight of the UAV is relatively close to the estimated situation, and the operating parameters are qualified. The UAV can continue to fly in the current state, reducing unnecessary parameter adjustments and improving flight efficiency. When the driving deviation parameter is greater than the second preset driving deviation parameter, the driving deviation of the UAV is large. In this case, the flight state is abnormal. Further adjust the operating parameters of the UAV based on the driving difference amount, effectively ensuring flight safety.

[0017] Further, based on the driving fluctuation parameter, it is determined whether the operating parameters of the UAV are qualified. The driving fluctuation parameter is obtained by calculating the variance of the driving deviation parameter, and the driving fluctuation parameter reflects the fluctuation of the driving deviation parameter within different preset monitoring time periods. When the driving fluctuation parameter is less than or equal to the preset driving fluctuation parameter, the fluctuation is small in this case, indicating that the flight state of the UAV is relatively stable. If there are regular abnormal deviations in the UAV in this case, slightly adjust the flight speed to correct the operating parameters of the UAV. When the driving fluctuation parameter is greater than the preset driving fluctuation parameter, the fluctuation is large. In this case, the flight state of the UAV is unstable and there may be potential risks. Further analyze and adjust the operating parameters of the UAV. Introducing the driving fluctuation parameter can more deeply evaluate the stability of the UAV flight state. For unstable situations, timely adjustment measures are taken, effectively improving the operating stability of the UAV.

[0018] Further, the operation parameters of the drone are adjusted based on the driving difference amount, which characterizes the abnormal situation of flight. When the driving difference amount is less than or equal to the first preset driving difference amount, the driving difference amount is small. In this case, due to the presence of falling obstacles, the moving obstacles affect the stable driving of the drone. In this case, the driving height is adjusted to avoid the obstacles. When the driving difference amount is less than or equal to the second preset driving difference amount and greater than the first preset driving difference amount, there are moving obstacles caused by wind in this case. In this case, adjustments are made based on the wind speed. Wind speed is an important environmental factor affecting the flight of the drone. When the wind speed is less than or equal to the preset wind speed, the wind speed is small, and the approaching speed of the obstacle poses a more prominent threat to the flight safety of the drone. At this time, adjusting the driving height can effectively avoid the obstacles and ensure flight safety. When the wind speed is greater than the preset wind speed, the wind speed is large. In this case, the excessive wind speed has a greater impact on the flight attitude and stability of the drone. By adjusting the flight angle to better utilize the wind force, the drone can maintain stable flight. By adopting different adjustment strategies according to the wind speed conditions, the drone can better adapt to different meteorological conditions. When the wind speed is small, it focuses on dealing with the threat of obstacles. When the wind speed is large, the flight angle is adjusted in a timely manner to cope with the wind force impact, improving the flight safety and stability of the drone in different environments.

[0019] Furthermore, the number of abnormal obstacles reflects the complexity of the drone flight environment. When the number of abnormal obstacles is small, it indicates that the flight environment is relatively stable, and there is no need to perform special processing on the abnormal obstacles, and the current operation parameters can continue to be used. When the number of abnormal obstacles is large, it indicates that the flight environment is relatively complex, and it is necessary to learn and store the characteristics of the abnormal obstacles, that is, the obstacle contours, so as to identify these obstacles in future flights. The contour coincidence degree characterizes the similarity between the currently identified obstacle and the previously stored abnormal obstacle. The higher the coincidence degree, the more likely the currently encountered obstacle is the dangerous obstacle encountered before. When the coincidence degree is greater than the preset coincidence degree, the flight speed is reduced to increase the reaction time and ensure flight safety. Regular repeated deletion operations are performed to reduce storage redundancy and improve the storage efficiency and recognition efficiency of the system. The drone system is equipped with learning and adaptive capabilities. By learning and storing abnormal obstacles, it is possible to identify dangerous obstacles in advance and take corresponding measures in future flights, reducing the probability of accidents. Adjusting the flight speed according to the coincidence degree makes the speed adjustment more accurate, ensuring flight safety and further improving flight efficiency. Brief Description of the Drawings

[0020] Figure 1 It is a block diagram of the modules of the low-altitude drone adaptive control system based on reinforcement learning according to an embodiment of the present invention; Figure 2 It is a step flow chart of the low-altitude drone adaptive control method based on reinforcement learning according to an embodiment of the present invention; Figure 3 This is a logic decision diagram for the analysis module of the embodiment of the present invention to determine whether the operating parameters of the drone are qualified based on the driving deviation parameters; Figure 4 This is a logic decision diagram for the analysis module of the embodiment of the present invention to determine whether the operating parameters of the drone are qualified based on the driving floating parameters. Detailed implementation manners

[0021] In order to make the purpose and advantages of the present invention more clear and understandable, the present invention will be further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0022] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention and do not limit the protection scope of the present invention.

[0023] It should be noted that in the description of the present invention, the terms indicating the direction or position relationship such as "upper", "lower", "left", "right", "inner", "outer", etc. are based on the direction or position relationship shown in the drawings. This is only for convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention.

[0024] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0025] Please refer to Figure 1 As shown, it is a module block diagram of the low-altitude drone adaptive control system based on reinforcement learning in the embodiment of the present invention. The low-altitude drone adaptive control system of the present invention includes: An environment detection module, which includes a wind speed sensor arranged on the surface of the drone to detect the wind speed, a plurality of radar transmitters for obtaining the actual distance between the drone and the obstacle, and a camera for obtaining image information; A flight simulation module, which is used to simulate and calculate the estimated distance between the drone and the obstacle at each time node based on the flight angle and flight speed of the drone; A data fitting module, which is respectively connected to the flight simulation module and the environment detection module, and is used to draw an actual distance time-domain curve based on the actual distance and draw an estimated distance time-domain curve based on the estimated distance; An analysis module, which is respectively connected to the environment detection module, the flight simulation module and the data fitting module, and is used to periodically determine whether the operating parameters of the drone are qualified based on the driving deviation parameters, and when it is determined that the driving of the drone is abnormal, adjust the operating parameters of the drone based on the driving difference amount, or determine that the operating parameters of the drone are qualified and control the drone to operate with the current operating parameters; A learning module, which is respectively connected to the environment detection module and the analysis module, and is used to periodically determine whether to store the contour of each obstacle based on the number of obstacles marked as abnormal obstacles by the analysis module obtained, and when the coincidence degree between the contour information recognized based on the image information and the obstacle contour is greater than the preset coincidence degree, determine to adjust the flight speed of the drone to the corresponding value based on the coincidence degree; A control module, which is respectively connected to the analysis module and the learning module, and is used to control the flight speed, flight angle or flight height of the drone based on the determination results of the analysis module and the learning module.

[0026] Specifically, by setting an environment detection module, a flight simulation module, a data fitting module, an analysis module, a learning module and a control module, it is determined whether the operating parameters of the drone are qualified based on the driving deviation parameters. When it is determined that the driving of the drone is abnormal, the operating parameters of the drone are adjusted based on the driving difference amount, and the operating parameters of the drone are adjusted in real time according to the environmental changes and flight states, improving the driving safety of the drone in a low-altitude complex environment.

[0027] Specifically, a high-precision wind speed sensor, an ultrasonic anemometer is adopted, and its measurement accuracy can reach ±0.1 m / s, and it can accurately obtain the current wind speed in real time.

[0028] Specifically, based on the flight angle and flight speed of the drone, combined with the current wind speed, a dynamic model is used to simulate and calculate the estimated distance between the drone and the obstacle at each time node. The flight angle is the included angle between the driving direction of the drone and the true north direction on the horizontal reference plane, and the measurement accuracy is ±0.1°. The measurement accuracy of the flight speed is ±0.1 m / s.

[0029] Specifically, the data fitting module draws an actual distance time-domain curve with time as the horizontal axis and the actual distance as the vertical axis according to the actual distance data obtained by the radar transmitter. Based on the estimated distance data calculated by the flight simulation module, a time-domain curve of the estimated distance is also drawn with time as the horizontal axis and the estimated distance as the vertical axis.

[0030] Please refer to Figure 2As shown, it is the step flow of the low-altitude UAV adaptive control method based on reinforcement learning in an embodiment of the present invention. The low-altitude UAV adaptive control method of the present invention includes: S1. Simulate and calculate the estimated distance between the UAV and the obstacle at each time node based on the flight angle and flight speed of the UAV; S2. Obtain the actual distance between the UAV and the obstacle; S3. Draw the actual distance time-domain curve based on the actual distance, draw the estimated distance time-domain curve based on the estimated distance, and determine the driving deviation parameter based on the two curves; S4. Determine whether the operating parameters of the UAV are qualified based on the driving deviation parameter, including determining that the UAV is driving abnormally, and adjusting the operating parameters of the UAV based on the driving difference amount, where adjusting the operating parameters of the UAV includes marking the obstacle, adjusting the flight speed, flight angle or flight height of the UAV; Or, determine that the operating parameters of the UAV are qualified, and control the UAV to operate with the current operating parameters; S5. Determine whether to store the contour of each obstacle based on the number of obstacles marked as abnormal obstacles obtained.

[0031] Please refer to Figure 3 As shown, it is the logical decision diagram for the analysis module in an embodiment of the present invention to determine whether the operating parameters of the UAV are qualified based on the driving deviation parameter. The analysis module of the present invention is used to determine whether the operating parameters of the UAV are qualified based on the driving deviation parameter under the condition that the operating duration of the UAV reaches the preset monitoring duration, including: Coincide the actual distance time-domain curve and the estimated distance time-domain curve based on the time axis, and calculate the area enclosed by the two coincident curves within the preset monitoring duration to obtain the driving deviation parameter; If the driving deviation parameter is less than or equal to the first preset driving deviation parameter, it is determined that the operating parameters of the UAV are qualified, and the UAV is controlled to operate with the current operating parameters; If the driving deviation parameter is less than or equal to the second preset driving deviation parameter and greater than the first preset driving deviation parameter, determine whether the operating parameters of the UAV are qualified based on the driving floating parameter; If the driving deviation parameter is greater than the second preset driving deviation parameter, it is determined that the UAV is driving abnormally, and the operating parameters of the UAV are adjusted based on the driving difference amount.

[0032] Specifically, the actual distance time-domain curve characterizes the change in the actual distance between the UAV and the obstacle during flight over time, and the estimated distance time-domain curve characterizes the theoretical distance change obtained based on flight simulation. The driving deviation parameter is determined by calculating the area enclosed by the two curves. The driving deviation parameter intuitively characterizes the deviation degree between the actual flight situation of the UAV and the estimated situation, comprehensively and objectively evaluating the operating state of the UAV. When the driving deviation parameter is less than or equal to the first preset driving deviation parameter, the driving deviation parameter is small. In this case, the actual flight of the UAV is relatively close to the estimated situation, the operating parameters are qualified, and the current state of flight can be maintained to reduce unnecessary parameter adjustment and improve flight efficiency. When the driving deviation parameter is greater than the second preset driving deviation parameter, the driving deviation of the UAV is large. In this case, the flight state is abnormal, and the operating parameters of the UAV are further adjusted based on the driving difference amount, effectively ensuring flight safety.

[0033] Specifically, the first preset driving deviation parameter L1 is selected within the interval [0.25, 0.3], and the second preset driving deviation parameter L2 is selected within the interval [0.6, 0.7], with the unit of m. 2 。

[0034] Please refer to Figure 4 as shown. It is a logical decision diagram for the analysis module of the embodiment of the present invention to determine whether the operating parameters of the UAV are qualified based on the driving floating parameter. The analysis module of the present invention is used to determine whether the operating parameters of the UAV are qualified based on the driving floating parameter, including: Solving the variance of the driving deviation parameter within each preset monitoring time period to obtain the driving floating parameter; If the driving floating parameter is less than or equal to the preset driving floating parameter, it is determined that the flight speed of the UAV is adjusted to the corresponding value based on the average value of the driving deviation parameters within each preset monitoring time period; If the driving floating parameter is greater than the preset driving floating parameter, it is determined that the driving of the UAV is abnormal, and the operating parameters of the UAV are adjusted based on the driving difference amount.

[0035] Specifically, based on the driving floating parameter, it is determined whether the operating parameters of the drone are qualified. The driving floating parameter is obtained by calculating the variance of the driving deviation parameter, and the driving floating parameter reflects the fluctuation of the driving deviation parameter within different preset monitoring time durations. When the driving floating parameter is less than or equal to the preset driving floating parameter, the fluctuation is small in this case, indicating that the flight state of the drone is relatively stable. In this case, if there are regular abnormal deviations in the drone, the flight speed is finely adjusted to correct the operating parameters of the drone. When the driving floating parameter is greater than the preset driving floating parameter, the fluctuation is large. In this case, the flight state of the drone is unstable and there may be potential risks, and the operating parameters of the drone are further analyzed and adjusted. Introducing the driving floating parameter can more deeply evaluate the stability of the drone's flight state. For unstable situations, adjustment measures are taken in a timely manner, effectively improving the operating stability of the drone.

[0036] Specifically, the preset driving floating parameter is selected within the range of [0.1, 0.3].

[0037] Specifically, the number of preset monitoring time durations used to determine the selection of the driving floating parameter is not limited. It can be understood that in order to obtain the historical driving situation of the drone, the number of preset monitoring time durations selected should be no less than 20.

[0038] Specifically, the control module is used to adjust the flight speed of the drone to the corresponding value based on the average value of the driving deviation parameters within each preset monitoring time duration, where the average value of the driving deviation parameters within each preset monitoring time duration is denoted as the average deviation parameter; the reduction amplitude of the flight speed is proportional to the average deviation parameter.

[0039] In this embodiment, optionally, the average deviation parameter is compared with the first preset average comparison threshold and the second preset average comparison threshold; if the average deviation parameter is less than or equal to the first preset average comparison threshold, the flight speed of the drone is adjusted to 0.92 times the initial flight speed; if the average deviation parameter is less than or equal to the second preset average comparison threshold and greater than the first preset average comparison threshold, the flight speed of the drone is adjusted to 0.82 times the initial flight speed; if the average deviation parameter is greater than the second preset average comparison threshold, the flight speed of the drone is adjusted to 0.73 times the initial flight speed; the first preset average comparison threshold is taken as 1.2L1, and the second preset average comparison threshold is taken as 1.5L1.

[0040] Specifically, when the analysis module determines that the driving of the drone is abnormal, it adjusts the operating parameters of the drone based on the driving difference amount, including: Solve for the difference between the driving deviation parameter and the second preset driving deviation parameter to obtain the driving difference amount; If the driving difference amount is less than or equal to the first preset driving difference amount, it is determined that the driving height of the drone is adjusted to the corresponding value based on the obtained approaching speed of the obstacle; If the driving difference amount is less than or equal to the second preset driving difference amount and greater than the first preset driving difference amount, the operating parameters of the drone are adjusted based on the obtained wind speed; If the driving difference amount is greater than the second preset driving difference amount, the obstacle is marked as an abnormal obstacle.

[0041] Specifically, under the condition that the analysis module marks the obstacle as an abnormal obstacle, it is determined that the control module controls the drone to adjust the flight angle to 1.3 times the initial flight angle and adjusts the flight speed of the drone to 1.3 times the initial flight speed.

[0042] Specifically, the first preset driving difference amount is selected within the interval [0.08, 0.11], and the second preset driving difference amount is selected within the interval [0.17, 0.23].

[0043] Specifically, the flight angle is the angle between the driving direction of the drone and the true north direction on the horizontal reference plane.

[0044] Specifically, there is no limitation on the determination method of the approaching speed. It can be determined by a radar transmitter set on the surface of the drone. The radar transmitter works by emitting electromagnetic waves and receiving reflected waves. According to the Doppler effect, when there is relative motion between the obstacle and the radar, the frequency of the reflected wave will change. By measuring the frequency change, the radial speed of the obstacle, that is, the approaching speed of the obstacle relative to the drone, can be calculated.

[0045] Specifically, the control module is used to adjust the driving height of the drone to the corresponding value based on the obtained approaching speed of the obstacle, where The increase amplitude of the driving height is proportional to the approaching speed.

[0046] In this embodiment, optionally, The approaching speed is compared with the first preset approaching speed and the second preset approaching speed; If the approaching speed is less than or equal to the first preset approaching speed, the driving height of the drone is adjusted to 1.11 times the initial driving height; If the approaching speed is less than or equal to the second preset approaching speed and greater than the first preset approaching speed, the driving height of the drone is adjusted to 1.21 times the initial driving height; If the approaching speed is greater than the second preset approaching speed, the driving height of the drone is adjusted to 1.27 times the initial driving height; The first preset approach speed is taken as 0.9 m / s, and the second preset approach speed is taken as 1.7 m / s.

[0047] Specifically, the analysis module is used to adjust the operating parameters of the drone based on the acquired wind speed, including: If the wind speed is less than or equal to the preset wind speed, it is determined that the driving height of the drone is adjusted to the corresponding value based on the acquired approach speed of the obstacle; If the wind speed is greater than the preset wind speed, the flight angle of the drone is adjusted to the corresponding value based on the wind speed.

[0048] Specifically, the preset wind speed is selected within the range [3.4 m / s, 4.8 m / s].

[0049] Specifically, the operating parameters of the drone are adjusted based on the driving difference amount. The driving difference amount characterizes the abnormal situation of flight. When the driving difference amount is less than or equal to the first preset driving difference amount, the driving difference amount is small. In this case, due to the presence of falling obstacles, the moving obstacles affect the stable driving of the drone. In this case, the driving height is adjusted to avoid the obstacles; when the driving difference amount is less than or equal to the second preset driving difference amount and greater than the first preset driving difference amount, in this case, there are moving obstacles caused by the wind. In this case, adjustments are made based on the wind speed; the wind speed is an important environmental factor affecting the flight of the drone. When the wind speed is less than or equal to the preset wind speed, the wind speed is small, and the threat of the approach speed of the obstacle to the flight safety of the drone is more prominent. At this time, adjusting the driving height can effectively avoid the obstacles and ensure flight safety. When the wind speed is greater than the preset wind speed, the wind speed is large. In this case, the excessive wind speed has a greater impact on the flight attitude and stability of the drone. By adjusting the flight angle to better utilize the wind force, the drone can maintain stable flight. By adopting different adjustment strategies according to the wind speed conditions, the drone can better adapt to different meteorological conditions. When the wind speed is small, it focuses on coping with the threat of obstacles; when the wind speed is large, it promptly adjusts the flight angle to cope with the wind force impact, improving the flight safety and stability of the drone in different environments.

[0050] Specifically, the learning module is used to determine whether to store the obstacle contours of each abnormal obstacle based on the number of abnormal obstacles marked as such obtained within the preset learning duration when the operating duration of the drone reaches the preset learning duration; If the number of abnormal obstacles is less than or equal to the preset number of obstacles, the drone is controlled to continue operating with the current operating parameters; If the number of abnormal obstacles is greater than the preset number of obstacles, based on the acquired image information of each abnormal obstacle and the acquired obstacle contours of each abnormal obstacle, the obstacle contours are stored. When the degree of overlap between the contour information recognized based on the image information and the obstacle contour is greater than the preset degree of overlap of the obstacle contour, it is determined that the flight speed of the drone is adjusted to the corresponding value based on the degree of overlap.

[0051] Specifically, the preset number of obstacles is selected within the range of [5, 8].

[0052] Specifically, the process of obtaining the obstacle contour based on the image information is not limited. The obstacle contour can be extracted by image preprocessing and using an edge detection algorithm, which is an existing technology and will not be elaborated here.

[0053] Specifically, the degree of overlap between the contour information and the obstacle contour is the ratio of the area of the overlapping region of the contour information and the obstacle contour to the sum of the areas of the overlapping parts of the contour information and the obstacle contour.

[0054] Specifically, for the stored obstacle contours, the learning module periodically performs repeated deletion, that is, only one of the obstacle contours with an overlap degree greater than 0.99 between two obstacle contours is retained.

[0055] Specifically, the number of abnormal obstacles reflects the complexity of the flight environment of the drone. When the number of abnormal obstacles is small, it indicates that the flight environment is relatively stable, and there is no need to perform special processing on the abnormal obstacles, and the current operating parameters can continue to be used. When the number of abnormal obstacles is large, it indicates that the flight environment is relatively complex, and it is necessary to learn and store the characteristics of the abnormal obstacles, that is, the obstacle contours, so as to identify these obstacles during future flights. The contour overlap degree characterizes the similarity between the currently recognized obstacle and the stored abnormal obstacles. The higher the overlap degree, the more likely the currently encountered obstacle is a dangerous obstacle encountered before. When the overlap degree is greater than the preset overlap degree, the flight speed is reduced to increase the reaction time and ensure flight safety. Regular repeated deletion operations are performed to reduce storage redundancy and improve the storage efficiency and recognition efficiency of the system. The drone system is thus equipped with learning and adaptive capabilities. By learning and storing abnormal obstacles, it is possible to identify dangerous obstacles in advance during future flights and take corresponding measures to reduce the probability of accidents. Adjusting the flight speed according to the overlap degree makes the speed adjustment more accurate, ensures flight safety, and further improves flight efficiency.

[0056] Specifically, the control module is used to adjust the flight speed of the drone to the corresponding value based on the overlap degree, where The reduction amplitude of the flight speed is proportional to the overlap degree.

[0057] In this embodiment, optionally, Compare the overlap degree with a first preset overlap degree and a second preset overlap degree; If the overlap degree is less than or equal to the first preset overlap degree, the flight speed is adjusted to 0.95 times the initial flight speed; If the degree of coincidence is less than or equal to the second preset degree of coincidence and greater than the first preset degree of coincidence, the flight speed is adjusted to 0.87 times the initial flight speed; If the degree of coincidence is greater than the second preset degree of coincidence, the flight speed is adjusted to 0.81 times the initial flight speed; The first preset degree of coincidence is 0.86, and the second preset degree of coincidence is 0.95.

[0058] So far, the technical solution of the present invention has been described in conjunction with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

[0059] The above are only the preferred embodiments of the present invention and are not used to limit the present invention; for those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A low-altitude UAV adaptive control system based on reinforcement learning, characterized in that: include: An environment detection module, which includes a plurality of radar transmitters arranged on the surface of the drone for obtaining the actual distance between the drone and obstacles and a camera arranged on the surface of the drone for obtaining image information; The flight simulation module is used to simulate and calculate the estimated distance between the drone and obstacles at each time point based on the flight angle and flight speed of the drone; A data fitting module, which is connected to the flight simulation module and the environment detection module respectively, and is used to draw an actual distance time domain curve based on the actual distance and draw an estimated distance time domain curve based on the estimated distance; an analysis module, which is respectively connected to the environment detection module, the flight simulation module and the data fitting module, and is used to periodically determine a driving deviation parameter based on the actual distance time domain curve and the estimated distance time domain curve, determine whether the operating parameters of the UAV are qualified based on the driving deviation parameter, and, when determining that the driving of the UAV is abnormal, adjust the operating parameters of the UAV based on the driving difference; A learning module, which is connected to the environment detection module and the analysis module respectively, and is used to determine whether to store each obstacle contour based on the number of obstacles marked as abnormal obstacles by the analysis module periodically, and to determine whether to adjust the flight speed of the UAV to a corresponding value based on the overlap degree when the overlap degree between the contour information identified based on the image information and the obstacle contour is greater than a preset overlap degree; A control module is connected to the analysis module and the learning module respectively, and is used to control the flight speed, flight angle or flight altitude of the UAV based on the determination results of the analysis module and the learning module.

2. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 1 is characterized in that: The analysis module is used to determine whether the operating parameters of the drone are qualified based on the driving deviation parameter when the operating time of the drone reaches the preset monitoring time, including: The actual distance time domain curve and the estimated distance time domain curve are overlapped based on the time axis, and the area enclosed by the two overlapped curves within the preset monitoring time is calculated to obtain the driving deviation parameter; If the driving deviation parameter is less than or equal to the first preset driving deviation parameter, it is determined that the operating parameters of the UAV are qualified, and the UAV is controlled to operate with the current operating parameters; If the driving deviation parameter is less than or equal to the second preset driving deviation parameter and greater than the first preset driving deviation parameter, determining whether the operating parameters of the UAV are qualified based on the driving floating parameter; If the driving deviation parameter is greater than the second preset driving deviation parameter, it is determined that the driving of the UAV is abnormal, and the operating parameters of the UAV are adjusted based on the driving difference.

3. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 2 is characterized in that: The analysis module is used to determine whether the operating parameters of the UAV are qualified based on the driving floating parameters, including: Solve the variance of the driving deviation parameter within each preset monitoring time to obtain the driving floating parameter; If the driving floating parameter is less than or equal to the preset driving floating parameter, it is determined that the flight speed of the UAV is adjusted to a corresponding value based on the average value of the driving deviation parameters within each preset monitoring time; If the driving floating parameter is greater than the preset driving floating parameter, the driving of the UAV is determined to be abnormal, and the operating parameters of the UAV are adjusted based on the driving difference.

4. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 3 is characterized in that: The control module is used to adjust the flight speed of the UAV to a corresponding value based on the average value of the driving deviation parameter within each preset monitoring time, wherein: The average value of the driving deviation parameter within each preset monitoring time is recorded as the average deviation parameter; The reduction in flight speed is proportional to the mean deviation parameter.

5. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 4 is characterized in that: The environment detection module also includes a wind speed sensor disposed on the surface of the drone for detecting wind speed; The analysis module adjusts the operating parameters of the drone based on the driving difference when determining that the drone is driving abnormally, including: Solving the difference between the driving deviation parameter and the second preset driving deviation parameter to obtain a driving difference amount; If the driving difference is less than or equal to the first preset driving difference, it is determined that the driving height of the drone is adjusted to a corresponding value based on the acquired approach speed of the obstacle; If the driving difference is less than or equal to the second preset driving difference and greater than the first preset driving difference, adjusting the operating parameters of the UAV based on the acquired wind speed; If the driving difference is greater than a second preset driving difference, the obstacle is marked as an abnormal obstacle.

6. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 5 is characterized in that: The control module is used to adjust the driving height of the drone to a corresponding value based on the acquired obstacle approach speed, wherein: The increase in ride height is proportional to the approach speed.

7. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 6 is characterized in that: The analysis module is used to adjust the operating parameters of the drone based on the acquired wind speed, including: If the wind speed is less than or equal to the preset wind speed, it is determined that the driving altitude of the UAV is adjusted to a corresponding value based on the obtained obstacle approach speed; If the wind speed is greater than the preset wind speed, the flight angle of the drone will be adjusted to the corresponding value based on the wind speed.

8. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 7 is characterized in that: The learning module is used to determine whether to store each obstacle profile for each abnormal obstacle based on the number of obstacles marked as abnormal obstacles obtained within the preset learning time when the operating time of the drone reaches the preset learning time; If the number of abnormal obstacles is less than or equal to the preset number of obstacles, the drone will be controlled to continue to operate using the current operating parameters; If the number of abnormal obstacles is greater than the preset number of obstacles, the obstacle contours of each abnormal obstacle are obtained based on the image information of each abnormal obstacle, and each obstacle contour is stored; The learning module determines to adjust the flight speed of the UAV to a corresponding value based on the overlap, under the condition that the overlap between the contour information identified based on the image information and the obstacle contour is greater than a preset overlap.

9. The low-altitude UAV adaptive control system based on reinforcement learning according to claim 8 is characterized in that: The control module is used to adjust the flight speed of the drone to a corresponding value based on the overlap, wherein: The reduction in flight speed is proportional to the overlap.

10. A low-altitude UAV adaptive control method using the low-altitude UAV adaptive control system according to any one of claims 1 to 9, characterized in that: include: S1, based on the flight angle and flight speed of the UAV, simulates and calculates the estimated distance between the UAV and the obstacle at each time node; S2, obtain the actual distance between the drone and the obstacle; S3, drawing an actual distance time domain curve based on the actual distance, drawing an estimated distance time domain curve based on the estimated distance, and determining a driving deviation parameter based on the two curves; S4, determining whether the operating parameters of the UAV are qualified based on the driving deviation parameter, including determining whether the UAV is driving abnormally, and adjusting the operating parameters of the UAV based on the driving difference, wherein adjusting the operating parameters of the UAV includes marking obstacles, adjusting the flight speed, flight angle or flight altitude of the UAV; Or, determine that the operating parameters of the drone are qualified, and control the drone to operate with the current operating parameters; S5: Determine whether to store each obstacle profile based on the acquired number of obstacles marked as abnormal.

Images