Unmanned aerial vehicle automatic landing control method and system

Abstract

The invention relates to the field of unmanned aerial vehicle control, and provides an unmanned aerial vehicle automatic landing control method, which comprises the following steps of: initializing an IMU (Inertial Measurement Unit) sensor, setting an initial attitude and a speed value, starting a binocular camera system and the IMU sensor on an unmanned aerial vehicle, calibrating a binocular camera, and ensuring the consistency and accuracy of image acquisition. The unmanned aerial vehicle can accurately identify ground markers and calculate accurate position and attitude information relative to the ground by adopting a data fusion technology of a binocular camera system and an IMU (Inertial Measurement Unit) sensor, so that not only is the positioning accuracy improved, but also the unmanned aerial vehicle can realize a more stable and accurate automatic landing process under a complex environment condition, and the unmanned aerial vehicle is more convenient to use. In addition, data processing is carried out in combination with an extended Kalman filter, the robustness of the system is further enhanced, it is ensured that good control performance can be kept even in the face of external interference, and therefore the safety and reliability of landing are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle control, and in particular to an automatic landing control method and system for unmanned aerial vehicle. Background Art

[0002] Against the backdrop of the rapid development of drone technology, automatic landing control has become a key research area. Traditional drone landing methods mainly rely on sensors such as GPS positioning and barometers to estimate altitude and position information, but these methods have limited accuracy in complex environments and are easily affected by signal interference. With the advancement of computer vision technology and inertial measurement units, automatic landing systems based on the fusion of vision and IMU have gradually demonstrated their unique advantages. Early vision-assisted landing solutions mostly used monocular cameras combined with landmark recognition technology. Although they could provide a certain position reference, there were large errors in distance estimation.

[0003] Therefore, a method and system for automatically landing a drone are now developed to address the above problems. Summary of the invention

[0004] In order to overcome the shortcomings of traditional UAV landing methods that mainly rely on GPS positioning and sensors such as barometers to estimate altitude and position information, but these methods have limited accuracy in complex environments and are easily affected by signal interference, the present invention provides a UAV automatic landing control method and system.

[0005] The technical solution of the present invention is: a method for controlling automatic landing of a drone, comprising:

[0006] Step 1: Initialize the IMU sensor, set the initial attitude and speed values, start the binocular camera system and IMU sensor on the drone, calibrate the binocular camera, and ensure the consistency and accuracy of image acquisition;

[0007] Step 2: Use a binocular camera to capture images of the ground and surrounding environment, apply a stereo matching algorithm to calculate the depth information of the target area, identify ground landmarks through feature point detection and tracking technology, and estimate the position and posture of the drone relative to the ground;

[0008] Step 3: The IMU continuously collects data from the accelerometer and gyroscope, calculates the speed change and rotation angle of the drone based on the IMU data, and synchronizes and updates the IMU data with the visual positioning data;

[0009] Step 4: Dynamically adjust the flight controller parameters based on the fused data, calculate and update the descent trajectory of the drone to ensure that it can descend smoothly along the predetermined path. When approaching the ground, reduce the descent speed to ensure a safe landing.

[0010] As a preferred embodiment of the present invention, when performing stereo matching calculation and feature point detection, the following formula can be used for calculation:

[0011]

[0012] Among them, I L (x, y) is the pixel value of the left image at position (x, y), I R (x, y) is the pixel value of the right image at position (x, y), d is the disparity, that is, the horizontal displacement between the corresponding points in the left and right images, w is the width of the matching window, h is the height of the matching window, and i is the compensation offset parameter. The optimal disparity is calculated according to the above formula, and then the depth value is converted based on the following formula:

[0013]

[0014] Among them, f is the focal length of the camera, and B is the baseline length, that is, the distance between the two cameras. The combined calculation is finally expressed as:

[0015]

[0016] Finally, the formula is integrated as follows:

[0017]

[0018] As a preferred embodiment of the present invention, the speed estimation algorithm and the attitude estimation algorithm are calculated according to the following formula:

[0019] The acceleration and angular velocity obtained from the IMU are defined as a t And ω t , where t represents time, the specific formula for speed estimation is:

[0020] where v t-1 is the velocity at the previous moment, a τ is the acceleration in the time interval [t-1, t];

[0021] The specific formula for posture estimation is:

[0022] R t =R t-1 ·exp(ω t Δt), where R t-1 is the posture rotation matrix of the previous moment, and Δt is the time step.

[0023] As a preferred embodiment of the present invention, the state update formula is:

[0024] in, is the state estimation data, F is the state transfer matrix, B is the control input matrix, u t is the control input data, K t is the Kalman gain data, z t is the visual positioning data, and H is the observation matrix.

[0025] As a preferred embodiment of the present invention, the formula for dynamically adjusting the controller parameters is as follows:

[0026]

[0027] Where K P , K I , K D They are proportional, integral, and differential gains, e t is the difference between the target trajectory and the actual trajectory, i.e., the error signal.

[0028] As a preferred embodiment of the present invention, the calculation method of the descending trajectory is:

[0029]

[0030] Among them, p des，t is the target position, v des，t is the target speed, a des，t is the target acceleration.

[0031] As a preferred embodiment of the present invention, before performing control operations, ensure that the IMU sensor is correctly connected to the power and data interfaces of the drone, check whether all connections are firm to avoid data loss or errors caused by looseness, place the drone on a horizontal and stable platform, ensure that it is stationary, calculate the initial attitude angle through the gravity acceleration component, and finally calculate the initial yaw angle through gyroscope integration.

[0032] An automatic landing control system for unmanned aerial vehicles includes a visual subsystem, an inertial navigation subsystem, a data fusion module, a controller adjustment module and an emergency response module. The visual subsystem uses a binocular camera as a main sensing device, obtains three-dimensional coordinate information of a target area through a stereoscopic vision algorithm, and uses computer vision technologies such as feature matching to identify ground markers, and estimates the position and posture of the unmanned aerial vehicle relative to the ground. The inertial navigation subsystem is responsible for collecting data from an accelerometer and a gyroscope for calculating the speed change and rotation angle of the unmanned aerial vehicle. The data fusion module uses an extended Kalman filter to integrate and process data from the visual subsystem and an IMU to generate a more accurate position estimation result. The controller adjustment module dynamically adjusts flight controller parameters according to the fused data to ensure that the unmanned aerial vehicle can smoothly descend along a predetermined trajectory until it lands successfully. The emergency response module is designed with rapid response measures for abnormal situations such as strong wind interference to ensure that a safe landing can be completed even under adverse conditions.

[0033] By adopting the above technical solution, the present invention has the following advantages:

[0034] 1. The present invention adopts the data fusion technology of binocular camera system and IMU sensor, so that the UAV can accurately identify ground markers and calculate the precise position and attitude information relative to the ground, which not only improves the positioning accuracy, but also enables the UAV to achieve a more stable and accurate automatic landing process under complex environmental conditions. In addition, the extended Kalman filter is combined for data processing to further enhance the robustness of the system, ensuring that good control performance can be maintained even in the face of external interference, thereby significantly improving the safety and reliability of landing.

[0035] 2. The present invention responds to changes in demand under different flight conditions by dynamically adjusting flight controller parameters. The emergency response module can quickly initiate corresponding safety measures to ensure that the UAV can adjust the flight status in time to avoid potential risks. By using feature matching algorithms and deep learning models, the UAV has stronger environmental perception capabilities and can adapt to various terrain characteristics more flexibly. It can successfully complete tasks in both open areas and densely populated urban areas, greatly broadening the application scenarios of UAVs. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The figure is a schematic flow chart of the steps of the automatic landing control method of the unmanned aerial vehicle of the present invention.

[0037] Figure 2 It is a structural schematic diagram of the automatic landing control system of the unmanned aerial vehicle of the present invention. DETAILED DESCRIPTION

[0038] Reference to an embodiment herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0039] A method for controlling automatic landing of a UAV, such as Figure 1 As shown, including:

[0040] Step 1: Initialize the IMU sensor, set the initial attitude and speed values, start the binocular camera system and IMU sensor on the drone, calibrate the binocular camera, and ensure the consistency and accuracy of image acquisition;

[0041] Step 2: Use a binocular camera to capture images of the ground and surrounding environment, apply a stereo matching algorithm to calculate the depth information of the target area, identify ground landmarks through feature point detection and tracking technology, and estimate the position and posture of the drone relative to the ground;

[0042] Step 3: The IMU continuously collects data from the accelerometer and gyroscope, calculates the speed change and rotation angle of the drone based on the IMU data, and synchronizes and updates the IMU data with the visual positioning data;

[0043] Step 4: Dynamically adjust the flight controller parameters based on the fused data, calculate and update the descent trajectory of the drone to ensure that it can descend smoothly along the predetermined path. When approaching the ground, reduce the descent speed to ensure a safe landing.

[0044] As a preferred embodiment of the present invention, when performing stereo matching calculation and feature point detection, the following formula can be used for calculation:

[0045]

[0046] Among them, I L (x, y) is the pixel value of the left image at position (x, y), I R (x, y) is the pixel value of the right image at position (x, y), d is the disparity, that is, the horizontal displacement between the corresponding points in the left and right images, w is the width of the matching window, h is the height of the matching window, and i is the compensation offset parameter. The optimal disparity is calculated according to the above formula, and then the depth value is converted based on the following formula:

[0047]

[0048] Among them, f is the focal length of the camera, and B is the baseline length, that is, the distance between the two cameras. The combined calculation is finally expressed as:

[0049]

[0050] Finally, the formula is integrated as follows:

[0051]

[0052] As a preferred embodiment of the present invention, the speed estimation algorithm and the attitude estimation algorithm are calculated according to the following formula:

[0053] The acceleration and angular velocity obtained from the IMU are defined as a t And ω t , where t represents time, the specific formula for speed estimation is:

[0054] where v t-1 is the velocity at the previous moment, a τ is the acceleration in the time interval [t-1, t];

[0055] The specific formula for posture estimation is:

[0056] R t =R t-1 ·exp(ω t Δt), where R t-1 is the posture rotation matrix of the previous moment, and Δt is the time step.

[0057] As a preferred embodiment of the present invention, the state update formula is:

[0058] in, is the state estimation data, F is the state transfer matrix, B is the control input matrix, u t is the control input data, K t is the Kalman gain data, z t is the visual positioning data, and H is the observation matrix.

[0059] As a preferred embodiment of the present invention, the formula for dynamically adjusting the controller parameters is as follows:

[0060]

[0061] Where K P , K I , K D They are proportional, integral, and differential gains, e t is the difference between the target trajectory and the actual trajectory, i.e., the error signal.

[0062] As a preferred embodiment of the present invention, the calculation method of the descending trajectory is:

[0063]

[0064] Among them, p des，t is the target position, v des，t is the target speed, a des，t is the target acceleration.

[0065] As a preferred embodiment of the present invention, before performing control operations, ensure that the IMU sensor is correctly connected to the power and data interfaces of the drone, check whether all connections are firm to avoid data loss or errors caused by looseness, place the drone on a horizontal and stable platform, ensure that it is stationary, calculate the initial attitude angle through the gravity acceleration component, and finally calculate the initial yaw angle through gyroscope integration.

[0066] like Figure 2 As shown, an automatic landing control system for unmanned aerial vehicles includes a visual subsystem, an inertial navigation subsystem, a data fusion module, a controller adjustment module and an emergency response module. The visual subsystem uses a binocular camera as the main perception device, obtains the three-dimensional coordinate information of the target area through a stereo vision algorithm, and uses computer vision technologies such as feature matching to identify ground markers, and estimates the position and posture of the unmanned aerial vehicle relative to the ground. The inertial navigation subsystem is responsible for collecting data from the accelerometer and gyroscope to calculate the speed change and rotation angle of the unmanned aerial vehicle. The data fusion module uses an extended Kalman filter to integrate and process the data from the visual subsystem and the IMU to generate a more accurate position estimation result. The controller adjustment module dynamically adjusts the flight controller parameters according to the fused data to ensure that the unmanned aerial vehicle can smoothly descend along the predetermined trajectory until it lands successfully. The emergency response module is designed with rapid response measures for abnormal situations such as strong wind interference to ensure that a safe landing can be completed even under adverse conditions.

[0067] In summary, by adopting the data fusion technology of binocular camera system and IMU sensor, the UAV can accurately identify ground landmarks and calculate the precise position and attitude information relative to the ground, which not only improves the positioning accuracy, but also enables the UAV to achieve a smoother and more accurate automatic landing process under complex environmental conditions. In addition, the extended Kalman filter is used for data processing to further enhance the robustness of the system, ensuring that good control performance can be maintained even in the face of external interference, thereby significantly improving the safety and reliability of landing. At the same time, by dynamically adjusting the flight controller parameters to cope with changes in demand under different flight conditions, the emergency response module can quickly initiate corresponding safety measures to ensure that the UAV can adjust the flight status in time to avoid potential risks. The feature matching algorithm and deep learning model are used to enable the UAV to have a stronger environmental perception ability and can adapt to various terrain characteristics more flexibly. Whether it is an open area or a densely populated urban area, it can successfully complete the task, greatly broadening the application scenarios of UAVs.

[0068] The above embodiments are provided for persons familiar with the art to implement or use the present invention. Personnel familiar with the art can make various modifications or changes to the above embodiments without departing from the concept of the present invention. Therefore, the protection scope of the present invention is not limited to the above embodiments, but should be the maximum scope of the innovative features mentioned in the claims.

Claims

1. A method for controlling automatic landing of a drone, characterized in that: Included are: Step 1: Initialize the IMU sensor, set the initial attitude and speed values, start the binocular camera system and IMU sensor on the drone, calibrate the binocular camera to ensure the consistency and accuracy of image acquisition; Step 2: Use a binocular camera to capture images of the ground and surrounding environment, apply a stereo matching algorithm to calculate the depth information of the target area, identify ground landmarks through feature point detection and tracking technology, and estimate the position and posture of the drone relative to the ground; Step 3: The IMU continuously collects data from the accelerometer and gyroscope, calculates the speed change and rotation angle of the drone based on the IMU data, and synchronizes and updates the IMU data with the visual positioning data; Step 4: Dynamically adjust the flight controller parameters based on the fused data, calculate and update the descent trajectory of the drone to ensure that it can descend smoothly along the predetermined path. When approaching the ground, reduce the descent speed to ensure a safe landing.

2. A method for controlling automatic landing of a UAV according to claim 1, characterized in that: When performing stereo matching calculations and feature point detection, the following formula can be used for calculation: Among them, I L (x, y) is the pixel value of the left image at position (x, y), I R (x, y) is the pixel value of the right image at position (x, y), d is the disparity, that is, the horizontal displacement between the corresponding points in the left and right images, w is the width of the matching window, h is the height of the matching window, and i is the compensation offset parameter. The optimal disparity is calculated according to the above formula, and then the depth value is converted based on the following formula: Among them, f is the focal length of the camera, and B is the baseline length, that is, the distance between the two cameras. The combined calculation is finally expressed as: Finally, the formula is integrated as follows:

3. A method and system for controlling automatic landing of a UAV according to claim 1, characterized in that: The velocity estimation algorithm and the attitude estimation algorithm are calculated according to the following formula: The acceleration and angular velocity obtained from the IMU are defined as a t And ω t , where t represents time, the specific formula for speed estimation is: where v t-1 is the velocity at the previous moment, a τ is the acceleration in the time interval [t-1, t]; The specific formula for posture estimation is: R t =R t-1 ·exp(ω t Δ t ), where R t-1 is the posture rotation matrix of the previous moment, and Δt is the time step.

4. A method and system for controlling automatic landing of a UAV according to claim 1, characterized in that: The state update formula is: in, is the state estimation data, F is the state transfer matrix, B is the control input matrix, u t is the control input data, K t is the Kalman gain data, z t is the visual positioning data, and H is the observation matrix.

5. A method and system for controlling automatic landing of a UAV according to claim 1, characterized in that: The formula for dynamically adjusting controller parameters is as follows: Where K P , K I , K D They are proportional, integral, and differential gains, e t is the difference between the target trajectory and the actual trajectory, i.e., the error signal.

6. A method and system for controlling automatic landing of a UAV according to claim 1, characterized in that: The descent trajectory is calculated as: Among them, p des，t is the target position, v des，t is the target speed, a des，t is the target acceleration.

7. A method for controlling automatic landing of a UAV according to claim 1, characterized in that: Before performing control operations, make sure that the IMU sensor is correctly connected to the power and data interfaces of the drone, check whether all connections are firm to avoid data loss or errors caused by looseness, place the drone on a level and stable platform, ensure that it is stationary, calculate the initial attitude angle through the gravity acceleration component, and finally calculate the initial yaw angle through gyroscope integration.

8. An automatic landing control system for a drone, characterized in that: It includes a visual subsystem, an inertial navigation subsystem, a data fusion module, a controller adjustment module and an emergency response module. The visual subsystem uses a binocular camera as the main perception device, obtains the three-dimensional coordinate information of the target area through a stereo vision algorithm, uses computer vision technologies such as feature matching to identify ground markers, and estimates the position and posture of the drone relative to the ground. The inertial navigation subsystem is responsible for collecting data from the accelerometer and gyroscope to calculate the speed change and rotation angle of the drone. The data fusion module uses an extended Kalman filter to integrate and process the data from the visual subsystem and IMU to generate a more accurate position estimation result. The controller adjustment module dynamically adjusts the flight controller parameters according to the fused data to ensure that the drone can descend smoothly along the predetermined trajectory until it lands successfully. The emergency response module is designed with rapid response measures for abnormal situations such as strong wind interference to ensure that a safe landing can be completed even under adverse conditions.

Images