Dynamic platform unmanned aerial vehicle autonomous tracking and landing method based on vision
Through the integration of visual guidance and multi-sensor information, the problem of low landing accuracy of drones in complex environments is solved, efficient and accurate dynamic platform autonomous tracking and landing of drones is achieved, and landing accuracy and safety are improved.
Patent Information
- Application Number
- CN202411139971.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-20
AI Technical Summary
The existing drone autonomous landing method has low drop accuracy in environmental impact, and the accumulated positioning error of GPS or inertial navigation systems in complex environments cannot guarantee landing accuracy, especially in densely packed areas, which may lead to signal loss and landing failure.
The autonomous tracking and landing method of the dynamic platform drone based on vision is adopted. Visual guidance is activated after preliminary positioning of GPS, image processing is used to obtain the coordinates of the landing platform, and speed estimation is carried out in combination with local weighted K proximity algorithm and attenuation memory volume Kalman filtering algorithm. Trajectory prediction, fuse multi-sensor information to improve landing accuracy.
It realizes efficient and accurate drone landing in a dynamic environment, breaks away from the position and speed limitations of the ground platform, improves the landing accuracy, and controls the average error within 12 cm, surpassing the accuracy of traditional methods.
Smart Images

Figure CN118938960A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of autonomous landing of unmanned aerial vehicles (UAVs), and in particular to a vision-based autonomous tracking and landing method for a dynamic platform UAV. Background Art
[0002] In recent years, air-ground collaboration has garnered widespread attention from the academic community due to its ability to execute more intelligent missions in complex and diverse environments with greater efficiency, flexibility, and robustness. Using unmanned ground vehicles to charge drones or transport them to target areas and recover them after mission completion effectively overcomes the shortcomings of drones' limited endurance. The ability for drones to quickly and accurately land on a ground platform for charging within a confined space is a prerequisite for multiple autonomous flights. Therefore, achieving stable and accurate landing at a designated location—in other words, achieving autonomous landing—is crucial for improving drone autonomy. However, the current common drone autonomous landing methods have two flaws: first, most solutions require the ground platform to be parked at a pre-selected location, remaining stationary or moving at a constant speed, resulting in low landing efficiency and difficulty ensuring uniform motion due to environmental influences; second, the currently commonly used navigation methods are based on the Global Positioning System (GPS) or Inertial Navigation System (INS). When using the INS, positioning errors accumulate over time, making it impossible to guarantee landing accuracy and affecting the flight safety of the drone. As for GPS, the positioning accuracy of commercial-grade GPS can only reach the meter level, which cannot meet the navigation and landing requirements in complex environments such as densely built cities. If there are too many obstructions, the GPS signal may even be lost, resulting in a landing failure. Summary of the Invention
[0003] This application provides a vision-based dynamic platform UAV autonomous tracking and landing method, the technical purpose of which is to improve the landing accuracy of the UAV.
[0004] The above technical objectives of this application are achieved through the following technical solutions:
[0005] A vision-based autonomous tracking and landing method for a dynamic platform UAV, comprising:
[0006] Step 1: Obtain GPS information of the landing platform on the unmanned vehicle and perform preliminary positioning. The drone flies to the vicinity of the landing platform based on the preliminary positioning and then starts the visual guidance program.
[0007] Step 2: The drone detects whether the landing platform is in its field of view. If so, it proceeds to step 3, otherwise it returns to step 1.
[0008] Step 3: Detect the landing platform's logo, perform image processing on the logo to obtain the landing platform's coordinates in the pixel coordinate system, and convert the pixel coordinates into the aircraft coordinate system;
[0009] Step 4: Estimate the speed of the landing platform relative to the drone based on the drone's coordinates and time interval at the two moments before and after. Use the local weighted K-nearest neighbor algorithm to eliminate abnormal speed data and obtain the estimated speed.
[0010] Step 5: Based on the coordinates of the landing platform's marker in the aircraft coordinate system, the decaying memory volumetric Kalman filter algorithm is used to predict the landing platform's trajectory;
[0011] Step 6: Based on the estimated speed of the drone and the trajectory of the landing platform, control the drone to fly towards the center of the landing platform;
[0012] Step 7: Fuse the multi-sensor information to obtain the height of the drone from the landing platform. If the drone reaches the landing height, proceed to step 8. Otherwise, descend 0.2 meters and return to step 2.
[0013] Step 8: The drone's rotors stop and the electromagnet turns on to attract the landing platform, completing the landing.
[0014] The beneficial effects of this application are as follows: the vision-based dynamic platform drone autonomous tracking and landing method described in this application breaks away from the limitations of the original drone landing solution on the landing platform position and speed, does not require the ground platform to be parked at a pre-selected position and remain stationary or move at a constant speed, is less affected by the environment, and has high landing efficiency; uses visual guidance, which is more accurate than GPS and INS, and does not affect landing accuracy due to signal weakening caused by surrounding obstructions. This application optimizes speed estimation through a local weighted K-nearest neighbor algorithm, uses a decaying memory volumetric Kalman filter for trajectory prediction, and adaptively fuses multi-source sensor information to estimate drone height. It performs well in speed estimation, trajectory prediction, and landing accuracy. The average error in trajectory prediction is 28% higher than that of the standard volumetric Kalman filter method, and the average landing offset can be controlled within 12 cm. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 This is a flow chart of the vision-based autonomous tracking and landing method for a dynamic platform drone in an embodiment of the present application;
[0016] Figure 2a : is a diagram showing the conversion relationship between the pixel coordinate system and the image coordinate system in the embodiment of the present application;
[0017] Figure 2b : is a diagram showing the conversion relationship between the image coordinate system and the camera coordinate system in the embodiment of the present application;
[0018] Figure 3 This is a comparison diagram of speed estimation effects in the embodiments of this application;
[0019] Figure 4 This is a comparison diagram of trajectory prediction errors in the embodiment of this application;
[0020] Figure 5 This is a diagram showing the landing accuracy effect in an embodiment of the present application. DETAILED DESCRIPTION
[0021] The technical solution of this application will be described in detail below with reference to the accompanying drawings.
[0022] This application takes a six-rotor drone performing an autonomous landing mission as an example. The camera is fixed to the drone vertically downward, with the centers 30 cm apart. The x-axis direction of the camera is the same as the y-axis direction of the drone. An AprilTag logo consisting of an outer white border and an inner 8×8 grid containing binary information is affixed to the center of the landing platform.
[0023] The process of the vision-based dynamic platform UAV autonomous tracking and landing method described in this application is as follows: Figure 1 As shown, specifically including:
[0024] Step 1: Obtain the GPS information of the landing platform on the unmanned vehicle and perform preliminary positioning. The drone flies to the vicinity of the landing platform based on the preliminary positioning and then starts the visual guidance program.
[0025] First, the landing platform uses its own GPS to obtain its own latitude and longitude. The drone obtains the platform's latitude and longitude information through communication between the landing platform and the drone. After the drone flies to the corresponding latitude and longitude position, it starts the visual guidance program.
[0026] Step 2: The drone detects whether the landing platform is in its field of view. If so, it proceeds to step 3; otherwise, it returns to step 1.
[0027] Specifically, the logo attached to the landing platform is detected. If the logo is within the camera's field of view, the subsequent landing process is entered, and the logo information is further processed and responded to. If the logo is outside the camera's field of view, the drone rises to expand the field of view and counts. If the logo is found within the set time range, the subsequent landing process is entered. If it times out, it returns to the previous step and re-communicates with the landing platform to obtain latitude and longitude information.
[0028] Step 3: Detect the landing platform's logo, perform image processing on the logo to obtain the coordinates of the landing platform in the pixel coordinate system, and convert the coordinates in the pixel coordinate system into coordinates in the body coordinate system.
[0029] After a series of image processing, including image grayscale, image thresholding, contour extraction, etc., the coordinates (u0, v0) of the logo center in the pixel coordinate system are obtained.
[0030] like Figure 2a As shown, the coordinates are converted from the pixel coordinate system (P) to the image coordinate system (I), which is expressed as:
[0031]
[0032] Among them, d x and d y Represents the length of the unit pixel in the x and y directions; (u, v) represents the coordinates of the logo center in the image coordinate system.
[0033] like Figure 2b As shown, the coordinates are converted from the image coordinate system to the camera coordinate system (C), which is expressed as:
[0034]
[0035] Where f represents the focal length of the camera; subscript C represents the coordinates in the camera coordinate system.
[0036] Convert the coordinates from the camera coordinate system to the body coordinate system (B), expressed as:
[0037]
[0038] The subscript B represents the coordinates in the body coordinate system.
[0039] Finally, the conversion formula from pixel coordinate system to body coordinate system is as follows:
[0040]
[0041] Step 4: Estimate the speed of the landing platform relative to the drone based on the drone body coordinates and time intervals at the two moments before and after. Use the local weighted K-nearest neighbor algorithm to eliminate abnormal speed data and obtain the estimated speed.
[0042] Specifically, step 4 includes:
[0043] Step 41: Obtain the coordinates of the drone body in the two frames of image at two different moments, and perform differential calculation on the coordinates of the drone body to obtain the speed of the landing platform relative to the drone, which is expressed as:
[0044]
[0045] in, It represents the speed of the landing platform relative to the drone on the x-axis in the body coordinate system at time t; Indicates the speed of the landing platform relative to the drone on the y-axis in the body coordinate system at time t; represents the x-axis coordinate of the drone body in the body coordinate system at time t; represents the x-axis coordinate of the drone body in the body coordinate system at time t-1; represents the UAV body coordinate of the y-axis in the body coordinate system at time t; represents the y-axis coordinate of the drone body in the body coordinate system at time t-1; Δt represents the time interval between time t and time t-1;
[0046] Step 42: When the relative speed of the landing platform is high, the target's position may vary significantly between frames. This can lead to large errors in velocity estimation, causing the calculated relative speed to be too high or too low, affecting the tracking of the drone. Anomalies in estimated velocity are considered contextual anomalies in time series data. Based on KNN, we propose a local weighted anomaly detection algorithm and a dynamic threshold to detect and eliminate anomalous data.
[0047] Specifically, the local weighted K-nearest neighbor algorithm is used to eliminate abnormal speed data, including: comparing the weighted average distance with the dynamic threshold. If the weighted average distance is greater than the dynamic threshold, it means that the estimated speed is abnormal, and the abnormal speed is eliminated. Then, the estimated speed at the previous moment is used to replace the abnormal speed. Specifically, it is expressed as:
[0048]
[0049] in, Represents the weighted average distance between the current sample data and the data points at the previous K moments; Represents the weighted average of the previous K data points; ω t-i represents the weight at time ti; λ represents the decay rate; T dyn represents the current threshold, i.e., the dynamic threshold; α represents the dynamic coefficient.
[0050] Local weighted KNN performs anomaly detection based on distance. It calculates the weighted average distance between a sample and its nearest K samples and compares the calculated weighted average distance with a dynamic threshold (the product of the weighted average of the nearest K samples and the dynamic coefficient). If the weighted average distance is greater than the threshold, the corresponding sample is considered abnormal, and the abnormal speed is then replaced by the estimated speed value of the previous moment.
[0051] Step 5: Based on the coordinates of the landing platform's marker in the body coordinate system, the decaying memory volumetric Kalman filter algorithm is used to predict the motion trajectory of the landing platform.
[0052] Aiming at the limitation of camera frame rate and the problem of sudden target loss, a position estimation method based on attenuated memory volumetric Kalman filter is proposed. The process includes prediction step, update step and estimation step. The premise is set as follows: the state vector Covariance matrix P k , process noise Q and measurement noise R. In this application, an attenuation factor b is introduced to process Q and R, which is expressed as:
[0053]
[0054] Specifically, step 5 includes:
[0055] Step 51: In the prediction step, the volume points are first calculated, which is expressed as:
[0056]
[0057] Among them, P k represents the covariance matrix; represents the state vector; represents the volume point; n represents the dimension of the state vector; Represents the volume point set, [I]=[I n ,-I n ];
[0058] Then the state of the volume point is predicted, which is expressed as:
[0059] in, represents the volume point where the output propagates through the state equation;
[0060] The target state vector and error covariance matrix are predicted based on the volume point state, which can be expressed as:
[0061]
[0062] in, represents the predicted value of the target state vector; P k+1|k represents the predicted value of the error covariance matrix; Q represents the process noise, b represents the attenuation factor;
[0063] Step 52: In the update step, the volume points are calculated based on the target state vector and the predicted value of the error covariance matrix in the prediction step, which is expressed as:
[0064]
[0065] The volume point measurement is predicted based on the volume point, which is expressed as: in, represents the volume point where the output propagates through the measurement equation;
[0066] The target measurement is predicted based on the volume point measurement, which is expressed as:
[0067]
[0068] in, represents the predicted value of the next measurement step;
[0069] The measurement error covariance, measurement cross-covariance moment and Kalman gain matrix are calculated and expressed as:
[0070]
[0071] in, represents the error covariance matrix, represents the cross-covariance matrix prediction value, K k+1 represents the Kalman gain matrix; R represents the measurement noise,
[0072] Step 53: In the estimation step, the state vector and the error covariance matrix are first estimated based on the Kalman gain matrix and the predicted value of the target state vector, which can be expressed as:
[0073]
[0074] in, Represents the state vector at the current moment, P k+1 Represents the error covariance matrix at the current moment;
[0075] Step 54: Repeat steps 51 to 54 according to the current state vector and error covariance matrix until the motion trajectory prediction of the landing platform is finally completed.
[0076] Step 6: Based on the estimated speed of the drone and the motion trajectory of the landing platform, control the drone to fly towards the center of the landing platform.
[0077] After obtaining the estimated velocity of the moving target, the speed control command for the drone is output. The drone's speed is composed of two parts: the estimated velocity of the dynamic target and the ratio of the target's offset relative to the camera center. By increasing the target's speed to keep the drone and the landing platform relatively stationary, the problem is simplified to a static landing. The target's offset is then used to control the drone to gradually approach the target, expressed as: v U =v p +μ*d; where subscript U represents the drone, subscript p represents the landing platform, and μ is in s -1 , and d is the horizontal distance between the landing platform and the UAV in the body coordinate system.
[0078] Step 7: Fuse the multi-sensor information to obtain the height of the drone from the landing platform. If the drone reaches the landing height, proceed to step 8. Otherwise, descend 0.2 meters and return to step 2.
[0079] During landing, altitude information is crucial to ensure timely stopping of the motors and activation of the electromagnets. This application proposes an adaptive weighted fusion algorithm that considers the sensor's inherent characteristics and actual measurement data, integrating information from multiple sensors, including cameras, laser rangefinders, and barometers, to detect altitude.
[0080] The weight consists of two parts: fixed weight and measurement weight. The fixed weight is determined by the sensor's own characteristics. The higher the sensor's accuracy, the greater its fixed weight proportion. At the same time, it also meets the following requirements:
[0081]
[0082] Among them, the subscript q=1, 2, 3 corresponds to three types of sensors, Represents the sensor fixed weight.
[0083] The measurement weight is calculated from the variance of the actual measurement data and is expressed as:
[0084]
[0085] Among them, the subscript q=1, 2, 3 corresponds to three types of sensors, represents the variance of each sensor data, Indicates the kth last data of the qth sensor, Represents the sensor measurement weight.
[0086] Finally, the fixed weights and measurement weights are combined to obtain the final weight of each sensor, which is expressed as:
[0087]
[0088] Among them, ω q represents the final weight of the sensor, α and β are the coefficients corresponding to the fixed weight and measurement weight respectively.
[0089] Step 8: The drone's rotors stop and the electromagnet turns on to attract the landing platform, completing the landing.
[0090] After it falls below the landing altitude, the drone's rotor motor stops and the electromagnet turns on to attract the landing platform to prevent the drone from falling from the landing platform and causing damage. At this point, the drone's autonomous landing process ends.
[0091] In order to verify the superiority of the proposed method, the velocity estimation experiment was compared with the difference-only method (without using K-nearest neighbors), and the estimation results were recorded; the trajectory prediction experiment was compared with the standard CKF method, and the estimation errors of the two were recorded; and the deviation between the UAV and the platform center was recorded in the landing experiment.
[0092] Speed estimation effect comparison Figure 3 As shown in the figure, it can be seen that the local weighted KNN anomaly detection eliminates the abnormal speed data that increases or decreases sharply, the estimated curve is smoother, and the tracking effect is improved; the trajectory prediction error is as follows Figure 4 As shown in the figure, it can be seen that during the landing process, the trajectory predicted by AMCKF is closer to the actual trajectory, and the prediction error is smaller than that of the standard CKF in most of the time. According to statistics, the average error of AMCKF is 0.0109, and the root mean square error is 0.01215. The average error of CKF is 0.01515, and the root mean square error is 0.01505. The method of the present invention has an improvement of 28% in average error and 17% in root mean square error compared with the standard CKF algorithm, and has higher accuracy. The landing accuracy is shown in the figure. Figure 5 As shown in the figure, the deviation of the drone in the x and y directions in each experiment was less than 12 cm. After statistics, the landing deviation was 7.85 cm and 7.05 cm in the x and y directions, respectively. In summary, the method of the present invention has improved speed estimation, trajectory prediction, and altitude estimation, ultimately helping to improve landing accuracy.
Claims
1. A vision-based dynamic platform UAV autonomous tracking and landing method, characterized in that: include: Step 1: Obtain the GPS information of the landing platform on the unmanned vehicle and perform preliminary positioning. The drone flies to the vicinity of the landing platform based on the preliminary positioning and then starts the visual guidance program; Step 2: The drone detects whether the landing platform is in its field of view. If so, it goes to step 3, otherwise it returns to step 1. Step 3: Detect the logo of the landing platform, perform image processing on the logo to obtain the coordinates of the landing platform in the pixel coordinate system, and convert the coordinates in the pixel coordinate system into the coordinates in the body coordinate system; Step 4: Estimate the speed of the landing platform relative to the drone based on the drone body coordinates and time interval at the previous and next two moments, and use the local weighted K-nearest neighbor algorithm to eliminate abnormal speed data to obtain the estimated speed of the landing platform; Step 5: According to the coordinates of the landing platform's logo in the body coordinate system, the motion trajectory of the landing platform is predicted using the decaying memory volumetric Kalman filter algorithm; Step 6: According to the estimated speed and motion trajectory of the landing platform, control the drone to fly toward the center of the landing platform; Step 7: Fuse the multi-sensor information to obtain the height information of the UAV from the landing platform. If the landing height is reached, execute step 8; otherwise, descend 0.2 meters and return to step 2; Step 8: The drone's rotor stops, and the electromagnet is turned on to attract the landing platform to complete the landing.
2. The method according to claim 1, characterized in that In the step 3, the logo of the landing platform is detected by a camera connected to the drone, and the logo is grayed, thresholded, and contour extracted to obtain the coordinates (u0, v0) of the center of the logo in the pixel coordinate system; wherein the x-axis direction of the camera is the same as the y-axis direction of the drone; The coordinates in the pixel coordinate system are converted to the coordinates in the body coordinate system, expressed as: Among them, d x and d y Represents the length of the unit pixel in the x and y directions; (u, v) represents the coordinates of the logo center in the image coordinate system; f represents the focal length of the camera; subscript C represents the coordinates in the camera coordinate system; subscript B represents the coordinates in the body coordinate system.
3. The method according to claim 2, characterized in that The step 4 comprises: Step 41: Obtain the coordinates of the drone body in the two frames of images at two moments in time, and perform differential calculation on the coordinates of the drone body to obtain the speed of the landing platform relative to the drone, which is expressed as: in, It represents the speed of the landing platform relative to the drone on the x-axis in the body coordinate system at time t; It represents the speed of the landing platform relative to the drone on the y-axis in the body coordinate system at time t; Represents the x-axis coordinate of the drone body in the body coordinate system at time t; Represents the x-axis coordinate of the drone body in the body coordinate system at time t-1; Represents the UAV body coordinate of the y-axis in the body coordinate system at time t; represents the UAV body coordinate of the y-axis in the body coordinate system at time t-1; Δt represents the time interval between time t and time t-1; Step 42: using the local weighted K-nearest neighbor algorithm to remove abnormal speed data, including: comparing the weighted average distance with the dynamic threshold, if the weighted average distance is greater than the dynamic threshold, it means that the estimated speed is an abnormal speed, the abnormal speed is removed, and then the estimated speed at the previous moment is used to replace the abnormal speed, which is specifically expressed as: in, Represents the weighted average distance between the current sample data and the data points at the previous K moments; Represents the weighted average of the data points at the previous K moments; ω t-i represents the weight at time ti; λ represents the decay rate; T dyn represents the current threshold, that is, the dynamic threshold; α represents the dynamic coefficient.
4. The method according to claim 3, characterized in that The step 5 comprises: Step 51: In the prediction step, the volume points are first calculated, which is expressed as: Among them, P k represents the covariance matrix; represents the state vector; represents the volume point; n represents the dimension of the state vector; represents the volume point set, [I] = [I n ,-I n ]; Then the state of the volume point is predicted, which is expressed as: in, represents the volume point through which the output propagates through the state equation; The target state vector and error covariance matrix are predicted according to the volume point state, which can be expressed as: in, represents the predicted value of the target state vector; P k+1|k represents the predicted value of the error covariance matrix; Q represents the process noise, b represents the attenuation factor; Step 52: In the update step, the volume point is calculated according to the target state vector of the prediction step and the predicted value of the error covariance matrix, which is expressed as: The volume point measurement is predicted based on the volume point, expressed as: in, represents the volume point where the output propagates through the measurement equation; The target measurement is predicted based on the volume point measurement, expressed as: in, represents the predicted value of the next measurement step; The measurement error covariance, measurement cross-covariance matrix and Kalman gain matrix are calculated and expressed as: in, represents the error covariance matrix, represents the cross-covariance matrix prediction value, K k+1 represents the Kalman gain matrix; R represents the measurement noise, Step 53: In the estimation step, the state vector and the error covariance matrix are first estimated based on the Kalman gain matrix and the predicted value of the target state vector, which is expressed as: in, represents the state vector at the current moment, P k+1 Represents the error covariance matrix at the current moment; Step 54: Repeat steps 51 to 54 according to the state vector and error covariance matrix at the current moment until the motion trajectory prediction of the landing platform is finally completed.
5. The method according to claim 4, characterized in that In step 6, the offset of the landing platform is used to control the drone to gradually approach the landing platform, which is expressed as: U =v p +μ*d; where subscript U represents the UAV, subscript p represents the landing platform, and μ represents the unit of s -1 The scale factor of d represents the horizontal distance between the landing platform and the UAV in the body coordinate system.
6. The method according to claim 5, characterized in that In step 7, the fixed weights of the multiple sensors satisfy: Among them, the subscript q = 1, 2, 3 corresponds to three types of sensors, represents the fixed weight of the sensor; The measurement weight of multiple sensors is expressed as: Among them, the subscript q = 1, 2, 3 corresponds to three types of sensors, Indicates the kth last data of the qth sensor; represents the variance of each sensor data, Represents the measurement weight of the sensor; The final weight of each sensor is expressed as: Among them, ω q represents the final weight of the sensor, α and β are the coefficients corresponding to the fixed weight and the measurement weight respectively.
Citation Information
Patent Citations
Dynamic target identification and tracking method for autonomous landing of unmanned aerial vehicle
CN110222612A
Robotic tracking navigation with data fusion
CN110546459A
Unmanned aerial vehicle landing pose filtering estimation method and system based on visual anchor points
CN112504261A
Unmanned aerial vehicle autonomous landing method based on vision assistance
CN114200948A
Unmanned aerial vehicle guided landing method based on pose prediction and trajectory optimization and unmanned aerial vehicle
CN116627154A
Cited By
Unmanned aerial vehicle landing positioning method based on visual identification
CN120953567A
Unmanned aerial vehicle landing positioning method based on visual recognition
CN120953567B
An unmanned aerial vehicle autonomous safe landing control method based on end-side visual calculation cooperation
CN122756232A
