A method, equipment and landing evaluation device for autonomous landing of unmanned aerial vehicle at sea

Abstract

The present invention relates to a method for autonomous landing of a drone at sea, comprising: performing image detection on a hull, and positioning the drone above a landing marker on the hull according to the image, and obtaining the landing marker image and the current altitude information of the drone; obtaining the posture of the drone, and correcting the landing marker image, calculating the real state of the hull, and obtaining the hull posture information; iteratively predicting the hull posture information, and then calculating the minimum cost trajectory planning based on the predicted hull state quantity; making the drone vertically descend while moving according to the minimum cost trajectory planning; judging whether the landing marker is directly below the drone; making the drone move according to the minimum cost trajectory planning; and making the drone stop its propellers and land. The method for autonomous landing of a drone at sea described in the present invention has the function of making the drone completely land on the landing platform of the hull on the sea surface according to inertia, and avoiding the occurrence of slipping and rollover of the drone during landing.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a method, equipment and a landing assessment device for autonomous landing of unmanned aerial vehicles at sea. Background Art

[0002] Drone landing has always been a key issue in drone control. When landing a drone through manual control, the user needs to have certain landing control experience, otherwise landing errors can easily cause damage to the drone.

[0003] Based on this, the existing drone automatic landing technology sets a marker at the landing position, allowing the drone to identify and locate according to the marker, thereby assisting the drone in landing and reducing the risk of damage to the drone due to the user's lack of expertise.

[0004] However, compared to the drone landing on the ground, the markers will not move, and the wind speed on land is lower, making the drone more stable, making it easier to achieve automatic landing. When the drone lands on the hull on the sea, it is easy for the drone to shake in the air due to the influence of wind and waves. At the same time, the hull will also float up and down and sway due to the waves and wind, which will cause the drone's camera to deform when identifying the markers. And because the drone and the hull are always in motion, the markers captured by the drone's camera cannot directly calculate the true angle of the hull relative to the sea level, which ultimately causes the drone to easily roll over and slide when landing on the hull on the sea. Summary of the invention

[0005] Based on this, the purpose of the present invention is to provide a method for autonomous landing of a drone at sea.

[0006] A method for autonomous landing of a drone at sea, comprising the following steps:

[0007] S1: Perform image detection on the hull, locate the landing mark on the hull based on the image, and obtain the landing mark image and the current altitude information of the UAV;

[0008] S2: Obtain the pitch angle and roll angle of the drone, correct the landing marker image, calculate the true state of the hull, and obtain the hull posture information;

[0009] S3: Iteratively predict the hull posture information through the extended Kalman filter, and then calculate several trajectory plans based on the predicted hull state quantities at several moments, and select the trajectory plan with the minimum cost based on the several trajectory plans;

[0010] S4: Determine whether the current altitude information of the drone meets the landing altitude: if not, execute step S5A; if yes, execute step S5B;

[0011] S5A: Send a descending signal to the drone, so that the drone descends vertically and moves according to the minimum cost trajectory planning; at the same time, execute step S1;

[0012] S5B: Acquire the landing platform image of the ship captured by the camera;

[0013] S6: judging whether the landing marker is directly below the UAV according to the landing platform image: if not, executing step S7A; if yes, executing step S7B;

[0014] S7A: Send a movement signal to the drone, so that the drone moves according to the minimum cost trajectory planning; at the same time, execute step S1;

[0015] S7B: Send a propeller stop signal to the drone, causing it to stop propellers and land.

[0016] Compared with the prior art, the present invention calculates the position information of the ship based on the landing markers on the ship by the drone, and predicts the trajectory planning at several moments based on the position information, ensuring that the drone will not be lost. At the same time, the trajectory planning is based on the perspective of the drone to calculate the position of the ship, so the trajectory of the ship is predicted and calculated relative to the current position of the drone, further ensuring the accuracy of the drone's movement according to the trajectory planning. In addition, in order to reduce the interference caused by the delay of the control operation when the drone lands, the present invention determines the best time to stop the propellers according to the trajectory planning corresponding to several moments, so that the drone can land according to inertia after stopping the propellers.

[0017] Furthermore, the step S1 includes the following steps:

[0018] S11: Acquire a hull image captured by a camera, detect and decode the image, and obtain the coordinates of the marker;

[0019] S12: sending a coordinate positioning signal to the drone according to the coordinates of the marker, so that the drone moves to the top of the landing marker;

[0020] S13: Acquire the landing marker image captured by the camera;

[0021] S14: Calculate the distance from the UAV to the hull platform according to the landing marker image, and obtain the current altitude information of the UAV.

[0022] Furthermore, the step S2 comprises the following steps:

[0023] S21: Acquire information from an inertial sensor of the drone, and perform attitude calculation based on the information to obtain a pitch angle and a roll angle of the drone;

[0024] S22: Correcting the landing marker image according to the pitch angle and roll angle of the UAV to obtain the deformation degree of the landing marker;

[0025] S23: Calculate the hull posture according to the deformation degree of the landing marker to obtain the hull posture information.

[0026] Among them, the present invention calculates the hull posture based on the deformation degree of the landing marker. Therefore, the hull posture is calculated according to the relative relationship of the UAV. Based on the relative relationship, it can be ensured that when the UAV is affected by environmental factors, the calculated hull posture will also change accordingly, so that the two maintain a dynamic balance and reduce the interference of environmental changes.

[0027] Furthermore, step S3 includes the following steps:

[0028] S31: Initialize the estimated value of the extended Kalman filter according to the hull posture information to obtain the estimated value of the hull state at the current moment;

[0029] S32: predicting the estimated value of the hull state at the current moment through an extended Kalman filter to obtain the hull state at the next moment;

[0030] S33: updating the estimated state of the hull according to the hull posture information and the hull state at the next moment, and obtaining an estimated value of the hull state at the next moment;

[0031] S34: Determine whether the current time exceeds a time threshold: if yes, execute step S35; if no, execute step S32;

[0032] S35: Calculate several ship trajectory plans according to the ship state quantities at several moments, and select the trajectory plan with the minimum cost.

[0033] Among them, by predicting the information of the next moment through the current moment, the hull state quantity at multiple moments can be predicted based on the hull estimation value obtained after the initial initialization, and then multiple trajectory plans can be calculated, and the best trajectory plan can be selected for the UAV to follow, synchronize and other movements.

[0034] In addition, as long as steps S32 to S33 are repeatedly executed multiple times, the state quantities of the hull at multiple timestamps can be predicted through the initial hull state quantities, thereby calculating the corresponding hull trajectory, and then obtaining the effect of short-term prediction of future trajectories. At the same time, in order to prevent the increase of errors caused by repeated execution, the present invention combines the initial hull posture information as a benchmark to reduce the prediction error.

[0035] Furthermore, the estimated value of the hull state at the current moment is obtained by:

[0036]

[0037] in, is the estimated value of the hull state, u is the input of the control state, w is Gaussian white noise, and the estimated value of the hull state at the current moment is The specific parameters are as follows:

[0038]

[0039] Among them, x and y are the two-dimensional coordinates of the hull, θ is the yaw angle of the hull, v is the linear velocity of the hull in the direction of the hull, r is the angular velocity of the hull, a is the linear acceleration of the hull in the direction of the hull, and The dynamic model is expressed as:

[0040]

[0041] Among them, △t is the time interval, and the hull acceleration and turning rate are constant values;

[0042] The estimated value of the hull state at the next moment is obtained by:

[0043]

[0044] Where K is the Kalman gain, The estimated value of the hull state at the current moment is the hull state obtained at the next moment. Update the value of the ship's state at the current moment. is the prior estimation output, z is the hull vector [x, y, θ] obtained by calculating the hull posture information; the hull state update value at the current moment As the estimated value of the hull state at the next moment.

[0045] The present invention adopts constant hull acceleration and turning rate to reduce the influence of changes in hull angular velocity, acceleration, etc.

[0046] A landing assessment device, comprising a landing marker detection unit, a relative posture calculation unit, a hull trajectory prediction unit, a UAV height judgment unit, a UAV synchronous descent control unit, a landing platform image acquisition unit, a UAV landing timing judgment unit, a UAV synchronous motion control unit and a UAV propeller stop control unit;

[0047] The landing marker detection unit is used to perform image detection on the hull, locate above the landing marker on the hull according to the image, and obtain the landing marker image and the current height information of the UAV;

[0048] The relative posture calculation unit is used to obtain the pitch angle and roll angle of the UAV, correct the landing marker image, calculate the real state of the hull, and obtain the hull posture information;

[0049] The hull trajectory prediction unit is used to iteratively predict the hull posture information through an extended Kalman filter, and then calculate a number of trajectory plans based on the predicted hull state quantities at a number of moments, and select a trajectory plan with a minimum cost based on the number of trajectory plans;

[0050] The drone altitude determination unit is used to determine whether the current altitude information of the drone meets the landing altitude: if not, the drone synchronous descent control unit is called; if yes, the landing platform image acquisition unit is called;

[0051] The UAV synchronous descent control unit is used to send a descent signal to the UAV, so that the UAV descends vertically while moving according to the minimum cost trajectory planning; at the same time, the landing marker detection unit is called.

[0052] The landing platform image acquisition unit is used to acquire the landing platform image of the hull taken by the camera;

[0053] The drone landing timing judgment unit is used to judge whether the landing marker is directly below the drone according to the landing platform image: if not, the drone synchronous motion control unit is called; if yes, the drone propeller stop control unit is called;

[0054] The UAV synchronous motion control unit is used to send a movement signal to the UAV, so that the UAV moves according to the minimum cost trajectory planning; at the same time, the landing marker detection unit is called;

[0055] The drone propeller stop control unit is used to send a propeller stop signal to the drone, so that the drone stops propellers and lands.

[0056] Further, the landing marker detection unit includes a landing marker coordinate acquisition module, a positioning module, a landing marker image acquisition module and a height acquisition module;

[0057] The landing marker coordinate acquisition module is used to acquire the hull image taken by the camera, detect and decode the image, and obtain the marker coordinates;

[0058] The positioning module is used to send a coordinate positioning signal to the drone according to the coordinates of the marker, so that the drone moves to the top of the landing marker;

[0059] The landing marker image acquisition module is used to acquire the landing marker image captured by the camera;

[0060] The height acquisition module is used to calculate the distance from the UAV to the hull platform according to the landing marker image, and obtain the current height information of the UAV;

[0061] The relative posture calculation unit includes a UAV posture calculation module, a landing marker image correction module and a hull posture acquisition module;

[0062] The UAV attitude calculation module is used to obtain information from the inertial sensor of the UAV, and perform attitude calculation based on the information to obtain the pitch angle and roll angle of the UAV;

[0063] The landing marker image correction module is used to correct the landing marker image according to the pitch angle and roll angle of the drone to obtain the deformation degree of the landing marker;

[0064] The hull posture acquisition module is used to calculate the hull posture according to the deformation degree of the landing marker to obtain the hull posture information;

[0065] The hull trajectory prediction unit includes an EKF initialization module, a hull state quantity prediction module, a hull state quantity update module, a hull trajectory planning judgment module and a minimum cost trajectory planning acquisition module;

[0066] The EKF initialization module is used to initialize the extended Kalman filter estimation value according to the hull posture information to obtain the hull state quantity estimation value at the current moment;

[0067] The hull state quantity prediction module is used to predict the hull state quantity estimation value at the current moment through the extended Kalman filter to obtain the hull state quantity at the next moment;

[0068] The hull state updating module is used to update the hull estimated state according to the hull posture information and the hull state at the next moment, and obtain the hull state estimation value at the next moment;

[0069] The hull trajectory planning judgment module is used to judge whether the current moment exceeds a moment threshold: if yes, the minimum cost trajectory planning acquisition module is called; if no, the hull state quantity prediction module is called;

[0070] The minimum cost trajectory planning acquisition module is used to calculate a number of hull trajectory plans according to the hull state quantities at a number of moments, and select the trajectory plan with the minimum cost.

[0071] Furthermore, the estimated value of the hull state at the current moment is obtained by:

[0072]

[0073] in, is the estimated value of the hull state, u is the input of the control state, w is Gaussian white noise, and the estimated value of the hull state at the current moment is The specific parameters are as follows:

[0074]

[0075] Among them, x and y are the two-dimensional coordinates of the hull, θ is the yaw angle of the hull, v is the linear velocity of the hull in the direction of the hull, r is the angular velocity of the hull, a is the linear acceleration of the hull in the direction of the hull, and The dynamic model is expressed as:

[0076]

[0077] Among them, △t is the time interval, and the hull acceleration and turning rate are constant values;

[0078] The estimated value of the hull state at the next moment is obtained by:

[0079]

[0080] Where K is the Kalman gain, The estimated value of the hull state at the current moment is the hull state obtained at the next moment. Update the value of the ship's state at the current moment. is the prior estimation output, z is the hull vector [x, y, θ] obtained by calculating the hull posture information; the hull state update value at the current moment As the estimated value of the hull state at the next moment.

[0081] An autonomous landing device for a drone at sea, comprising a drone and the above-mentioned landing assessment device, wherein the drone is provided with a camera and an inertial sensor;

[0082] The camera is used to capture images and input the images into the landing assessment device;

[0083] The inertial sensor is used to obtain the current attitude information of the UAV and input the information into the landing assessment device;

[0084] The landing assessment device is used to receive the image captured by the camera and the information of the inertial sensor, and combine the image and the information to predict and calculate the motion trajectory of a landing platform through an extended Kalman filter, and control the UAV to land on the landing platform according to the motion trajectory of the landing platform.

[0085] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 It is a structural schematic diagram of the landing assessment device according to the present invention;

[0087] Figure 2 This is a flow chart of the method for autonomous landing of a UAV at sea according to the present invention. DETAILED DESCRIPTION

[0088] In order to achieve precise autonomous landing of a UAV on a ship on the sea, the present invention obtains the current posture information of the UAV, and corrects the image of the hull landing marker taken by the UAV through the posture information, so as to calculate the current state of the hull. The calculated current state of the hull is predicted through an extended Kalman filter to obtain multiple hull trajectories. Finally, by judging the timing when the positions of the landing marker and the UAV are synchronized in the trajectories of the UAV and the hull, a propeller stop signal is sent to the UAV according to the timing, thereby achieving the landing of the UAV on the hull on the sea and avoiding the occurrence of slipping and capsizing of the UAV.

[0089] Based on the above design, the present invention proposes a method for autonomous landing of a UAV at sea, and forms an autonomous landing device for a UAV at sea based on this method.

[0090] Please also see Figure 1 and Figure 2 , Figure 1 is a schematic diagram of the structure of the landing assessment device according to the present invention, Figure 2 This is a flow chart of the method for autonomous landing of a UAV at sea according to the present invention.

[0091] The autonomous landing equipment at sea of ​​the UAV includes a UAV and a landing assessment device. A camera and an inertial sensor are provided directly below the UAV. The camera is used to capture images and input the images into the landing assessment device. The inertial sensor is used to obtain the current UAV posture information and input the information into the landing assessment device. The landing assessment device includes a landing marker detection unit 1, a relative posture calculation unit 2, a hull trajectory prediction unit 3, a UAV height judgment unit 4, a UAV synchronous descent control unit 5A, a landing platform image acquisition unit 5B, a UAV landing timing judgment unit 6, a UAV synchronous motion control unit 7A and a UAV propeller stop control unit 7B.

[0092] The landing marker detection unit 1 is used to execute step S1: perform image detection on the hull, locate above the landing marker of the hull according to the image, and obtain the landing marker image and the current height information of the UAV.

[0093] Specifically, the landing marker detection unit 1 includes a landing marker coordinate acquisition module 11 , a positioning module 12 , a landing marker image acquisition module 13 and a height acquisition module 14 .

[0094] The landing marker coordinate acquisition module 11 is used to execute step S11: acquire a hull image taken by a camera, detect and decode the image, and obtain the marker coordinates.

[0095] Specifically, there is a landing marker on the hull image; then, the landing marker is detected using the function (apriltag_detector_detect) of the AprilTag (visual reference system) tool, and the coordinates of the four corner points of the circumscribed rectangular frame of the marker (including the complete marker) and the identification ID of the landing marker (used to avoid landing on the wrong marker) can be obtained. Since the use of AprilTag has detailed documentation, the specific calling steps are not specifically described and limited in the present invention.

[0096] The decoding operation calculates the coordinates of the marker in the following way:

[0097] First, since the real size of the landing marker is known in meters, assuming that the real size is w, the coordinates of the four corner points in the world coordinate system can be calculated. Then, assuming that the origin of the world coordinate system is at the exact center of the landing marker, the X-axis direction is horizontal to the left, and the Y-axis direction is vertically upward, the coordinates of the four corner points in the clockwise direction are and Then, the opencv function (cv::solvePnP) is called to calculate and obtain the transfer matrix R and translation vector T of the corner point from the world coordinates to the camera coordinates.

[0098] According to the relationship between the camera coordinate system and the world coordinate system:

[0099] X C =RX W +T

[0100] Among them, X C is the camera coordinate system, X W is the world coordinate system, R is the rotation matrix, and T is the translation matrix. Since the position of the camera in the world coordinate system is X W =-R -1 T, so the coordinate of the center of the landing marker in the camera coordinate system is X C =T, and finally the coordinates of the marker are obtained and transmitted to the positioning module 12.

[0101] The coordinates of the marker obtained by the above decoding operation are the conversion and calculation between the camera coordinate system and the world coordinate system in the camera calibration, that is, the rigid body transformation. The calibration data required for the calculation process is obtained by the user's own calibration. Therefore, the present invention will not be specifically described and limited here.

[0102] In this embodiment, the landing marker is a nested QR code, which includes a first area QR code and a second area QR code. The first area QR code is proportionally larger than the second area QR code, and the first area QR code wraps the second area QR code, while ensuring that the centers of the first area QR code and the second area QR code coincide.

[0103] The nested QR codes can enable the drone to detect the QR code in the first area at a long distance, while ensuring that the QR code in the second area can be detected at a close distance, preventing the situation where the QR code is too large for the drone's camera to capture at a close distance, or too small for the drone's camera to capture at a long distance.

[0104] The positioning module 12 is used to execute step S12: sending a coordinate positioning signal to the drone according to the coordinates of the marker, so that the drone moves to the top of the landing marker.

[0105] Specifically, the coordinate positioning signal contains the coordinates of the marker, and the drone can smoothly move to the top of the landing marker according to the coordinates of the marker. Since the specific control method of the drone moving according to the coordinate information is a conventional technical means in the field, the present invention does not make specific limitations here.

[0106] Among them, since the coordinates of the marker are the coordinates of the drone corresponding to the center point of the landing marker, that is, relative coordinates, the error when the drone moves to the coordinate object will be smaller than the meter-level error of GPS positioning.

[0107] The landing marker image acquisition module 13 is used to execute step S13: acquiring a landing marker image captured by a camera.

[0108] Among them, since the camera is set directly below the drone, the landing marker image taken after positioning is almost on the same axis as the drone, and the acquired landing marker image is the most complete, making subsequent calculations more accurate.

[0109] The altitude acquisition module 14 is used to execute step S14: calculate the distance from the UAV to the hull platform according to the landing marker image, and obtain the current altitude information of the UAV.

[0110] Specifically, since the width of the landing marker is known, let the width be W, and the distance from the drone to the landing marker be D, the distance D can be calculated based on the monocular ranging principle by measuring the pixel width P of the landing marker image, that is:

[0111]

[0112] Where F is the focal length of the camera.

[0113] The height of the current UAV is calculated by using the known width of the landing marker combined with monocular ranging to ensure the relative relationship between the UAV and the landing platform of the hull.

[0114] The relative posture calculation unit 2 is used to execute step S2: obtain the pitch angle and roll angle of the UAV, correct the landing marker image, calculate the real state of the hull, and obtain the hull posture information.

[0115] Specifically, the relative posture calculation unit 2 includes a UAV posture solving module 21, a landing marker image correction module 22 and a hull posture acquisition module 23.

[0116] The UAV attitude calculation module 21 is used to execute step S21: obtain information from the inertial sensor of the UAV, and perform attitude calculation based on the information to obtain the pitch angle and roll angle of the UAV.

[0117] Specifically, the attitude solution is to obtain the pitch angle and roll angle of the drone by calculating the data of the gyroscope, accelerometer and magnetometer through quaternion and rotation matrix or Euler angle and rotation matrix or Rodriguez formula.

[0118] Among them, attitude solution is a conventional technical means in the art to calculate the current posture of the drone, which is not the innovation of the present invention and will not be specifically described or limited here.

[0119] The landing marker image correction module 22 is used to execute step S22: correcting the landing marker image according to the pitch angle and roll angle of the drone to obtain the deformation degree of the landing marker.

[0120] Specifically, the pitch angle and roll angle of the drone are combined to obtain the rotation matrix R,

[0121]

[0122] Then the landing marker image is rotated with the rotation matrix, that is, all pixel coordinates of the landing marker image are rotated according to the rotation matrix R to obtain a rotated image, and then the point pairs of the rotated image and the landing marker image are matched to obtain matching feature points pt1 and pt2, that is, the deformation degree of the landing marker. The image rotation is a conventional technical means in the art, and the present invention is not specifically limited here.

[0123] Among them, since the drone is located at sea and is easily affected by environmental factors, the drone is prone to tilt angles, which causes the landing marker image obtained by taking pictures to fail to truly reflect the actual tilt level of the hull landing platform. Therefore, it is necessary to first calculate the current attitude information of the drone, and then correct the captured landing marker image through the current attitude information to obtain the real information of the landing marker, and then calculate the real tilt level of the hull landing platform.

[0124] The hull posture acquisition module 23 is used to execute step S23: calculate the hull posture according to the deformation degree of the landing marker to obtain the hull posture information.

[0125] Specifically, according to the matched feature points pt1 and pt2 in the deformation degree of the landing marker, the feature points pt1 and pt2 are projected onto a plane P, and the plane P satisfies the equation:

[0126] n T P+d=0

[0127] Where n is the normal vector of the plane, n T is the transpose of the normal vector, P is the three-dimensional coordinate of a point on the plane, and d is the distance of the plane. Therefore, the relationship between the feature points can be obtained:

[0128]

[0129] Among them, K is the camera internal parameter matrix, and t is the translation between two feature points. Based on this, we only need to project four pairs of matching feature points onto a plane to solve Finally, it is decomposed to obtain the pitch angle and roll angle of the hull, that is, the position and posture information of the hull.

[0130] The hull trajectory prediction unit 3 is used to execute step S3: iteratively predict the hull posture information through the extended Kalman filter, then calculate several trajectory plans based on the predicted hull state quantities at several moments, and select the trajectory plan with the minimum cost based on the several trajectory plans.

[0131] Specifically, the hull trajectory prediction unit 3 includes an EKF initialization module 31 , a hull state quantity prediction module 32 , a hull state quantity update module 33 , a hull trajectory planning judgment module 34 and a minimum cost trajectory planning acquisition module 35 .

[0132] The EKF initialization module 31 is used to execute step S31: initialize the extended Kalman filter estimation value according to the hull posture information to obtain the hull state quantity estimation value at the current moment.

[0133] Specifically, the pitch angle and roll angle of the hull in the hull posture information are solved to obtain the hull vector, and the solution is to convert the pitch angle and roll angle of the hull in the hull posture information into a hull vector [x, y, θ], where x and y are the two-dimensional coordinates of the hull, and θ is the yaw angle of the hull (calculated by measuring counterclockwise along the x-axis), and the estimated value is calculated by the extended Kalman filter, that is:

[0134]

[0135] in, is the estimated value of the hull state, u is the input of the control state, w is Gaussian white noise, and the estimated value of the hull state at the current moment is The specific parameters are as follows:

[0136]

[0137] Among them, x and y are the two-dimensional coordinates of the hull, θ is the yaw angle of the hull, v is the linear velocity of the hull in the direction of the hull, r is the angular velocity of the hull, a is the linear acceleration of the hull in the direction of the hull, and The dynamic model is expressed as:

[0138]

[0139] Among them, △t is the time interval.

[0140] In order to reduce the influence of changes in the angular velocity and acceleration of the hull, the present invention adopts a constant hull acceleration and turning rate.

[0141] The hull state prediction module 32 is used to execute step S32: predicting the estimated value of the hull state at the current moment through an extended Kalman filter to obtain the hull state at the next moment.

[0142] Among them, by predicting the information of the next moment through the current moment, the hull state quantity at multiple moments can be predicted based on the hull estimation value obtained after the initial initialization, and then multiple trajectory plans can be calculated, and the best trajectory plan can be selected for the UAV to follow, synchronize and other movements.

[0143] The hull state updating module 33 is used to execute step S33: updating the hull estimated state according to the hull posture information and the hull state at the next moment, and obtaining the hull state estimation value at the next moment.

[0144] Specifically, the estimated state is updated in the following manner:

[0145]

[0146] Where K is the Kalman gain, The estimated value of the hull state at the current moment is the hull state obtained at the next moment. Update the value of the ship's state at the current moment. is the prior estimation output, z is the hull vector [x, y, θ] obtained by calculating the hull posture information, and the updated value of the hull state at the current moment is used as the estimated value of the hull state at the next moment.

[0147] Based on this, as long as steps S32 to S33 are repeated multiple times, the state of the hull at multiple timestamps can be predicted through the initial hull state, so as to calculate the corresponding hull trajectory, and then obtain the effect of short-term prediction of future trajectories. At the same time, in order to prevent the increase of errors due to repeated execution, the present invention combines the initial hull posture information as a benchmark to reduce the prediction error.

[0148] The hull trajectory planning judgment module 34 is used to execute step S34: determine whether the current moment exceeds a moment threshold: if yes, call the UAV height judgment unit 35; if no, call the hull state quantity prediction module 32.

[0149] Specifically, the time threshold is the number of extended Kalman filter prediction iterations or time interval set by the user according to needs and environment. The larger the threshold is, the greater the error that may exist in the predicted hull state quantity.

[0150] The minimum cost trajectory planning acquisition module 35 is used to execute step S35: calculate a number of hull trajectory plans according to the hull state quantities at a number of moments, and select the trajectory plan with the minimum cost.

[0151] Specifically, the trajectory planning is calculated according to the predicted hull state at each moment through a fast polynomial trajectory generation method, that is, it is achieved by minimizing the first-order derivative of acceleration, and finally the trajectory planning corresponding to all moments is obtained, and the trajectory planning with the minimum cost is selected.

[0152] Among them, the trajectory planning corresponding to all the moments is obtained by calculating the hull state quantity corresponding to each moment, so the trajectory planning corresponding to each moment has different movement schemes, among which the trajectory planning with the minimum cost is the trajectory planning for relatively smooth movement of the UAV. Since the fast polynomial trajectory generation algorithm is a conventional technical means in this field, the present invention will not be specifically described and limited here.

[0153] The drone altitude determination unit 4 is used to execute step S4: determine whether the current altitude information of the drone meets the landing altitude: if not, call the drone synchronous descent control unit 5A; if yes, call the landing platform image acquisition unit 5B.

[0154] Specifically, the landing height is 0.5m which is the best. If it is higher than 0.5m, the drone needs to continue to descend. The landing height is the height for the drone to stop propellers and land, which is obtained by combining the landing experience of the drone operator in the present invention.

[0155] The drone synchronous descent control unit 5A is used to execute step S5A: sending a descent signal to the drone, so that the drone descends vertically and moves according to the minimum cost trajectory planning. At the same time, the landing marker image acquisition module 13 is called.

[0156] The landing platform image acquisition unit 5B is used to execute step S5B: acquiring the landing platform image of the hull taken by the camera.

[0157] The drone landing timing judgment unit 6 is used to execute step S6: judging whether the landing marker is directly below the drone based on the landing platform image: if not, calling the drone synchronous motion control unit 7A; if yes, calling the drone propeller stop control unit 7B.

[0158] Specifically, if the landing marker is directly below the drone, it can be identified that at the current moment, the drone and the hull are in a synchronous motion state, and it can be considered that the current moment meets the landing opportunity.

[0159] This method can ensure that the drone is directly above the landing marker, preventing misplaced landing.

[0160] The drone synchronous motion control unit 7A is used to execute step S7A: sending a movement signal to the drone, so that the drone moves according to the minimum cost trajectory planning. At the same time, the landing marker image acquisition module 13 is called.

[0161] The drone propeller stopping control unit 7B is used to execute step S7B: sending a propeller stopping signal to the drone to make the drone stop propellers and land.

[0162] Among them, by using the previously calculated trajectory planning and synchronization time, the drone can be completely landed on the landing marker according to inertia after the propellers are stopped.

[0163] Compared with the prior art, the present invention calculates the position information of the ship based on the landing markers on the ship by the drone, and predicts the trajectory planning at several moments based on the position information, ensuring that the drone will not be lost. At the same time, the trajectory planning is based on the perspective of the drone to calculate the position of the ship, so the trajectory of the ship is predicted and calculated relative to the current position of the drone, further ensuring the accuracy of the drone's movement according to the trajectory planning. In addition, in order to reduce the interference caused by the delay of the control operation when the drone lands, the present invention determines the best time to stop the propellers according to the trajectory planning corresponding to several moments, so that the drone can land according to inertia after stopping the propellers.

[0164] Based on the same inventive concept, the present application also provides an electronic device, which may be a terminal device such as a server, a desktop computing device or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). The device includes one or more processors and a memory, wherein the processor is used to execute a program to implement a method for autonomous landing of a drone at sea according to an embodiment of the present invention; and the memory is used to store a computer program executable by the processor.

[0165] Based on the same inventive concept, the present application also provides a computer-readable storage medium, corresponding to an embodiment of the aforementioned method for autonomous landing of an unmanned aerial vehicle at sea, wherein the computer-readable storage medium stores a computer program thereon, and when the program is executed by a processor, the steps of the method for autonomous landing of an unmanned aerial vehicle at sea recorded in any of the aforementioned embodiments are implemented.

[0166] The present application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0167] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, and the present invention is also intended to include these modifications and modifications.

Claims

1. A method for autonomous landing of a drone at sea, characterized in that: The following steps are involved: S1: Perform image detection on the hull, locate the landing mark on the hull based on the image, and obtain the landing mark image and the current altitude information of the UAV; S2: Obtain the pitch angle and roll angle of the drone, correct the landing marker image, calculate the true state of the hull, and obtain the hull posture information; S3: Iteratively predict the hull posture information through the extended Kalman filter, and then calculate several trajectory plans based on the predicted hull state quantities at several moments, and select the trajectory plan with the minimum cost based on the several trajectory plans; Wherein, the step S3 includes the following sub-steps: S31: Initializing the estimated value of the extended Kalman filter according to the hull posture information to obtain the estimated value of the hull state at the current moment; wherein the estimated value of the hull state at the current moment is obtained by: in, is the estimated value of the hull state, u is the input of the control state, w is Gaussian white noise, and the estimated value of the hull state at the current moment is The specific parameters are as follows: Among them, x and y are the two-dimensional coordinates of the hull, θ is the yaw angle of the hull, v is the linear velocity of the hull in the direction of heading, r is the angular velocity of the hull, a is the linear acceleration of the hull in the direction of heading, and The dynamic model is expressed as: Among them, △t is the time interval, and the hull acceleration and turning rate are constant values; S32: predicting the estimated value of the hull state at the current moment through an extended Kalman filter to obtain the hull state at the next moment; S33: updating the estimated state of the hull according to the hull posture information and the hull state at the next moment, and obtaining an estimated value of the hull state at the next moment; wherein the estimated value of the hull state at the next moment is obtained by: Where K is the Kalman gain, The estimated value of the hull state at the current moment is the hull state obtained at the next moment. Update the value of the ship's state at the current moment. is the prior estimation output, z is the hull vector [x, y, θ] obtained by calculating the hull posture information; the hull state update value at the current moment As the estimated value of the hull state at the next moment; S34: Determine whether the current time exceeds a time threshold: if yes, execute step S35; if no, execute step S32; S35: Calculate several ship trajectory plans according to the ship state quantities at several moments, and select the trajectory plan with the minimum cost; S4: Determine whether the current altitude information of the drone meets the landing altitude: if not, execute step S5A; if yes, execute step S5B; S5A: Send a descending signal to the drone, so that the drone descends vertically and moves according to the minimum cost trajectory planning; at the same time, execute step S1; S5B: Acquire the landing platform image of the ship captured by the camera; S6: judging whether the landing marker is directly below the UAV according to the landing platform image: if not, executing step S7A; if yes, executing step S7B; S7A: Send a movement signal to the drone, so that the drone moves according to the minimum cost trajectory planning; at the same time, execute step S1; S7B: Send a propeller stop signal to the drone, causing it to stop propellers and land.

2. The method for autonomous landing of a drone at sea according to claim 1, characterized in that: The step S1 comprises the following steps: S11: Acquire a hull image captured by a camera, detect and decode the image, and obtain the coordinates of the marker; S12: sending a coordinate positioning signal to the drone according to the coordinates of the marker, so that the drone moves to the top of the landing marker; S13: Acquire the landing marker image captured by the camera; S14: Calculate the distance from the UAV to the hull platform according to the landing marker image, and obtain the current altitude information of the UAV.

3. The method for autonomous landing of a drone at sea according to claim 2, characterized in that: The step S2 comprises the following steps: S21: Acquire information from an inertial sensor of the drone, and perform attitude calculation based on the information to obtain a pitch angle and a roll angle of the drone; S22: Correcting the landing marker image according to the pitch angle and roll angle of the UAV to obtain the deformation degree of the landing marker; S23: Calculate the hull posture according to the deformation degree of the landing marker to obtain the hull posture information.

4. A landing assessment device, characterized in that: It includes a landing marker detection unit, a relative posture calculation unit, a hull trajectory prediction unit, a UAV height judgment unit, a UAV synchronous descent control unit, a landing platform image acquisition unit, a UAV landing timing judgment unit, a UAV synchronous motion control unit and a UAV propeller stop control unit; The landing marker detection unit is used to perform image detection on the hull, locate above the landing marker on the hull according to the image, and obtain the landing marker image and the current height information of the UAV; The relative posture calculation unit is used to obtain the pitch angle and roll angle of the UAV, correct the landing marker image, calculate the real state of the hull, and obtain the hull posture information; The hull trajectory prediction unit is used to iteratively predict the hull posture information through an extended Kalman filter, and then calculate a number of trajectory plans based on the predicted hull state quantities at a number of moments, and select a trajectory plan with a minimum cost based on the number of trajectory plans; The hull trajectory prediction unit further includes an EKF initialization module, a hull state quantity prediction module, a hull state quantity update module, a hull trajectory planning judgment module and a minimum cost trajectory planning acquisition module; The EKF initialization module is used to initialize the extended Kalman filter estimation value according to the hull posture information to obtain the hull state quantity estimation value at the current moment; wherein the hull state quantity estimation value at the current moment is obtained by: in, is the estimated value of the hull state, u is the input of the control state, w is Gaussian white noise, and the estimated value of the hull state at the current moment is The specific parameters are as follows: Among them, x and y are the two-dimensional coordinates of the hull, θ is the yaw angle of the hull, v is the linear velocity of the hull in the direction of the hull, r is the angular velocity of the hull, a is the linear acceleration of the hull in the direction of the hull, and The dynamic model is expressed as: Among them, △t is the time interval, and the hull acceleration and turning rate are constant values; The hull state quantity prediction module is used to predict the hull state quantity estimation value at the current moment through the extended Kalman filter to obtain the hull state quantity at the next moment; The hull state quantity updating module is used to update the hull estimated state according to the hull posture information and the hull state quantity at the next moment, and obtain the hull state quantity estimated value at the next moment; wherein the hull state quantity estimated value at the next moment is obtained by the following method: Where K is the Kalman gain, The estimated value of the hull state at the current moment is the hull state obtained at the next moment. Update the value of the ship's state at the current moment. is the prior estimation output, z is the hull vector [x, y, θ] obtained by calculating the hull posture information; the hull state update value at the current moment As the estimated value of the hull state at the next moment; The hull trajectory planning judgment module is used to judge whether the current moment exceeds a moment threshold: if yes, the minimum cost trajectory planning acquisition module is called; if no, the hull state quantity prediction module is called; The minimum cost trajectory planning acquisition module is used to calculate a number of hull trajectory plans according to the hull state quantities at a number of moments, and select the trajectory plan with the minimum cost; The drone altitude determination unit is used to determine whether the current altitude information of the drone meets the landing altitude: if not, the drone synchronous descent control unit is called; if yes, the landing platform image acquisition unit is called; The UAV synchronous descent control unit is used to send a descent signal to the UAV, so that the UAV descends vertically and moves according to the minimum cost trajectory planning; at the same time, the landing marker detection unit is called; The landing platform image acquisition unit is used to acquire the landing platform image of the hull taken by the camera; The drone landing timing judgment unit is used to judge whether the landing marker is directly below the drone according to the landing platform image: if not, the drone synchronous motion control unit is called; if yes, the drone propeller stop control unit is called; The UAV synchronous motion control unit is used to send a movement signal to the UAV, so that the UAV moves according to the minimum cost trajectory planning; at the same time, the landing marker detection unit is called; The drone propeller stop control unit is used to send a propeller stop signal to the drone, so that the drone stops propellers and lands.

5. The landing assessment device according to claim 4, characterized in that: The landing marker detection unit includes a landing marker coordinate acquisition module, a positioning module, a landing marker image acquisition module and a height acquisition module; The landing marker coordinate acquisition module is used to acquire the hull image taken by the camera, detect and decode the image, and obtain the marker coordinates; The positioning module is used to send a coordinate positioning signal to the drone according to the coordinates of the marker, so that the drone moves to the top of the landing marker; The landing marker image acquisition module is used to acquire the landing marker image captured by the camera; The height acquisition module is used to calculate the distance from the UAV to the hull platform according to the landing marker image, and obtain the current height information of the UAV; The relative posture calculation unit includes a UAV posture calculation module, a landing marker image correction module and a hull posture acquisition module; The UAV attitude calculation module is used to obtain information from the inertial sensor of the UAV, and perform attitude calculation based on the information to obtain the pitch angle and roll angle of the UAV; The landing marker image correction module is used to correct the landing marker image according to the pitch angle and roll angle of the drone to obtain the deformation degree of the landing marker; The hull posture acquisition module is used to calculate the hull posture according to the deformation degree of the landing marker to obtain the hull posture information.

6. An autonomous landing device for a drone at sea, comprising a drone, wherein the drone is provided with a camera and an inertial sensor, and wherein: Also includes a landing assessment device as described in any one of claims 4 to 5; The camera is used to capture images and input the images into the landing assessment device; The inertial sensor is used to obtain the current attitude information of the UAV and input the information into the landing assessment device; The landing assessment device is used to receive the image captured by the camera and the information of the inertial sensor, and combine the image and the information to predict and calculate the motion trajectory of a landing platform through an extended Kalman filter, and control the UAV to land on the landing platform according to the motion trajectory of the landing platform.

7. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for autonomous landing of a drone at sea as described in any one of claims 1 to 3 when executing the computer program.

Images