Unmanned aerial vehicle autonomous take-off and landing decision and control method based on ship attitude perception and dynamic compensation

Abstract

The invention belongs to the technical field of unmanned aerial vehicle autonomous take-off and landing. The invention provides an unmanned aerial vehicle autonomous take-off and landing decision and control method based on ship attitude perception and dynamic compensation. According to the embodiment of the invention, a ship-aircraft cooperative sensing architecture is introduced, and bidirectional verification and fusion are carried out through the sensor data of the ship end and the unmanned aerial vehicle end, so that the reliability and fault-tolerant capability of the system are greatly improved. When the unmanned aerial vehicle end is in visual failure, the system can perform guidance by depending on visual data of the ship end; when the ship end sensor goes wrong, the unmanned aerial vehicle end sensing system can still work independently, and the safety of the take-off and landing process under extreme conditions is ensured. Data fusion is carried out through an extended Kalman filter, and the rolling angle, the pitching angle and the heaving amount of the ship deck are solved in real time; meanwhile, in combination with a motion prediction algorithm based on a long-short-term memory network model, a future deck motion trajectory can be predicted, so that the system can plan a take-off and landing opportunity in advance, and forced take-off and landing of the unmanned aerial vehicle during strenuous motion of the deck are avoided.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous take-off and landing technology for unmanned aerial vehicles (UAVs), and more particularly to a method for autonomous take-off and landing decision-making and control of UAVs based on ship attitude perception and dynamic compensation. Background Technology

[0002] With the rapid development of unmanned aerial vehicle (UAV) technology, its application in the maritime field is becoming increasingly widespread, covering multiple aspects such as ship reconnaissance and surveillance, maritime search and rescue (SAR), cargo transportation, waterway inspection, and marine scientific research. In these missions, enabling UAVs to take off and land autonomously and safely on underway ship platforms is a key technology for building an efficient ship-aircraft collaborative system, and a core prerequisite for expanding the operational range and effectiveness of UAVs at sea.

[0003] However, as a dynamic platform, a ship's motion is influenced by factors such as waves and wind, exhibiting complex movements with multiple degrees of freedom, primarily including roll, pitch, and vertical heave. This continuous and irregular motion makes the ship's deck an unstable take-off and landing platform, posing a severe challenge to the autonomous take-off and landing control of unmanned aerial vehicles (UAVs). To ensure safety, UAVs must complete precise trajectory tracking and landing relative to the dynamic deck at the exact moment.

[0004] Currently, a typical approach in this field is to integrate high-precision real-time dynamic differential GPS (RTK) with a ship motion reference system (MRS). By acquiring the absolute position information of the UAV and the ship, the relative motion is indirectly derived and compensated. The limitation of this method lies in its heavy reliance on continuous and stable satellite signals and data communication. In real-world marine environments, obstructions from the ship's superstructure, multipath effects on the sea surface, and severe weather conditions can all lead to signal quality degradation or interruption, thereby affecting the reliability and availability of the system.

[0005] Another technical approach is to directly inject compensation commands into the underlying layer of the UAV flight control system. This method achieves high-frequency motion compensation by modifying the expected values ​​of the internal control loops of the flight control system. Although the response speed may be faster, this method requires a deep understanding of the underlying code and architecture of the specific flight control system, has a high technical threshold, and is difficult to port across platforms. It is particularly difficult to adapt to commercial UAVs with closed architectures, resulting in poor system versatility and maintainability.

[0006] Currently, the following technical shortcomings exist in the field of ship attitude perception and dynamic compensation for unmanned aerial vehicles (UAVs): (1) Environmental adaptability defects: excessive dependence on specific navigation signals Existing technologies rely entirely on the continuous stability of RTK-GPS signals and data links. However, in marine environments, signal obstruction by shipboard superstructures, multipath effects caused by sea surface reflection, and communication interruptions in severe weather can all lead to the loss or distortion of critical sensing data, resulting in complete failure of the guidance function.

[0007] (2) System generality defects: rigid architecture and poor platform compatibility The technical approach of "modifying the underlying flight control code" requires the system to be deeply customized for a specific flight control model. This presents extremely high technical barriers, resulting in a severe lack of universality and portability across different UAV platforms (and even different models of the same brand), leading to high R&D and maintenance costs and hindering large-scale application and promotion.

[0008] (3) Robustness defects: single perception dimension and lack of redundancy mechanism The existing solution relies solely on externally inputted GPS and ship MRS data, lacking airborne autonomous, multi-source perception and verification methods. If external data malfunctions or becomes erroneous, the UAV cannot autonomously identify and switch to a backup solution. In the final critical landing phase, there is a lack of reliable redundancy protection mechanisms, resulting in insufficient overall system fault tolerance and safety.

[0009] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0010] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0011] The purpose of this disclosure is to provide a method for autonomous take-off and landing decision-making and control of unmanned aerial vehicles based on ship attitude perception and dynamic compensation, thereby overcoming, to at least to some extent, one or more problems caused by the limitations and defects of related technologies.

[0012] According to embodiments of this disclosure, a method for autonomous take-off and landing decision-making and control of unmanned aerial vehicles (UAVs) based on ship attitude perception and dynamic compensation is provided, including: Construct a perception-decision-control closed-loop system; wherein, the perception-decision-control closed-loop system includes a ship-aircraft relative attitude perception module, a dynamic prediction and take-off and landing timing decision module, and a real-time status monitoring and emergency support module. The ship-aircraft relative attitude perception module includes a ship-end perception unit, an UAV-end perception unit, and a central fusion unit. The ship's motion data and the UAV's motion data are collected by the ship's sensing unit and the UAV's sensing unit, respectively. The central fusion unit then fuses the ship's motion data and the UAV's motion data using an extended Kalman filter to obtain the relative pose information between the UAV and the ship's deck. Furthermore, the ship-side sensing unit includes: Deck visual beacon system and ship inertial reference system; among which, The deck visual beacon system is used to deploy visual markers in the take-off and landing area of ​​a ship's deck and monitor the position and attitude of drones relative to the deck through a wide-angle surveillance camera; Ship inertial reference systems are used to acquire the absolute attitude and heave displacement data of the ship.

[0013] Furthermore, the drone-side sensing unit includes: Airborne vision unit and airborne inertial measurement unit; among which, The airborne vision unit is used to identify visual markers on the deck through target detection algorithms and to calculate the three-dimensional pose of the UAV relative to the deck through the PnP algorithm. The airborne inertial measurement unit is used to collect the acceleration and angular velocity data of the UAV and output the attitude estimate of the UAV itself through a complementary filtering algorithm.

[0014] Furthermore, the PnP algorithm is as follows:

[0015] in, Scale factor; Let i be the position of the i-th feature point on the image plane. Vertical coordinates; This is the camera intrinsic parameter matrix; Let R be the camera extrinsic parameter matrix, where R describes rotation and t describes translation; Let be the 3D position coordinates of the i-th feature point in the world coordinate system; The vertical axis coordinates are... For depth axis coordinates; A homogeneous world coordinate point; The complementary filtering algorithm is as follows:

[0016] Where angle is the estimated angle; gyro is the angular velocity reading of the gyroscope; and accel is the angle calculated by the accelerometer. dt represents the filter coefficients; dt represents the sampling time interval.

[0017] Furthermore, the extended Kalman filter in the central fusion unit uses the ship's motion data measured by the ship's inertial reference system as the absolute motion reference. It performs cross-validation and weighted fusion on the relative pose of the UAV measured by the ship's vision and the relative pose of the UAV itself measured by the UAV's vision to obtain the relative pose information between the UAV and the ship's deck.

[0018] Furthermore, the extended Kalman filter in the central fusion unit is constructed as follows: The state vector is defined as ;in, This indicates the three-dimensional relative position between the drone and the deck. Represents three-dimensional relative velocity. Quaternions representing relative attitude; Using the ship's absolute motion data provided by the ship's inertial reference system as a benchmark, and combining it with data from the UAV inertial measurement unit, a discrete-time nonlinear state transition equation is established. ;in, This is a state transition function based on rigid body kinematics. To control the input, This refers to system process noise. Using the motion data of the ship's inertial reference system as the absolute motion reference, observations are made at the ship's end. With drone observation Cross-validation is performed, and the state estimates are recursively updated based on their respective observation noise covariance to obtain the relative pose information between the UAV and the ship deck.

[0019] The dynamic prediction and take-off and landing timing decision module uses a long short-term memory network model to predict the motion trajectory of the ship's deck based on historical relative pose information. When the predicted motion trajectory of the ship's deck meets the preset safe take-off and landing window conditions, take-off and landing control commands are generated. Furthermore, the dynamic prediction and takeoff / landing timing decision module uses a long short-term memory network model to predict the ship's deck trajectory based on historical relative pose information. When the predicted ship's deck trajectory meets the preset safe takeoff / landing window conditions, the step of generating takeoff / landing control commands includes: The historical relative pose information output by the central fusion unit is processed into a time series, and a historical data sequence with a fixed time window length is constructed as the model input. Historical data sequences are input into a pre-trained long short-term memory network model, and the model outputs a predicted sequence of the ship's deck roll angle, pitch angle, and heave displacement over a future period of time. The predicted sequence is compared with the preset safe take-off and landing window conditions. When the predicted motion trajectory in a continuous time period in the future meets the safe take-off and landing window conditions, it is determined that a safe take-off and landing window exists, and take-off and landing control commands are generated.

[0020] The UAV takes off and lands based on takeoff and landing control commands. The real-time status monitoring and emergency support module monitors the system's status parameters in real time. When the status parameters exceed the preset threshold, the emergency response mechanism is triggered to realize the UAV's autonomous takeoff and landing decision and control.

[0021] Furthermore, the real-time monitored system status parameters include relative pose, prediction error, communication delay, and sensor health status; among them, the preset thresholds include three levels of safety thresholds, corresponding to warning, hovering, and emergency abort levels, respectively.

[0022] Furthermore, the emergency response mechanism adopts a finite state machine design for exception handling logic. When an emergency response is triggered, the drone is controlled to enter a hovering state and a hover-wait-retry mechanism or an emergency return-to-home procedure is initiated.

[0023] Furthermore, the method also includes: Self-testing and degradation strategies are used to identify sensor failures or communication interruptions and switch to backup sensors or execute emergency return-to-home procedures.

[0024] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: In the embodiments disclosed herein, the above-described method for autonomous take-off and landing decision-making and control of unmanned aerial vehicles (UAVs) based on ship attitude perception and dynamic compensation introduces a ship-aircraft collaborative perception architecture. This architecture utilizes bidirectional verification and fusion of sensor data from both the ship and UAV ends, significantly improving the system's reliability and fault tolerance. It addresses the problem of potential failure of relying solely on airborne perception in complex environments (such as brief occlusion of the UAV camera, strong light, or contamination of deck markers). When the UAV's vision fails, the system can rely on visual data from the ship for guidance; conversely, when the ship's sensors malfunction, the UAV's perception system can still operate independently, forming a valuable redundancy backup and ensuring the safety of the take-off and landing process under extreme conditions. Furthermore, data fusion via an extended Kalman filter allows for real-time calculation of the ship's deck roll, pitch, and heave angles with accuracies of 0.1° and 0.01 meters, respectively. Simultaneously, combined with a motion prediction algorithm based on a long short-term memory network model, the system can predict the deck's trajectory within the next 0.5-2 seconds, enabling it to plan take-off and landing opportunities in advance and preventing forced take-off and landing of the UAV during periods of severe deck movement. On the other hand, through the coordinated work of the three core modules, the entire process from environmental awareness to takeoff and landing execution is automated. The dynamic prediction and takeoff / landing timing decision module can calculate the optimal takeoff and landing timing in real time, avoiding delays and operational errors caused by manual judgment. The real-time status monitoring and emergency support module provides all-weather, multi-dimensional safety assurance. Through multi-source status monitoring and a rapid emergency response mechanism, it ensures that the mission can be promptly suspended and safety plans executed in case of emergencies, significantly improving the system's safety and reliability. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0026] Figure 1 A flowchart illustrating the steps of an autonomous take-off and landing decision-making and control method for unmanned aerial vehicles based on ship attitude perception and dynamic compensation in an exemplary embodiment of this disclosure; Figure 2 This diagram illustrates a framework of a perception-decision-control closed-loop system in an exemplary embodiment of this disclosure. Figure 3 A schematic diagram illustrating the emergency response mechanism in an exemplary embodiment of this disclosure is shown; Figure 4 This diagram illustrates a flowchart of the ship-aircraft relative attitude sensing module in an exemplary embodiment of this disclosure; Figure 5 A diagram illustrating the prediction and decision-making implementation in an exemplary embodiment of this disclosure is shown. Detailed Implementation

[0027] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0028] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0029] This example implementation provides a method for autonomous takeoff and landing decision-making and control of unmanned aerial vehicles (UAVs) based on ship attitude perception and dynamic compensation. (Reference) Figure 1 As shown, the UAV autonomous takeoff and landing decision-making and control method based on ship attitude perception and dynamic compensation may include: Step S101: Construct a perception-decision-control closed-loop system; wherein, the perception-decision-control closed-loop system includes a ship-aircraft relative attitude perception module, a dynamic prediction and take-off and landing timing decision module, and a real-time status monitoring and emergency support module, and the ship-aircraft relative attitude perception module includes a ship-end perception unit, an UAV-end perception unit, and a central fusion unit. Step S102: Collect motion data of the ship and the UAV using the ship-end sensing unit and the UAV-end sensing unit respectively, and use the central fusion unit to fuse the motion data of the ship and the UAV through an extended Kalman filter to obtain the relative pose information between the UAV and the ship deck. Step S103: The dynamic prediction and take-off and landing timing decision module uses a long short-term memory network model to predict the motion trajectory of the ship deck based on historical relative pose information. When the predicted motion trajectory of the ship deck meets the preset safe take-off and landing window conditions, a take-off and landing control command is generated. Step S104: Control the take-off and landing of the UAV based on the take-off and landing control command. The real-time status monitoring and emergency support module monitors the status parameters of the system in real time. When the status parameters exceed the preset threshold, the emergency response mechanism is triggered to realize the autonomous take-off and landing decision and control of the UAV.

[0030] The aforementioned method for autonomous take-off and landing decision-making and control of unmanned aerial vehicles (UAVs) based on ship attitude perception and dynamic compensation significantly improves system reliability and fault tolerance by introducing a ship-UAV collaborative perception architecture. This involves bidirectional verification and fusion of sensor data from both the ship and UAV ends. It also addresses the potential failure of relying solely on airborne perception in complex environments (such as brief camera obstruction, strong light, or deck marking contamination). When UAV vision fails, the system can rely on ship-side visual data for guidance; conversely, even when ship-side sensors malfunction, the UAV perception system can still operate independently, providing valuable redundancy and ensuring safety during take-off and landing in extreme conditions. Furthermore, data fusion using an extended Kalman filter allows for real-time calculation of the ship's deck roll, pitch, and heave angles with accuracies of 0.1° and 0.01 meters, respectively. Simultaneously, a motion prediction algorithm based on a long short-term memory network model can predict deck movement trajectories within 0.5-2 seconds, enabling the system to plan take-off and landing times in advance and preventing forced take-off and landing of UAVs during periods of severe deck movement. On the other hand, through the coordinated work of the three core modules, the entire process from environmental awareness to takeoff and landing execution is automated. The dynamic prediction and takeoff / landing timing decision module can calculate the optimal takeoff and landing timing in real time, avoiding delays and operational errors caused by manual judgment. The real-time status monitoring and emergency support module provides all-weather, multi-dimensional safety assurance. Through multi-source status monitoring and a rapid emergency response mechanism, it ensures that the mission can be promptly suspended and safety plans executed in case of emergencies, significantly improving the system's safety and reliability.

[0031] Below, we will refer to Figures 1 to 5 The steps of the above-described method for autonomous take-off and landing decision-making and control of unmanned aerial vehicles based on ship attitude perception and dynamic compensation in this example embodiment will be explained in more detail.

[0032] In step S101, a perception-decision-control closed-loop system is constructed; wherein, the perception-decision-control closed-loop system includes a ship-aircraft relative attitude perception module, a dynamic prediction and take-off and landing timing decision module, and a real-time status monitoring and emergency support module, and the ship-aircraft relative attitude perception module includes a ship-side perception unit, an UAV-side perception unit, and a central fusion unit. For example... Figure 2 The diagram shown is a framework diagram of a perception-decision-control closed-loop system.

[0033] In step S102, the ship's motion data and the UAV's motion data are collected by the ship-end sensing unit and the UAV-end sensing unit, respectively. The central fusion unit then fuses the ship's motion data and the UAV's motion data using an extended Kalman filter to obtain the relative pose information between the UAV and the ship's deck.

[0034] Specifically, the ship-aircraft relative attitude perception module This module employs ship-aircraft collaborative multi-source sensor fusion technology to achieve precise perception of the relative motion attitude of the ship and the UAV. The system consists of three parts working collaboratively: a ship-side sensing unit, a UAV-side sensing unit, and a central fusion unit.

[0035] a. Ship-side sensing unit: Deck visual beacon system: Deploy multiple high-precision visual markers (such as AprilTag codes) around the ship's deck take-off and landing area, and install wide-angle monitoring cameras to continuously monitor the precise position of the drone relative to the deck.

[0036] Ship inertial reference system: directly connected to the ship's inherent high-precision motion reference system (MRS) to obtain the ship's absolute attitude (roll, pitch, bow) and heave displacement data in the world coordinate system.

[0037] b. UAV-side sensing unit: Airborne vision unit: It uses a deep learning-based target detection algorithm (YOLOv5) to identify visual markers on the deck in real time, and uses the perspective n-point (PnP) algorithm to calculate the three-dimensional pose of the UAV relative to the deck coordinate system.

[0038] Airborne Inertial Measurement Unit (IMU): Collects three-axis acceleration and three-axis angular velocity data of the UAV body, performs data preprocessing through algorithms such as complementary filtering, and outputs the estimated attitude angle of the UAV itself.

[0039] c. Center fusion algorithm: Establish a distributed extended Kalman filter (EKF). A central fusion unit (which can be deployed on a ship or via a wireless link) receives pose data from both the ship and the UAV.

[0040] The fusion algorithm uses ship MRS data as the absolute motion reference and performs cross-validation and weighted fusion of the relative pose of the UAV measured by the ship's vision and the relative pose of the UAV itself measured by the UAV's vision.

[0041] The final output is a high-frequency, smooth, accurate and reliable relative pose information between the UAV and the deck, including relative roll angle, relative pitch angle and relative heave, with accuracy superior to single-end sensing.

[0042] In one embodiment, the extended Kalman filter in the central fusion unit is constructed as follows: The state vector is defined as , where p rel V represents the three-dimensional relative position between the drone and the deck. rel q represents the three-dimensional relative velocity. rel Quaternions representing relative attitudes.

[0043] The system model uses the ship's absolute motion data provided by the ship's inertial reference system as a benchmark, and combines it with data from the UAV inertial measurement unit to establish discrete-time nonlinear state transition equations. ,in, This is a state transition function based on rigid body kinematics. To control the input, This refers to system process noise.

[0044] The observation model includes two independent observation sources: ship-side observations. The relative pose of the UAV measured from the deck visual beacon system; UAV-side observation. The relative pose of the UAV itself is calculated by the PnP algorithm from the airborne vision unit; Fusion process: Using the motion data of the ship's inertial reference system as the absolute motion reference, the ship-side observations are performed. With drone observation Cross-validation is performed, and the state estimates are recursively updated based on their respective observation noise covariance to obtain the relative pose information between the UAV and the ship deck.

[0045] In step S103, the dynamic prediction and take-off and landing timing decision module uses a long short-term memory network model to predict the motion trajectory of the ship deck based on historical relative pose information. When the predicted motion trajectory of the ship deck meets the preset safe take-off and landing window conditions, a take-off and landing control command is generated.

[0046] Specifically, the dynamic prediction and takeoff / landing timing decision module This module runs on an embedded processor platform (NVIDIA Jetson Nano) and performs motion prediction and takeoff / landing timing decisions. Motion prediction algorithm: A time-series prediction model based on Long Short-Term Memory (LSTM) network is adopted. Using historical attitude data as input, it predicts the deck motion trajectory within the next 0.5-2 seconds. The model is trained offline with a large amount of ship motion data and can accurately capture the periodic characteristics of wave motion.

[0047] Takeoff and landing timing decision logic: Set safe takeoff and landing window conditions, including roll angle threshold ±5°, pitch angle threshold ±3°, and heave displacement threshold ±0.2 meters. When it is predicted that the window conditions will be met continuously for a certain period of time in the future, a corresponding control command sequence is generated.

[0048] Command generation and transmission: Through the SDK interface provided by the drone manufacturer (such as DJI MSDK), high-level control commands, including position setpoints and speed commands, are sent to the flight control system to achieve motion compensation and ensure that the drone can complete take-off and landing operations when the deck is relatively stable.

[0049] In step S104, the UAV takes off and lands based on takeoff and landing control commands. The real-time status monitoring and emergency support module monitors the system's status parameters in real time. When the status parameters exceed the preset threshold, the emergency response mechanism is triggered to realize the UAV's autonomous takeoff and landing decision and control.

[0050] Specifically, the real-time status monitoring and emergency support module like Figure 3 As shown, this module runs in parallel during the decision-making process, providing round-the-clock security: Multi-source status monitoring: Real-time monitoring of key parameters such as relative pose, prediction error, communication delay, and sensor health status; setting three levels of safety thresholds, corresponding to warning, hovering, and emergency abort levels, respectively.

[0051] Emergency response mechanism: The exception handling logic is designed using a finite state machine. When the parameter exceeds the threshold, an emergency stop command is immediately sent to control the drone to enter a hovering state and start the "hover-wait-retry" mechanism.

[0052] System self-test and degradation strategy: The built-in self-test function can identify abnormal situations such as sensor failure and communication interruption, and take corresponding degradation strategies, including switching to backup sensors and executing emergency return procedures.

[0053] In one specific embodiment, the implementation of the ship-aircraft relative attitude perception module is as follows: First, several high-precision visual markers (using AprilTag encoding) were pre-installed on fixed structures around the take-off and landing area on the ship's deck. The three-dimensional positions of these markers in the ship's coordinate system were known and precisely measured. Simultaneously, wide-angle monitoring cameras were installed at appropriate locations on the deck to ensure their field of view completely covers the UAV take-off and landing area. These cameras continuously capture images of the UAVs during take-off and landing. Furthermore, the system directly connects to the ship's own high-precision ship motion reference system (MRS) to directly acquire real-time data on the ship's roll, pitch, bow angles, and heave displacement. On the ship's end, a computing unit runs a visual recognition algorithm to process the images captured by the cameras, identifying the UAV in real time and calculating its attitude and position relative to the deck coordinate system. Subsequently, this visually calculated position and attitude data is time-synchronized and aligned with the ship's own motion data obtained from the motion reference system, and finally sent to the central data fusion computing unit located on the ship via the ship's internal local area network.

[0054] The drone itself is also equipped with a perception system. A downward-facing camera mounted on the bottom of the drone captures images of the deck at a fixed frequency (e.g., 25Hz). These images are fed into a target detection process based on a YOLOv5 deep learning model, which is trained to quickly and accurately identify pre-placed visual markers on the deck. After identifying the markers, the system uses the perspective n-point (PnP) algorithm to calculate the drone's six-degree-of-freedom pose (i.e., three-dimensional position and three-dimensional attitude) relative to the deck based on the known three-dimensional coordinates of the markers and their two-dimensional projection points in the image.

[0055] PnP core formula:

[0056] in, The scale factor is a non-zero scalar used to represent the homogeneous coordinate scale in the projection process from 3D world coordinates to 2D image coordinates. The image pixel coordinates represent the position of the i-th feature point on the image plane, and the horizontal coordinates are in pixels. These are vertical coordinates, which are typically obtained through camera calibration. The camera intrinsic parameter matrix is ​​a 3×3 matrix containing the camera's intrinsic parameters. It is used to project points in the camera coordinate system onto the image plane. The camera extrinsic parameter matrix is ​​a 3×4 matrix composed of a rotation matrix R (3×3) and a translation vector t (3×1). It represents the rigid body transformation from the world coordinate system to the camera coordinate system, where R describes the rotation and t describes the translation. is the world coordinate, representing the 3D position coordinates of the i-th feature point in the world coordinate system, and is the horizontal axis coordinate. The vertical axis coordinates are... For depth axis coordinates; For homogeneous world coordinates, it is a 4×1 vector that simplifies projection calculations by adding a homogeneous coordinate value of 1.

[0057] Detailed process of PnP algorithm: a. Input data: a set of 3D world points (known coordinates of deck visual markers in the ship coordinate system), corresponding 2D image points (pixel coordinates of marker points detected from the image), and camera intrinsic parameter matrix K obtained through camera calibration, including focal length and principal point parameters.

[0058] b. Construct the projection equation: For each pair of points, establish the projection relationship (Formula 1).

[0059] c. Solving the PnP problem: Minimize the reprojection error using an iterative optimization method (Levenberg-Marquardt algorithm):

[0060] Where Π is the projection function, which converts 3D points into 2D pixel coordinates. After initializing the extrinsic parameter estimation, R and t are iteratively adjusted to minimize the error.

[0061] d. Obtain the rotation matrix R and translation vector t, and then calculate the relative position and attitude of the UAV with respect to the deck.

[0062] Meanwhile, the inertial measurement unit (IMU) on the drone is also continuously operating, acquiring raw three-axis angular velocity and three-axis acceleration data of the drone body at a higher frequency (e.g., 100Hz). This raw IMU data is immediately preprocessed, a key step of which is the application of a complementary filtering algorithm. This algorithm cleverly combines the advantages of the gyroscope's accurate short-term measurements (capable of rapidly responding to drastic attitude changes) and the accelerometer's stable long-term measurements (providing an absolute attitude reference). It uses a filtering coefficient α (typically around 0.98) to weight and fuse the two sets of data, ultimately outputting a rapid-response and drift-free estimate of the drone's attitude. Figure 4 It is a drone perception system.

[0063] Complementary filtering algorithm formula:

[0064] Where angle is the estimated angle; gyro is the angular velocity reading of the gyroscope; accel is the angle calculated by the accelerometer; α is the filter coefficient, ranging from 0 to 1; and dt is the sampling time interval.

[0065] The specific process of the complementary filtering algorithm: Input data: gyroscope angular velocity reading (gyro), accelerometer-calculated angle (accel), filter coefficient (α), sampling time interval (dt).

[0066] Algorithm steps: Gyroscope integral: Calculates the angle change based on the angular velocity, where angleprev is the angle estimated at the previous moment.

[0067] Accelerometer compensation: Accelerometers provide an absolute attitude reference, but are susceptible to vibration interference.

[0068] Complementary fusion: Combining data from the gyroscope and accelerometer is equivalent to high-pass filtering the gyroscope data (reducing drift) and low-pass filtering the accelerometer data (reducing noise).

[0069]

[0070] c. Output: Estimated angle, used for UAV attitude control.

[0071] Once the ship and UAV each generate and transmit multi-source sensor data, the core of the entire system—the central fusion phase—begins. We employ the Extended Kalman Filter (EKF) as the core algorithm for data fusion. EKF is an advanced state estimation algorithm suitable for nonlinear systems. Its operation is a recursive "prediction-update" loop: in each computation cycle, EKF first performs a "time update" (also called a prediction step), which uses the optimal estimate of the UAV-deck relative pose from the previous moment, combined with a system kinematic model built based on UAV IMU data and ship MRS data, to predict the current relative pose and simultaneously estimate the uncertainty (covariance) of this prediction. Next comes the "measurement update" (also known as the update step), which is key to embodying our "ship-aircraft collaboration" concept: When the central fusion unit receives the UAV's relative pose data calculated by the ship's vision system, EKF immediately treats this data as an actual observation, compares it with the previously predicted value, and performs optimal fusion to correct the prediction and obtain a more accurate new estimate. Similarly, when it receives its own relative pose data calculated by the UAV's vision system, EKF treats it as an independent observation and performs the same fusion correction process again. Through this continuous bidirectional data injection and recursive optimization, EKF can effectively integrate data from four information sources: ship vision, UAV vision, ship inertia, and UAV inertia, ultimately outputting a high-frequency, smooth, accurate, and highly reliable real-time pose information of the UAV relative to the deck. This unique ship-aircraft collaborative architecture constitutes a robust redundancy design: for example, when the UAV's downward-looking camera cannot identify markers due to deck mist, strong glare, or temporary obstruction, the system can continue operating primarily by relying on observations from the ship's visual system; conversely, if the ship's camera's field of view is obstructed, the UAV's visual and inertial data can still support system operation. This greatly enhances the robustness and reliability of the entire perception system under complex and non-ideal sea conditions, laying a solid data foundation for subsequent intelligent decision-making and safety control.

[0072] In a specific embodiment, the implementation of the dynamic prediction and takeoff / landing timing decision module is as follows: In the trajectory prediction and decision-making stage, the core of this module lies in creatively applying the existing Long Short-Term Memory (LSTM) network model to the specific technical problem of "ship deck motion prediction," and constructing a complete data processing and decision-making process adapted to it. The LSTM model itself is a well-known recurrent neural network structure in existing technology, particularly suitable for processing time-series data. The key to this invention lies in its specific application method, model construction, training method, and integration with the takeoff and landing decision system.

[0073] a. Model building and training (offline phase): Input and Output Definitions: The model input is a fixed-length time series segment. Historical relative pose information of the ship's deck and the UAV, sampled at a frequency of 100Hz and output by the central fusion unit within the past 3 seconds, is selected. This information mainly includes roll, pitch, and heave displacements. The model output is a predicted sequence of deck motion over a future period of 0.5 to 2 seconds, also including predicted roll, pitch, and heave values.

[0074] Network Structure: The LSTM model comprises an input layer, at least one LSTM hidden layer, and a fully connected output layer. Through offline training on a large amount of ship motion data collected under different sea conditions, the model is able to accurately learn and capture the periodic and nonlinear dynamic characteristics of ship motion under wave excitation.

[0075] Training objective: The optimization objective of the training process is to minimize the mean square error between the predicted trajectory and the true trajectory to ensure the accuracy of the prediction.

[0076] b. Online forecasting and decision-making process (online operation phase): Data preprocessing steps: The relative pose information output in real time by the central fusion unit is sent to a first-in-first-out (FIFO) data buffer. The system extracts a historical data sequence of equal length from the buffer at fixed time intervals (e.g., 10ms) and performs preprocessing such as normalization to form a tensor that meets the model input requirements.

[0077] Trajectory prediction steps: The preprocessed historical data sequence is input into an LSTM model deployed on an embedded processor (NVIDIA Jetson Nano). The model performs one forward propagation calculation and outputs a predicted sequence of deck motion trajectories for the next 0.5-2 seconds.

[0078] The takeoff and landing window determination process (decision logic) involves the decision unit receiving the prediction sequence and comparing it point-by-point with the preset safe takeoff and landing window conditions (roll angle threshold ±5°, pitch angle threshold ±3°, heave displacement threshold ±0.2 meters). The system does not require a single point to meet the conditions; rather, it requires all prediction points within a consecutive 0.5-second time interval in the prediction sequence to simultaneously meet all safety conditions before determining the existence of a usable "safe takeoff and landing window."

[0079] Command Generation: Once a safety window is identified, the decision module immediately generates and sends takeoff and landing control commands. This prediction-based decision-making mechanism enables the UAV to respond in advance and perform takeoff and landing maneuvers when the deck movement reaches the most stable phase, rather than passively following the current state, thereby greatly improving the safety and success rate of takeoff and landing.

[0080] Finally, the command generation and compensation unit generates the final position and velocity setpoints based on the decision results and real-time pose, and sends them to the flight control system via the UAV SDK. Figure 5 As shown.

[0081] In one specific embodiment, the implementation of the real-time status monitoring and emergency support module is as follows: To ensure absolute safety under complex sea conditions, the system is designed with a continuous safety monitoring mechanism. The safety monitoring module performs real-time monitoring of multiple parameters from the sensing module and internal system data.

[0082] These parameters will be dynamically compared with preset three-level safety thresholds and trigger corresponding graded response strategies: when the parameters are close to but do not exceed the safety boundary, the system triggers a warning-level response, sends an alarm message to the control station, but the mission continues; when the parameters exceed the safe operating range but do not pose an immediate danger, the system immediately triggers a hovering command, and the UAV maintains its current position in the air, waiting for the status to return to normal or to receive operator instructions; when a serious fault, a possible collision, or a critical parameter seriously exceeds the limit is detected, the system immediately triggers an emergency abort command, terminates the current take-off and landing mission, and directs the UAV to execute a predetermined emergency climb procedure to fly to a safe airspace.

[0083] The entire monitoring and state transition process is managed by a finite state machine with an exception handling mechanism. The finite state machine flowchart is as follows: Figure 3 .

[0084] The aforementioned method for autonomous take-off and landing decision-making and control of unmanned aerial vehicles (UAVs) based on ship attitude perception and dynamic compensation significantly improves system reliability and fault tolerance by introducing a ship-UAV collaborative perception architecture. This involves bidirectional verification and fusion of sensor data from both the ship and UAV ends. It also addresses the potential failure of relying solely on airborne perception in complex environments (such as brief camera obstruction, strong light, or deck marking contamination). When UAV vision fails, the system can rely on ship-side visual data for guidance; conversely, even when ship-side sensors malfunction, the UAV perception system can still operate independently, providing valuable redundancy and ensuring safety during take-off and landing in extreme conditions. Furthermore, data fusion using an extended Kalman filter allows for real-time calculation of the ship's deck roll, pitch, and heave angles with accuracies of 0.1° and 0.01 meters, respectively. Simultaneously, a motion prediction algorithm based on a long short-term memory network model can predict deck movement trajectories within 0.5-2 seconds, enabling the system to plan take-off and landing times in advance and preventing forced take-off and landing of UAVs during periods of severe deck movement. On the other hand, through the coordinated work of the three core modules, the entire process from environmental awareness to takeoff and landing execution is automated. The dynamic prediction and takeoff / landing timing decision module can calculate the optimal takeoff and landing timing in real time, avoiding delays and operational errors caused by manual judgment. The real-time status monitoring and emergency support module provides all-weather, multi-dimensional safety assurance. Through multi-source status monitoring and a rapid emergency response mechanism, it ensures that the mission can be promptly suspended and safety plans executed in case of emergencies, significantly improving the system's safety and reliability.

[0085] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0086] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A method for unmanned aerial vehicle autonomous take-off and landing decision and control based on ship attitude perception and dynamic compensation, characterized in that, include: Construct a perception-decision-control closed-loop system; wherein, the perception-decision-control closed-loop system includes a ship-aircraft relative attitude perception module, a dynamic prediction and take-off and landing timing decision module, and a real-time status monitoring and emergency support module. The ship-aircraft relative attitude perception module includes a ship-end perception unit, an UAV-end perception unit, and a central fusion unit. The ship's motion data and the UAV's motion data are collected by the ship's end sensing unit and the UAV's end sensing unit, respectively. The central fusion unit then fuses the ship's motion data and the UAV's motion data using an extended Kalman filter to obtain the relative pose information between the UAV and the ship's deck. The dynamic prediction and take-off and landing timing decision module uses a long short-term memory network model to predict the motion trajectory of the ship's deck based on historical relative pose information. When the predicted motion trajectory of the ship's deck meets the preset safe take-off and landing window conditions, take-off and landing control commands are generated. The UAV takes off and lands based on takeoff and landing control commands. The real-time status monitoring and emergency support module monitors the system's status parameters in real time. When the status parameters exceed the preset threshold, the emergency response mechanism is triggered to realize the UAV's autonomous takeoff and landing decision and control.

2. The method of claim 1, wherein, Ship-based sensing units include: Deck visual beacon system and ship inertial reference system; among which, The deck visual beacon system is used to deploy visual markers in the take-off and landing area of ​​a ship's deck and monitor the position and attitude of drones relative to the deck through a wide-angle surveillance camera; Ship inertial reference systems are used to acquire the absolute attitude and heave displacement data of the ship.

3. The method of claim 2, wherein, The drone's end-sensing unit includes: Airborne vision unit and airborne inertial measurement unit; among which, The airborne vision unit is used to identify visual markers on the deck through target detection algorithms and to calculate the three-dimensional pose of the UAV relative to the deck through the PnP algorithm. The airborne inertial measurement unit is used to collect the acceleration and angular velocity data of the UAV and output the attitude estimate of the UAV itself through a complementary filtering algorithm.

4. The method of claim 3, wherein, The PnP algorithm is as follows: wherein, is a scale factor; is the position of the i-th feature point on the image plane, is the vertical coordinate; is the camera intrinsic matrix; is the camera extrinsic matrix, R describes the rotation and t describes the translation; is the 3D position coordinate of the i-th feature point in the world coordinate system; is the vertical axis coordinate, is the depth axis coordinate; is the homogeneous world coordinate point; The complementary filtering algorithm is as follows: where angle is the estimated angle; gyro is the gyroscope angular velocity reading; accel is the accelerometer calculated angle; is the filter coefficient; dt is the sampling time interval.

5. The method of claim 4, wherein, The extended Kalman filter in the central fusion unit uses the ship's motion data measured by the ship's inertial reference system as the absolute motion reference. It performs cross-validation and weighted fusion on the relative pose of the UAV measured by the ship's vision and the relative pose of the UAV itself measured by the UAV's vision to obtain the relative pose information between the UAV and the ship's deck.

6. The method for autonomous take-off and landing decision-making and control of unmanned aerial vehicles based on ship attitude perception and dynamic compensation according to claim 5, characterized in that, The extended Kalman filter in the central fusion unit is constructed as follows: The state vector is defined as ;in, This indicates the three-dimensional relative position between the drone and the deck. Represents three-dimensional relative velocity. Quaternions representing relative attitude; Using the ship's absolute motion data provided by the ship's inertial reference system as a benchmark, and combining it with data from the UAV inertial measurement unit, a discrete-time nonlinear state transition equation is established. ;in, The state transition function is based on rigid body kinematics. To control the input, This refers to system process noise. Using the motion data of the ship's inertial reference system as the absolute motion reference, observations are made at the ship's end. With drone observation Cross-validation is performed, and the state estimates are recursively updated based on their respective observation noise covariance to obtain the relative pose information between the UAV and the ship deck.

7. The method for autonomous take-off and landing decision-making and control of unmanned aerial vehicles based on ship attitude perception and dynamic compensation according to claim 6, characterized in that, The real-time monitored system status parameters include relative pose, prediction error, communication delay, and sensor health status; among them, the preset thresholds include three levels of safety thresholds, corresponding to warning, hovering, and emergency abort levels, respectively.

8. The method for autonomous take-off and landing decision-making and control of unmanned aerial vehicles based on ship attitude perception and dynamic compensation according to claim 7, characterized in that, The emergency response mechanism uses a finite state machine to design the exception handling logic. When an emergency response is triggered, the drone is controlled to enter a hovering state and the hover-wait-retry mechanism or emergency return-to-home procedure is initiated.

9. The method for autonomous take-off and landing decision-making and control of unmanned aerial vehicles based on ship attitude perception and dynamic compensation according to claim 8, characterized in that, The method also includes: Self-testing and degradation strategies are used to identify sensor failures or communication interruptions and switch to backup sensors or execute emergency return-to-home procedures.

10. The method for autonomous take-off and landing decision-making and control of unmanned aerial vehicles based on ship attitude perception and dynamic compensation according to claim 9, characterized in that, The dynamic prediction and takeoff / landing timing decision module uses a long short-term memory network model to predict the ship's deck trajectory based on historical relative pose information. When the predicted ship's deck trajectory meets the preset safe takeoff / landing window conditions, the step of generating takeoff / landing control commands includes: The historical relative pose information output by the central fusion unit is processed into a time series, and a historical data sequence with a fixed time window length is constructed as the model input. Historical data sequences are input into a pre-trained long short-term memory network model, and the model outputs a predicted sequence of the ship's deck roll angle, pitch angle, and heave displacement over a future period of time. The predicted sequence is compared with the preset safe take-off and landing window conditions. When the predicted motion trajectory in a continuous time period in the future meets the safe take-off and landing window conditions, it is determined that a safe take-off and landing window exists, and take-off and landing control commands are generated.

Images