Method for multi-source sensor combination to guide autonomous landing of a drone on a ship deck

Abstract

This application belongs to the field of design technology for unmanned aerial vehicle (UAV) landing on ship decks. Specifically, it relates to a method for guiding UAVs to autonomously land on ship decks using a combination of multi-source sensors. This method integrates multi-source perception, has adaptive processing capabilities, and can adapt to complex shipboard environments. It can fully leverage the advantages of various sensors at different distance stages, gradually improve navigation accuracy, and ultimately achieve a highly reliable and high-precision autonomous landing mission for shipborne UAVs on ship decks.

Description

Technical Field

[0001] This application belongs to the field of design technology for unmanned aerial vehicle (UAV) landing on ship decks, specifically relating to a method for guiding a UAV to autonomously land on a ship deck using a combination of multi-source sensors. Background Technology

[0002] With the continuous development of drone technology, it plays a vital role in reconnaissance and surveillance, communication relay, and material delivery. Especially in maritime operations, shipborne drones, with their advantages of small size, flexible deployment, and high level of intelligence, have become key equipment for enhancing the mission capabilities of fleets. Compared to traditional manned aircraft, which are limited by pilot physiological load, face high safety risks, and are unable to perform missions continuously, drones possess significant advantages such as being unrestricted by physiological limitations, having low mission risks, rapid deployment, and all-weather operation, making them more suitable for mission requirements in high-intensity and complex environments.

[0003] However, achieving safe and accurate autonomous landing of drones on ship decks still faces many technical challenges. Due to the relative motion between the ship and the drone, and the significant impact of external disturbances such as waves and wind speed, coupled with the limited space on the deck, the reliability and accuracy of traditional navigation methods (such as GNSS or INS) in this environment are insufficient to meet the requirements of drone terminal guidance.

[0004] To improve the accuracy of UAV landings on ship decks, existing research has attempted to introduce visual navigation methods. However, at long distances beyond visual range, visual systems struggle to reliably identify target ships. Furthermore, many solutions still rely on traditional image processing algorithms for feature extraction, which are less adaptable to changes in lighting, occlusion, and scale variations. This results in performance degradation and insufficient robustness in complex environments such as strong light, rain, fog, and nighttime, making it difficult to meet the requirements for stable identification and high-precision positioning under real-world conditions.

[0005] This application is made in view of the aforementioned technical deficiencies. Summary of the Invention

[0006] The purpose of this application is to provide a method for guiding a UAV to autonomously land on a ship deck using a combination of multi-source sensors. This method integrates multi-source perception, has adaptive processing capabilities, and can adapt to complex shipboard environments. It can fully leverage the advantages of various sensors at different distance stages, gradually improve navigation accuracy, and ultimately achieve a highly reliable and high-precision autonomous landing mission for shipborne UAVs on ship decks.

[0007] The technical solution of this application is:

[0008] A method for guiding a drone to autonomously land on a ship deck using a combination of multi-source sensors includes:

[0009] Initial guidance phase based on GPS and IMU:

[0010] The ship is outside the line-of-sight range of the drone, and the drone is guided to fly towards the ship by a combined navigation module consisting of GPS and inertial measurement unit (IMU).

[0011] Ship capture phase based on monocular vision:

[0012] Once the ship enters the line-of-sight range of the drone, the lightweight deep learning target detection model NanoDet is used to identify and capture the ship at a long distance. Based on the change in the ship's relative position in the image, the drone's flight attitude is adjusted.

[0013] Binocular vision-based target locking phase during descent:

[0014] The drone gets closer to the ship and can observe the landing target on the ship's deck. Using images captured by the binocular camera, the NanoDet deep learning target detection model identifies the landing target on the ship's deck. The controller dynamically adjusts the drone's attitude angle, forward speed, and vertical altitude to keep it aligned with the landing target in space.

[0015] High-precision positioning landing phase:

[0016] For the final landing phase, based on the locked landing target, high-precision key point identification and pose calculation are completed.

[0017] Optionally, in the above-mentioned method of using a combination of multi-source sensors to guide a drone to land autonomously on a ship deck, during the initial guidance phase based on GPS and IMU, the ship periodically broadcasts its own position information and transmits it to the drone in real time via a data link. After obtaining its own GPS positioning information, the drone calculates the navigation target point based on its relative position information with the ship, guiding the drone to approach the ship from its current starting position.

[0018] Optionally, in the above-mentioned method for guiding a UAV to autonomously land on a ship deck using a combination of multi-source sensors, the initial guidance phase based on GPS and IMU involves fusing the multi-source sensor data using an extended Kalman filter, specifically including:

[0019] When GPS is unavailable, the drone's current position and attitude are predicted by high-frequency inertial integration based on acceleration and angular velocity data provided by the IMU.

[0020] When GPS updates become available, they are used as observation inputs to correct previous predictions, thereby effectively suppressing the accumulation of errors caused by IMU integral drift and achieving dynamic correction of the UAV's status.

[0021] Optionally, in the above-mentioned method for guiding a UAV to autonomously land on a ship deck using a combination of multi-source sensors, during the target locking stage based on binocular vision, the target detection box is optimized and expanded to extract the region of interest containing the complete outline of the target.

[0022] The disparity between matching points in the left and right eye images of the region of interest is calculated to estimate the three-dimensional spatial position of the landing target.

[0023] Optionally, in the above-mentioned method of guiding a UAV to autonomously land on a ship deck using a combination of multi-source sensors, during the target locking phase based on binocular vision, the center point of the region of interest in the current detection frame is extracted, and the pixel offset error between the center point of the optical axis of the camera image is calculated.

[0024] By fusing pixel offset error with depth information recovered by stereo matching, the spatial error between the UAV and the landing target is obtained. As a feedback input, the controller dynamically adjusts the UAV's attitude angle, forward speed and vertical altitude, so that the UAV continuously aligns with the landing target in space and approaches the preset landing guidance point.

[0025] Optionally, in the above-mentioned method of guiding a UAV to autonomously land on a ship deck using a combination of multi-source sensors, during the high-precision positioning and landing phase, key points within the landing target are detected, and the coordinates of each point are finely extracted using a sub-pixel-level optimization algorithm.

[0026] A 3D-2D point correspondence relationship is constructed, the PnP algorithm is called to calculate the pose between the UAV and the landing target landing point, and the RANSAC algorithm is introduced to screen key points for consistency and eliminate potential outliers.

[0027] Optionally, in the above-mentioned method of guiding a UAV to autonomously land on a ship deck using a combination of multi-source sensors, during the high-precision positioning and landing phase, the real-time vertical distance between the UAV and the deck is obtained by a laser ranging module, and then fused with the relative Z-axis coordinates calculated by the PnP algorithm using a dynamic weighted fusion strategy based on reprojection error to output the final result.

[0028] Optionally, in the above method of guiding a UAV to autonomously land on a ship deck using a combination of multi-source sensors, a dataset of long-distance images of the ship covering multiple perspectives, sea conditions, and lighting conditions is used to iteratively train the deep learning object detection model NanoDet until the model converges. The deep learning object detection model NanoDet outputs the coordinates of the detection box of the ship in the image coordinate system.

[0029] Optionally, in the above-mentioned method of using a combination of multi-source sensors to guide a drone to land autonomously on a ship deck, the trained deep learning target detection model NanoDet is converted into TensorRT format and deployed on the JetsonOrin embedded computing platform carried by the drone to achieve real-time ship detection.

[0030] Optionally, in the above method of using a combination of multi-source sensors to guide a drone to land autonomously on a ship deck, the center of the landing target is an AprilTag pattern with a unique ID.

[0031] The AprilTag pattern features a symmetrical "cross" structure around its perimeter, along with a black ring graphic.

[0032] The black ring-shaped graphic has circular markers at its four corners, and an infrared light source is installed inside.

[0033] The landing target is surrounded by several black square reflective films. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall architecture of the method for guiding a drone to autonomously land on a ship deck using a combination of multi-source sensors, as provided in this application embodiment.

[0035] Figure 2 This is a flowchart of GPS and IMU-based guidance outside the line-of-sight range provided in the embodiments of this application;

[0036] Figure 3 This is a schematic diagram of the state estimation and error compensation mechanism for a UAV based on extended Kalman filtering provided in an embodiment of this application;

[0037] Figure 4 This is a vision-based guidance flowchart within the line-of-sight range provided in an embodiment of this application;

[0038] Figure 5 This is the landing target pattern provided in the embodiments of this application.

[0039] To better illustrate this embodiment, some content in the accompanying drawings may be omitted, enlarged, or reduced. They are for illustrative purposes only and should not be construed as limiting the scope of this application. Detailed Implementation

[0040] To make the technical solution and advantages of this application clearer, the technical solution of this application will be described in a clearer and more complete manner below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only some embodiments of this application, and are only used to explain this application, not to limit this application. It should be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, and other related parts can be referred to the general design.

[0041] Furthermore, unless otherwise defined, the technical or scientific terms used in this application description shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The word "comprising" as used in this application description indicates that the concept preceding the word encompasses the concepts listed following the word and their equivalents, without excluding other related concepts.

[0042] Furthermore, the terms indicating location used in the description of this application are only used to indicate relative directions or positional relationships. When the absolute position of the described object changes, its relative positional relationship may also change accordingly. It should also be noted that, unless otherwise explicitly specified and limited, terms such as "installation" and "connection" used in the description of this application should be interpreted broadly. For example, a connection can be a fixed connection or a detachable connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand its specific meaning in this application according to the specific circumstances.

[0043] A method for guiding a drone to autonomously land on a ship deck using a combination of multi-source sensors includes:

[0044] Initial guidance phase based on GPS and IMU:

[0045] During the initial guidance phase, the ship is outside the UAV's line-of-sight range. The UAV is guided to fly towards the ship by a combined navigation module consisting of GPS and an inertial measurement unit (IMU).

[0046] The ship regularly broadcasts its own location information and transmits it to the drone in real time via a data link. After obtaining its own GPS positioning information, the drone calculates the navigation target point based on its relative position with the ship and guides the drone to approach the ship from its current starting position.

[0047] The fusion processing of multi-source sensor data using an extended Kalman filter specifically includes:

[0048] State prediction phase: When GPS is unavailable, high-frequency inertial integration is performed using acceleration and angular velocity data provided by the IMU to predict the current position and attitude of the UAV.

[0049] Status update phase: When GPS updates are available, they are used as observation inputs to correct previous predictions, thereby effectively suppressing the accumulation of errors caused by IMU integral drift and achieving dynamic correction of the UAV status.

[0050] Through the above mechanism, the system can maintain the continuity of the UAV's flight path and ensure the smoothness of navigation output when GPS is unstable or intermittently unavailable. The integrated navigation module calculates the desired yaw angle in real time based on the current coordinate difference between the UAV and the ship, and combines this with the flight controller to schedule the heading angle and forward speed, thereby achieving closed-loop control of the flight trajectory.

[0051] Ship capture phase based on monocular vision:

[0052] Once the ship gradually enters the visual range of the drone, the drone assesses the perception conditions based on its flight status and then enters the visual acquisition phase.

[0053] The lightweight deep learning object detection model NanoDet is used for long-range identification and capture of ships. The specific process is as follows:

[0054] Model training phase: Using a dataset of long-distance ship images covering various viewpoints, sea conditions, and lighting conditions, the NanoDet deep learning object detection model is iteratively trained until convergence. The NanoDet deep learning object detection model outputs the coordinates of the detection box of the ship in the image coordinate system, which is used for subsequent guidance and control.

[0055] Model deployment phase: To improve the inference performance of edge devices, the NanoDet deep learning object detection model was trained, converted to TensorRT format, and deployed on the JetsonOrin embedded computing platform carried by the drone to achieve real-time ship detection.

[0056] Based on the ship recognition results, a guidance and control strategy based on image plane error is further implemented. The UAV's flight attitude is adjusted based on the relative position change of the ship in the image. Specifically, the center point of the camera image's optical axis is used as the ship alignment reference position. Simultaneously, the coordinates of the ship's center point detected in the current frame are extracted, and the pixel offset error between the two is calculated in real time. Based on this, the current deviation direction of the ship is determined. When the error exceeds a set threshold, the flight control system adjusts the UAV's yaw angle to reduce lateral offset, while simultaneously coordinating and adjusting the forward speed to maintain trajectory stability and visual alignment.

[0057] Binocular vision-based target locking phase during descent:

[0058] As the drone approaches the ship further, the ship occupies a larger area in the image, making the landing target on the ship's deck visible. At this point, the spatial resolution corresponding to the baseline parallax between the left and right cameras on the drone meets the requirements, and the system switches to a binocular vision-based spatial servo control mode. In this stage, using the image pairs acquired by the binocular cameras, the NanoDet deep learning object detection model identifies the landing target on the ship's deck, and optimizes and expands the detection boxes, extracting the complete outline of the landing target as the region of interest. Then, the parallax value between matching points in the left and right images within the region of interest is calculated to estimate the three-dimensional spatial position of the landing target.

[0059] Simultaneously, the center point of the region of interest in the current detection frame is extracted, and its pixel offset error relative to the center point of the camera image's optical axis is calculated. By fusing the pixel offset error with the depth information recovered from stereo matching, the spatial error between the UAV and the landing target is obtained. Based on this, using the currently estimated spatial error as feedback input, the controller dynamically adjusts the UAV's attitude angle, forward velocity, and vertical altitude, ensuring that it continuously aligns with the landing target in space and approaches the preset landing guidance point.

[0060] High-precision positioning and landing phase integrating laser and key points of the landing target:

[0061] During the final landing phase, based on the locked landing target, high-precision key point identification and pose calculation are completed.

[0062] First, the key points within the landing target are detected, and the coordinates of each point are extracted precisely using a sub-pixel-level optimization algorithm.

[0063] Next, a 3D-2D point correspondence relationship is constructed, the PnP algorithm is called to calculate the pose between the UAV and the landing target landing point, and the RANSAC algorithm is introduced to screen key points for consistency and eliminate potential outliers.

[0064] In addition, the real-time vertical distance between the UAV and the deck is obtained by introducing a laser ranging module, and the dynamic weighted fusion strategy based on reprojection error is used to fuse it with the relative Z-axis coordinates calculated by the PnP algorithm pose to output the final result.

[0065] A method for guiding a drone to autonomously land on a ship's deck using a combination of multi-source sensors, such as... Figure 1 As shown, based on the relative distance between the drone and the ship, the drone's landing process is divided into four guidance stages, each corresponding to a different sensor combination and navigation strategy, to guide the drone to autonomously land on the ship's deck.

[0066] Initial guidance phase based on GPS and IMU: In the long-range beyond-line-of-sight phase (the distance between the UAV and the target ship is greater than 2 kilometers), the initial guidance is completed by acquiring the GPS coordinates of the target ship and the UAV itself, combined with IMU inertial navigation, and using the designed extended Kalman filter state estimation and error compensation mechanism.

[0067] Visual capture phase based on monocular vision system: As the UAV approaches the ship and enters the line-of-sight range (distance between the UAV and the target ship is 100~2000m), the guidance strategy is switched to monocular vision, and the onboard optical camera is used to capture and coarsely locate the ship, realizing the initial takeover of visual navigation.

[0068] Cooperative target locking phase based on binocular system: After entering the medium-to-close range phase (distance between the UAV and the target ship is 20m~100m), the binocular vision system is activated to identify the preset landing target on the deck, and the UAV is guided to approach the landing target based on visual servoing.

[0069] High-precision positioning and landing phase integrating laser altimetry and key points of the landing target: During the landing phase on the ship deck (the height distance between the UAV and the target ship is less than 20m), the pose calculation results based on the PnP algorithm and the height information obtained by the laser altimetry sensor are used to analyze the reprojection error and dynamically adjust the fusion weight of laser and visual data to achieve high-precision three-dimensional pose positioning and control of the UAV, guiding the UAV to land on the ship deck.

[0070] like Figure 2 As shown, a combined navigation module consisting of GPS and IMU is used for initial guidance of the UAV during the initial long-distance navigation phase. During this initial phase, GPS signals may experience issues such as low update frequency and short-term interruptions. To enhance the system's state estimation capability and navigation robustness, an extended Kalman filter is introduced to dynamically fuse GPS and IMU data. This filtering algorithm jointly estimates position, velocity, and attitude, maintaining track continuity and navigation stability even in the event of temporary GPS failure or error fluctuations.

[0071] The ship periodically acquires and updates its own global positioning information, broadcasting it in real time to the UAV in the air via a data link. After acquiring its own GPS information, the UAV calculates the spatial three-dimensional coordinate difference between itself and the ship, i.e., the navigation target vector, by comparing it with the received ship position information. Based on the calculated relative position vector, the desired yaw angle and heading commands are calculated. Combined with the yaw control and forward speed control commands of the UAV dispatched by the flight controller, a closed-loop control link is constructed to continuously correct the flight trajectory and guide the UAV to gradually approach the ship from its initial position. Once the ship enters the UAV's line-of-sight range, if the UAV's flight altitude and attitude are stable, the visual sensors are functioning normally, and there are no obvious problems such as frame drops, defocusing, or exposure abnormalities during image acquisition, the navigation phase based on visual servoing can be switched.

[0072] Figure 3 This paper presents a state estimation and error compensation mechanism for a UAV based on extended Kalman filtering. The UAV's state variables are its position, velocity, and attitude in the world coordinate system.

[0073] ;

[0074] in, Spatial location of the drone; For the space velocity of the drone; For the roll angle, pitch angle, and yaw angle of the drone.

[0075] The current state is calculated based on the drone's previous state and current IMU measurements:

[0076] ;

[0077] in, The predicted state; This is an estimate of the state at the previous time step; Control input from the IMU; For the motion model function (Newtonian motion + attitude update); This refers to system process noise.

[0078] Uncertainty in state prediction also needs to be propagated, using a covariance matrix. Indicates the estimation error:

[0079] ;

[0080] in, Let be the Jacobian matrix of the state transition function with respect to the state; Let be the process noise covariance.

[0081] When GPS data arrives, the update phase begins, using the location measured by GPS. The prediction results are then corrected. The observation model is as follows:

[0082] ;

[0083] in, The actual measured value (location provided by GPS); For the observation function (here, the position part is extracted from the state); To measure noise, linearization is used to obtain the observation matrix. Then calculate the Kalman gain:

[0084] ;

[0085] in, Kalman gain; To observe the noise covariance, the state is updated by fusing the predicted state with GPS measurements:

[0086] ;

[0087] Finally, update the covariance matrix in preparation for the next iteration:

[0088] .

[0089] The above mechanism can maintain the continuity of the UAV's flight path and ensure smooth navigation output even when GPS is unstable or intermittently unavailable.

[0090] Figure 4 This is a vision-based guidance flowchart within the line-of-sight range. The NanoDet deep learning object detection model, which acquires images in real-time using a monocular camera, outputs the ship's image coordinates.

[0091] ;

[0092] in, The coordinates of the detection frame center; Width and height; Confidence level for the ship.

[0093] Once the ship's confidence level is detected to be greater than the confidence threshold, the center point of the ship and the center point of the optical axis of the camera image are calculated. Pixel offset error between:

[0094] ;

[0095] This is used as the basis for attitude adjustment, guiding the drone to adjust its orientation. The camera intrinsic parameter matrix is... Then the error can be normalized to the angular offset:

[0096] ;

[0097] in, , The yaw and pitch angle errors of the ship relative to the camera's optical axis are represented. These errors are mapped to control inputs to achieve continuous adjustment of the UAV's attitude.

[0098] Yaw rate command: ;

[0099] Pitch angle command: ;

[0100] in, For the yaw rate gain, This is the pitch angle gain.

[0101] As the ship occupies more of the area in the image, the target locking phase begins. Images are acquired using a binocular camera mounted on the drone. and .exist The NanoDet deep learning target detection model and target detection algorithm were used to obtain the center coordinates of the landing target. Through binocular stereo matching, the depth corresponding to the landing target position is:

[0102] ;

[0103] in, B is the camera focal length; B is the baseline length of the left and right cameras. This represents the disparity value at the location of the landing target. It is based on the position of the landing target in the image. With corresponding depth It can be converted into three-dimensional spatial coordinates:

[0104] ;

[0105] The corresponding forward propulsion velocity vector: ;

[0106] Yaw angle control: ;

[0107] Altitude-rate control: ;

[0108] in, For the camera intrinsic parameter matrix, For velocity vector gain, For high rate control gain, , , This refers to the deviations between the target and the drone in the horizontal, longitudinal, and vertical directions.

[0109] The aforementioned control input is filtered by a PID controller and then input into the flight control system to ensure that the UAV's movement direction continuously points towards the landing target position.

[0110] In the final landing phase, based on the locked landing target as a region of interest (ROI) for cropping and extraction, high-precision key point identification and pose calculation are achieved. First, OpenCV is used to perform Gaussian smoothing, threshold segmentation detection, and edge detection to complete the detection of key points within the landing target and the unique ID identification of the AprilTag pattern.

[0111] For the pixel coordinates of the key points The coordinates of each point are extracted precisely using a sub-pixel-level optimization algorithm. Specifically, this is used... A two-dimensional Gaussian function is used for fitting within the neighborhood of key points to obtain sub-pixel positions with pixel-level precision. For LED dots, an ellipse fitting method is used, which is called... Obtain high-precision circle center coordinates After completing the keypoint optimization, all detected points are integrated to construct a 3D-2D point correspondence:

[0112] ;

[0113] in, Then call The function calculates the pose between the UAV and its landing point on the ship deck by solving the PnP problem, and introduces the RANSAC algorithm to screen the input key point data for consistency, remove potential outliers, and improve the accuracy and stability of attitude estimation.

[0114] During the terminal phase of the drone's descent onto the ship's deck, it is necessary to accurately estimate the drone's vertical height relative to the deck. Considering that visual PnP estimation may have errors under conditions such as drastic attitude changes, in order to improve the accuracy of altitude estimation in the Z direction, real-time vertical distance data between the UAV and the deck obtained by the laser ranging module is introduced, and a dynamic weighted fusion strategy based on reprojection error is adopted to fuse it with the relative Z-axis coordinates output by PnP pose calculation.

[0115] The Z-axis displacement calculated by the PnP algorithm is denoted as:

[0116] ;

[0117] in The Z-axis translation in the camera coordinate system represents the relative depth estimate obtained from the PnP attitude calculation.

[0118] For each point Projection error: ,in, For the first The actual observed pixel coordinates of each key point in the image. This represents the rotational relationship between the target coordinate system and the camera coordinate system. The three-dimensional position of the origin of the target coordinate system in the camera coordinate system; This is a projection function that projects points in three-dimensional space onto a two-dimensional image plane.

[0119] Average reprojection error: ;

[0120] Set the effective visual error range Then the fusion coefficient (visual weight) Defined as:

[0121] ;

[0122] when This indicates that the PnP solution is reliable and the visual weights are valid. ;

[0123] when When the visual estimation quality is poor, the visual weighting is reduced. Complete trust in lasers;

[0124] Linear interpolation is used for transition within the intermediate range.

[0125] The final output fused depth estimate in the Z direction is:

[0126] .

[0127] in, The vertical distance between the UAV and the deck is obtained by the laser ranging module.

[0128] This fusion strategy can adaptively adjust the confidence level of vision and laser ranging based on the fluctuation of the PnP reprojection error in each frame.

[0129] Figure 5This is the landing target pattern. The pattern uses a combined structure: the center of the landing target features an AprilTag pattern with a unique ID, used to identify the specific landing area of ​​the UAV on the ship's deck. A symmetrical "cross" structure is designed around the AprilTag pattern, with a black ring graphic to highlight the center and provide heading indication, enhancing the focusing effect of the visual detection algorithm on the area. Circular markers are placed at the four corners, each containing an infrared light source for clear visibility at night or in low-visibility environments. Several dotted lines connect the central pattern to the four corner circular areas for structural guidance. Multiple evenly distributed black square reflective films surround the entire edge of the landing target pattern, forming a coarse positioning auxiliary structure.

[0130] The method for guiding a drone to autonomously land on a ship deck using a combination of multi-source sensors disclosed in the above embodiments has the following advantages:

[0131] This paper proposes a guidance mechanism for autonomous landing of UAVs on ship decks. The mechanism dynamically switches navigation strategies based on distance range, achieving continuous and precise guidance from far to near. Beyond visual line of sight (BVR), a combination of GPS and IMU navigation is used for initial guidance. Once the ship enters BVR range, firstly, monocular vision is used for ship acquisition and coarse positioning. Then, binocular vision is used to identify landing targets deployed on the deck, focusing on the target area to improve visual guidance accuracy. Finally, PNP-based pose calculations and laser altimetry results are fused to achieve high-precision landing of the UAV on the ship deck. This scheme fully leverages the performance advantages of multi-source sensors at different stages, ensuring the safe landing of the UAV on the ship deck.

[0132] To address the issues of low GPS update frequency and signal interruption due to obstruction in long-range navigation, a state estimation and error compensation mechanism based on extended Kalman filtering is constructed. When GPS signal is unavailable, inertial navigation prediction is performed using angular velocity and acceleration measured by the IMU to ensure the continuity of state estimation. Once GPS is restored, observation updates are immediately performed to effectively correct accumulated errors and suppress inertial drift, thereby improving positioning reliability in the long-range phase.

[0133] A novel cooperative visual landing target is provided. The target features a unique AprilTag pattern at its center, used to identify the specific landing area of ​​a drone on a ship. A symmetrical "cross" structure surrounds the center, with a black ring around the perimeter to highlight the center and provide heading guidance. Several dotted lines connect the central pattern to the four corner circular areas for structural guidance. Circular markers at the four corners contain infrared light sources for clear visibility at night or in low-visibility environments, ensuring nighttime visibility. The target also features evenly distributed black square reflective film on its exterior, providing spatial reference for drone terminal guidance.

[0134] During the final stage of landing on the ship's deck, a laser altimeter assistance mechanism is introduced. By analyzing the reprojection error of the cooperative target, the fusion weights of laser and visual data are dynamically adjusted to achieve adaptive integration of laser altimeter information, ensuring the positioning stability and accuracy of the UAV during landing.

[0135] The technical solution of this application has been described in conjunction with the preferred embodiments shown in the accompanying drawings. Those skilled in the art should understand that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.

Claims

1. A method for guiding a drone to autonomously land on a ship deck using a combination of multi-source sensors, characterized in that, include: Initial guidance phase based on GPS and IMU: The ship is outside the line-of-sight range of the drone, and the drone is guided to fly towards the ship by a combined navigation module consisting of GPS and inertial measurement unit (IMU). Ship capture phase based on monocular vision: Once the ship enters the line-of-sight range of the drone, the lightweight deep learning target detection model NanoDet is used to identify and capture the ship at a long distance. Based on the change in the ship's relative position in the image, the drone's flight attitude is adjusted. Binocular vision-based target locking phase during descent: The drone gets closer to the ship and can observe the landing target on the ship's deck. Using the image pairs captured by the binocular camera, the NanoDet deep learning target detection model identifies the landing target on the ship's deck. The controller dynamically adjusts the drone's attitude angle, forward speed and vertical altitude to keep it aligned with the landing target in space. High-precision positioning landing phase: For the final landing phase, based on the locked landing target, high-precision key point identification and pose calculation are completed.

2. The method for guiding a UAV to autonomously land on a ship deck using a combination of multi-source sensors according to claim 1, characterized in that, During the initial guidance phase based on GPS and IMU, the ship periodically broadcasts its own position information and transmits it to the UAV in real time via a data link. After obtaining its own GPS positioning information, the UAV calculates the navigation target point based on its relative position with the ship, guiding the UAV to approach the ship from its current starting position.

3. The method for guiding a UAV to autonomously land on a ship deck using a combination of multi-source sensors according to claim 2, characterized in that, In the initial guidance phase based on GPS and IMU, multi-source sensor data is fused using an extended Kalman filter, specifically including: When GPS is unavailable, the drone's current position and attitude are predicted by high-frequency inertial integration based on acceleration and angular velocity data provided by the IMU. When GPS updates become available, they are used as observation inputs to correct previous predictions, thereby effectively suppressing the accumulation of errors caused by IMU integral drift and achieving dynamic correction of the UAV's status.

4. The method for guiding a drone to autonomously land on a ship deck using a combination of multi-source sensors according to claim 3, characterized in that, In the landing target locking stage based on binocular vision, the landing target detection box is optimized and expanded, and the region of interest is extracted containing the complete outline of the landing target. The disparity between matching points in the left and right eye images of the region of interest is calculated to estimate the three-dimensional spatial position of the landing target.

5. The method for guiding a drone to autonomously land on a ship deck using a combination of multi-source sensors according to claim 4, characterized in that, In the target locking phase based on binocular vision, the center point of the region of interest in the current detection frame is extracted, and the pixel offset error between the center point of the optical axis of the camera image is calculated. By fusing pixel offset error with depth information recovered by stereo matching, the spatial error between the UAV and the landing target is obtained. As a feedback input, the controller dynamically adjusts the UAV's attitude angle, forward speed and vertical altitude, so that the UAV continuously aligns with the landing target in space and approaches the preset landing guidance point.

6. The method for guiding a drone to autonomously land on a ship deck using a combination of multi-source sensors according to claim 5, characterized in that, During the high-precision positioning landing phase, key points within the landing target are detected, and sub-pixel-level optimization algorithms are used to finely extract the coordinates of each point. A 3D-2D point correspondence relationship is constructed, the PnP algorithm is called to calculate the pose between the UAV and the landing target landing point, and the RANSAC algorithm is introduced to screen key points for consistency and eliminate potential outliers.

7. The method for guiding a drone to autonomously land on a ship deck using a combination of multi-source sensors according to claim 6, characterized in that, During the high-precision positioning and landing phase, the real-time vertical distance between the UAV and the deck is obtained by a laser ranging module. The dynamic weighted fusion strategy based on reprojection error is used to fuse the relative Z-axis coordinates calculated by the PnP algorithm pose to output the final result.

8. The method for guiding a drone to autonomously land on a ship deck using a combination of multi-source sensors according to claim 7, characterized in that, Using a dataset of long-distance images of ships covering various perspectives, sea conditions, and lighting conditions, the NanoDet deep learning object detection model was iteratively trained until the model converged. The NanoDet deep learning object detection model outputs the coordinates of the detection box of the ship in the image coordinate system.

9. The method for autonomously landing a UAV on a ship deck guided by a combination of multi-source sensors according to claim 8, characterized in that, The trained deep learning object detection model NanoDet was converted into TensorRT format and deployed on the JetsonOrin embedded computing platform carried by the drone to achieve real-time ship detection.

10. The method for guiding a UAV to autonomously land on a ship deck using a combination of multi-source sensors according to claim 9, characterized in that, The center of the landing target features an AprilTag pattern with a unique ID; The AprilTag pattern features a symmetrical "cross" structure around its perimeter, along with a black ring graphic. The black ring-shaped graphic has circular markers at its four corners, and an infrared light source is installed inside. The landing target is surrounded by several black square reflective films.

Images