Visual guidance landing method and system from unmanned aerial vehicle to unmanned ship

Abstract

The invention provides a visual guidance landing method and system from an unmanned aerial vehicle to an unmanned ship, and relates to the technical field of unmanned system autonomous control, and the method comprises the steps: collecting a target landing region image; according to the target landing area image, through a YOLO11-OBB algorithm, identifying rotation bounding box information of the unmanned ship, and determining an initial guide pose; controlling the unmanned aerial vehicle to move towards the direction of the unmanned ship according to the initial guide pose, and entering a switching threshold region; in the switching threshold region, the unmanned aerial vehicle is controlled to move in the direction of the unmanned ship through double-model parallel detection and a weighted decision model; under the condition that the distance between the unmanned aerial vehicle and the unmanned ship is smaller than a preset distance value, the final guide pose of the unmanned aerial vehicle relative to the unmanned ship is solved through an AprilTag algorithm; and driving the unmanned aerial vehicle to land on the unmanned ship according to the final guide pose. The method can improve the remote positioning precision of the unmanned aerial vehicle, achieves the smooth switching of algorithms, and guarantees the landing stability of the unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of autonomous control technology for unmanned systems, and in particular to a visually guided landing method and system for unmanned aerial vehicles (UAVs) to unmanned vessels. Background Technology

[0002] With the accelerated development of marine resources and the intelligentization of maritime operations, the collaborative operation mode of UAVs and unmanned vessels has become a research hotspot. Among them, the precise landing of UAVs on the decks of mobile unmanned vessels is a core key technology for achieving collaboration between the two. UAVs complete landing through visual guidance, which has advantages such as low cost, fast response, and strong environmental adaptability. It can be widely used in scenarios such as marine monitoring, material delivery, and emergency rescue. Its landing accuracy and stability directly determine the efficiency and safety of collaborative operations. Therefore, the development of efficient and reliable UAV visual guidance landing technology has significant practical application value.

[0003] In existing technologies, UAV vision-guided landing schemes are mainly divided into two categories: one is a long-range coarse positioning scheme based on target detection algorithms, such as using YOLO series algorithms to identify unmanned surface vessel targets, outputting the target's bounding box information and calculating the initial guidance pose to achieve long-range approximation of the UAV; the other is a short-range fine positioning scheme based on artificial marking, such as using marking algorithms such as AprilTag and ArUco to identify marker points on the deck of the unmanned surface vessel, and combining the PnP algorithm to output high-precision pose to complete the final landing.

[0004] However, traditional target detection algorithms use axis-aligned bounding boxes to detect rotating targets. The angle representation has periodic ambiguity and gradient instability, resulting in large errors in long-distance pose calculation. Furthermore, the algorithm switching is triggered only by a fixed distance threshold, without taking into account key influencing factors such as environmental interference and image quality, which can easily cause instantaneous changes in the control process. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a visually guided landing method for UAVs to unmanned ships. This method can solve the problems of traditional target detection algorithms that use axis-aligned bounding boxes to detect rotating targets, which have periodic ambiguities and gradient instability in angle representation, resulting in large errors in long-distance pose calculation. Furthermore, the algorithm switching is triggered only by a fixed distance threshold, without taking into account key influencing factors such as environmental interference and image quality, which can easily lead to instantaneous changes in the control process.

[0006] A first aspect of this invention provides a visually guided landing method for transferring a drone to an unmanned surface vessel, comprising: S1: Acquire images of the target landing area; S2: Based on the target landing area image, the YOLO11-OBB algorithm is used to identify the rotating bounding box information of the unmanned vessel and determine the initial guidance pose; S3: Based on the initial guidance pose, control the drone to move towards the unmanned vessel and enter the switching threshold area; S4: In the switching threshold region, the UAV is controlled to move towards the unmanned vessel through a dual-model parallel detection and weighted decision model; S5: When the distance between the UAV and the unmanned vessel is less than the preset distance value, the final guidance pose of the UAV relative to the unmanned vessel is calculated using the AprilTag algorithm. S6: Based on the final guided pose, drive the drone to land on the unmanned vessel.

[0007] A second aspect of the present invention provides a visually guided landing system for unmanned aerial vehicles (UAVs) to unmanned surface vessels (USVs), comprising: a processor and a memory; The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the visually guided landing method for UAV-to-unmanned vessel as described in the first aspect.

[0008] A third aspect of the present invention provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the visually guided landing method for unmanned aerial vehicles to unmanned surface vessels as described in the first aspect.

[0009] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, the YOLO11-OBB algorithm is used to replace the traditional axis-aligned bounding box detection method, which can alleviate the periodic ambiguity of angle representation and gradient instability, thereby improving the detection accuracy of the unmanned vessel's rotating bounding box information and reducing the long-distance pose calculation error. Introducing a dual-model parallel detection and weighted decision model in the switching threshold region can reduce control fluctuations during algorithm switching and improve the stability of the UAV's motion process. Attached Figure Description

[0010] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0011] Figure 1 This is a flowchart illustrating a visually guided landing method for transferring a drone to an unmanned vessel, provided in an embodiment of the present invention.

[0012] Figure 2 This is a schematic diagram of the overall system architecture and air-sea collaborative communication provided by an embodiment of the present invention.

[0013] Figure 3 This is a hardware connection topology diagram of a drone terminal and an unmanned vessel terminal provided in an embodiment of the present invention.

[0014] Figure 4 This is a schematic diagram of a visually guided landing system for unmanned aerial vehicles (UAVs) to unmanned ships, provided in an embodiment of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] The visually guided landing method for unmanned aerial vehicles (UAVs) to unmanned surface vessels provided by the present invention will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0017] Reference manual attached Figure 1 The diagram shows a flowchart of a visually guided landing method for unmanned aerial vehicles (UAVs) to unmanned surface vessels, provided by an embodiment of the present invention.

[0018] Reference manual attached Figure 2 The diagram illustrates a system architecture and air-sea collaborative communication schematic provided by an embodiment of the present invention.

[0019] Reference manual attached Figure 3 The diagram illustrates a hardware connection topology between a drone and an unmanned vessel provided in an embodiment of the present invention.

[0020] This invention provides a visually guided landing method for unmanned aerial vehicles (UAVs) to unmanned surface vessels (USVs), which may include the following steps: S1: Acquire images of the target landing area.

[0021] S2: Based on the target landing area image, the YOLO11-OBB algorithm is used to identify the rotating bounding box information of the unmanned vessel and determine the initial guidance pose.

[0022] The YOLO11-OBB algorithm is an improved version of the YOLO11 target detection algorithm for rotating target scenarios. Based on the traditional axis-aligned bounding box (AABB), it adds a rotation angle parameter to achieve accurate detection and localization of targets rotating at any angle. Using the target landing area image captured by the UAV's downward-facing camera as input, image features are extracted through a backbone network, and then the rotation bounding box information of the target is output through feature fusion and a detection head. This information includes the center pixel coordinates of the UAV target, the bounding box length, the bounding box width, and the hull rotation angle. To address the periodic boundary ambiguity of the rotation angle, the algorithm employs an angle encoding scheme based on unit circle mapping, converting the rotation angle into a two-dimensional encoded vector. This encoded vector is then optimized using a gradient correction loss function to eliminate jump errors during angle regression. Finally, based on the optimized rotation bounding box information, the initial guidance pose of the UAV relative to the UAV is calculated through inverse projection using a pinhole imaging model.

[0023] It should be noted that within the first distance range (30-100 meters), the YOLO11-OBB algorithm is activated to identify the rotating bounding box information (center point, size, rotation angle) of the unmanned vessel as the initial guiding pose.

[0024] Furthermore, in the far-field phase, within a range of 30-100 meters from the target, the system calls the RK3588's NPU unit to run the quantized YOLO11-OBB model.

[0025] In one possible implementation, S2 specifically includes sub-steps S201 to S203: S201: Input the target landing area image into the YOLO11-OBB model and output the rotated bounding box information of the unmanned vessel. The rotated bounding box information specifically includes: the pixel coordinates of the center point of the unmanned vessel, the length of the bounding box, the width of the bounding box, and the rotation angle of the hull.

[0026] Specifically, the rotated bounding box information is as follows: in, Represents the rotated bounding box vector. Represents pixel coordinates X Axis coordinates Represents pixel coordinates Y Axis coordinates w Indicates the length of the bounding box. h Indicates the width of the bounding box. Indicates the angle of rotation of the ship's hull.

[0027] S202: By using the pinhole imaging model, the coordinates of the center point pixel are inversely projected to determine the position of the UAV in the camera coordinate system.

[0028] Among them, the pinhole imaging model is a basic geometric model that connects image pixel coordinates with three-dimensional physical coordinates. In the scenario of UAV vision-guided landing, it is used to convert the pixel coordinates of the center point of the UAV's rotating bounding box detected by the YOLO11-OBB algorithm into the three-dimensional relative position of the UAV with respect to the UAV, and then solve the initial guidance pose.

[0029] S203: Determine the position of the UAV in the camera coordinate system and the hull rotation angle as the initial guidance pose.

[0030] Specifically, the position of the drone in the camera coordinate system is as follows: in, Indicates the camera coordinate system X Relative position of axes Indicates the camera coordinate system Z Absolute height of the axis Represents pixel coordinates X Axis coordinates Indicates the principal point of the camera X Axis coordinates Indicates camera X Axis-normalized focal length, Indicates the camera coordinate system Y Relative position of axes Represents pixel coordinates Y Axis coordinates Indicates the principal point of the camera Y Axis coordinates Indicates camera Y Axis-normalized focal length.

[0031] In one possible implementation, it also includes: S204: The hull rotation angle is encoded using an angle encoding method based on unit circle mapping to obtain the encoded vector of the hull rotation angle.

[0032] Specifically, the encoding vector for the hull rotation angle is as follows: in, The encoded vector representing the hull rotation angle, sin ( ) represents the sine function. cos ( ) represents the cosine function. Indicates the angle of rotation of the ship's hull.

[0033] Among them, the unit circle-based angle encoding method is an unambiguous angle representation method designed for rotation angle regression tasks. It transforms a one-dimensional rotation angle into a two-dimensional vector through trigonometric function mapping, thereby solving the regression error problem caused by periodicity and boundary jumps in traditional angle regression. Utilizing the periodicity and orthogonality of sine and cosine functions, the rotation angle is mapped to a point on the unit circle, and the corresponding encoding vector is... ,in The settings can further improve the resolution of angle representation, ensuring that adjacent angles have continuous distances in the vector space.

[0034] It should be noted that the angle encoding scheme based on unit circle mapping can solve the periodic boundary problem of angle regression.

[0035] S205: Based on the encoded vector, the angle prediction accuracy of the YOLO11-OBB model is optimized using the gradient correction loss function.

[0036] The Gradient Correction Loss Function (GCL) is a loss calculation method designed for optimizing rotation angle encoded vectors, addressing the gradient instability problem that occurs during model training for angle encoded vectors based on unit circle mapping. In the UAV vision-guided landing scenario, the function uses the rotation angle encoded vector output by the YOLO11-OBB algorithm as the optimization object. By weighting or constraining the error between the predicted and true values ​​of the encoded vector, it avoids gradient explosion, gradient vanishing, or optimization direction confusion caused by the periodicity of trigonometric functions.

[0037] It should be noted that introducing the gradient correction loss function (GCL) can optimize the accuracy of angle prediction for slender ship targets.

[0038] In this embodiment of the invention, through customized improvements and multi-stage precise optimization of the YOLO11-OBB algorithm, the quantization model is efficiently run by the RK3588 NPU unit in the 30-100 meter far-field stage. This achieves accurate detection and positioning of the unmanned surface vessel rotating at any angle. Furthermore, the angle encoding method based on unit circle mapping solves the periodic boundary ambiguity problem of rotation angle regression. Combined with the gradient correction loss function, the angle prediction accuracy of the slender hull target is further optimized, avoiding the gradient instability problem caused by the periodicity of trigonometric functions. Finally, through the inverse projection calculation of the pinhole imaging model, the rotation bounding box information at the two-dimensional pixel level is transformed into the three-dimensional initial guiding pose of the UAV relative to the unmanned surface vessel. This provides a stable and accurate pose reference for the UAV to approach the unmanned surface vessel at a long distance, while taking into account both the real-time performance of the algorithm and the reliability of the pose calculation.

[0039] S3: Based on the initial guidance pose, control the drone to move towards the unmanned vessel and enter the switching threshold area.

[0040] In this embodiment of the invention, the initial guiding pose provides clear heading guidance for the UAV, driving the UAV to smoothly approach the switching threshold area from the far field stage of 30-100 meters. This not only ensures the directional accuracy of the UAV's movement process and avoids flight path deviations caused by target loss or positioning errors, but also allows the UAV to accurately enter the preset scenario for subsequent algorithm switching. This creates the necessary conditions for a smooth transition from the YOLO11-OBB algorithm to the AprilTag algorithm, ensuring uninterrupted and error-free pose guidance for the UAV throughout the entire process from long-distance coarse positioning to short-distance fine positioning.

[0041] S4: In the switching threshold region, the UAV is controlled to move towards the unmanned ship through a dual-model parallel detection and weighted decision model.

[0042] Among them, the weighted decision model is the core decision mechanism for achieving smooth switching between the YOLO11-OBB algorithm and the AprilTag algorithm during the visual-guided landing process of the UAV. Based on the relative distance between the UAV and the unmanned vessel and the output quality of the two algorithms, it dynamically determines the optimal pose calculation algorithm to ensure seamless connection between long-range coarse positioning and short-range fine positioning.

[0043] It should be noted that within the switching threshold range (5-30 meters), the switching of the dominant algorithm is controlled by a weighted decision model, which comprehensively evaluates the detection confidence, marker recognition quality, and image clarity.

[0044] Furthermore, when the drone descends to the 5-30 meter range, the system runs the YOLO11-OBB and AprilTag algorithms in parallel. To avoid control jumps during algorithm switching, a switching weight function is constructed.

[0045] Optionally, the switching weight function in the weighted decision model is constructed as follows: Based on the target detection confidence score output by the YOLO11-OBB algorithm and the Hamming distance decoding quality index output by the AprilTag algorithm, a switching weight function is constructed.

[0046] The specific weight switching function is as follows: in, W Indicates the switching weight function. Indicates the regulating factor. This represents the target detection confidence score of the YOLO11-OBB algorithm. This represents the Hamming distance decoding quality metric for the AprilTag algorithm.

[0047] The adjustment factor is constructed using a nonlinear mapping based on the Sigmoid function, specifically as follows: in, k This represents the curve slope adjustment factor. d This indicates the current relative distance between the drone and the unmanned vessel. Indicates the center distance of the switching threshold region. This indicates the maximum distance within the switching threshold region. This indicates the minimum distance within the switching threshold region.

[0048] It should be noted that the curve slope adjustment factor is used to adjust the slope of the weight change, and its value depends on the span of the switching threshold region.

[0049] It should be noted that the adjustment factor changes rapidly in the middle of the switching region, while changing gradually at both ends. This further eliminates abrupt changes in derivatives at the boundaries, making the UAV flight control smoother. The switching weight function is the core mathematical expression of the weighted decision model, used to quantitatively evaluate the output quality of the YOLO11-OBB algorithm and the AprilTag algorithm, providing a quantifiable basis for algorithm switching during UAV vision-guided landing. In the formula, This represents an adjustment factor between 0 and 1, used to dynamically balance the weight ratios of the two algorithms in the long-range phase. Larger values ​​indicate a greater emphasis on the target detection confidence of the YOLO11-OBB algorithm. This ensures the effectiveness of large-scale coarse positioning. (Close-range phase) Decreasing the value emphasizes the Hamming distance decoding quality metric of the AprilTag algorithm. This ensures the reliability of high-precision positioning. When the drone approaches the unmanned vessel and enters the switching threshold area, as... reduce, As the weight value increases, it triggers a preset threshold condition, thereby completing a smooth switch from the YOLO11-OBB algorithm to the AprilTag algorithm, achieving a seamless connection between long-range coarse positioning and short-range fine positioning.

[0050] It should be noted that when continuous N When the frame is higher than the threshold and the distance is less than the set fine-tuning distance (e.g., 10 meters), the system completely shuts down the YOLO process on the NPU and transfers control completely to the AprilTag solving module on the CPU.

[0051] In this embodiment of the invention, by employing a dual-model parallel detection combined with a weighted decision model within the 5-30 meter switching threshold region, the design leverages the wide-range target acquisition capability of the YOLO11-OBB algorithm and the high-precision positioning advantage of the AprilTag algorithm, achieving complementary advantages between the two algorithms. Furthermore, it utilizes a switching weight function to dynamically balance the output weights of the two algorithms, combining multiple dimensions such as detection confidence and marker recognition quality to achieve smooth algorithm switching, effectively avoiding control jump problems that may occur during single-algorithm switching. Simultaneously, by setting shutdown conditions, the YOLO process is shut down while ensuring positioning accuracy, reducing system computing power consumption. Ultimately, this ensures uninterrupted, high-precision guidance for the UAV from long-range coarse positioning to short-range fine positioning, laying a stable and reliable foundation for subsequent precise landing.

[0052] S5: When the distance between the UAV and the unmanned vessel is less than the preset distance value, the final guidance pose of the UAV relative to the unmanned vessel is calculated using the AprilTag algorithm.

[0053] The AprilTag algorithm is a high-precision pose calculation algorithm based on two-dimensional artificial markers, specifically designed for close-range visual guidance landing scenarios for UAVs. Its core function is to output the final guided pose of the UAV relative to the unmanned surface vessel. Using images acquired after the UAV enters the switching threshold region (close-range phase) as input, the algorithm first identifies AprilTag markers in the image and extracts their corner coordinates through a feature extraction module. Then, it matches the image pixel corners with the real-world coordinates of the markers. Subsequently, it preliminarily calculates the rotation matrix and translation vector based on homography matrix decomposition. Finally, it iterative optimization is performed by combining the Perspective-n-Point (PnP) algorithm with inertial measurement unit (IMU) data to minimize corner reprojection errors, thereby outputting millimeter-level precision 6-DoF (six degrees of freedom) pose information.

[0054] Those skilled in the art can set the preset distance value according to actual needs, and the present invention does not limit this.

[0055] It should be noted that during the close-range phase (<10 meters), the AprilTag algorithm is run using the RK3588 CPU. By extracting feature corner points from the image and matching them with the geometric corner points marked by the AprilTag algorithm in the real physical world, the PnP algorithm is used to solve the camera pose relative to the AprilTag algorithm mark, thereby obtaining the high-precision six-degree-of-freedom (6-DoF) pose of the UAV relative to the deck.

[0056] Furthermore, the AprilTag algorithm establishes a homography matrix by establishing the correspondence between the camera intrinsic parameter matrix and the marked corner points. After obtaining the rotation matrix and translation vector through matrix decomposition, it combines the PnP (Perspective-n-Point) algorithm with inertial measurement unit (IMU) data to complete the pose iterative optimization.

[0057] In one possible implementation, S5 specifically includes sub-steps S501 to S504: S501: Extract feature corner points from the target landing area image using the AprilTag algorithm.

[0058] S502: Match the feature corner points with the geometric corner points marked by the AprilTag algorithm in the real physical world.

[0059] S503: Based on the matching results, the pose of the UAV relative to the AprilTag algorithm marker is solved using the PnP algorithm.

[0060] Among them, the PnP algorithm is a 3D pose calculation algorithm based on multiple sets of corresponding points. It solves the rotation matrix and translation vector of the camera coordinate system relative to the world coordinate system by the correspondence between 2D pixels in the image plane and 3D spatial points in the real physical world. It is the core link of the AprilTag algorithm in visual-guided landing of UAVs to solve the final guidance pose. In the close landing scenario of UAVs, the known geometric corner points (3D world coordinates) marked by AprilTag and the feature corner points (2D pixel coordinates) extracted from the image are used as inputs. The projection equation is constructed using the camera intrinsic parameter matrix. The solution is iteratively solved by minimizing the reprojection error of the corner points, and finally outputs high-precision 6-DoF pose information.

[0061] Optionally, the projection equation of the PnP algorithm is calculated as follows: Construct the camera projection matrix.

[0062] Obtain the homogeneous world coordinates of the unmanned vessel deck markers.

[0063] Calculate the product of the camera projection matrix and the homogeneous world coordinates.

[0064] Based on the product result and combined with the scale factor, the homogeneous pixel coordinates of the unmanned vessel deck marker points are obtained, and the projection transformation is completed.

[0065] The specific formula for calculating the projection equation of the PnP algorithm is as follows: in, s Indicates the scale factor. Represents homogeneous pixel coordinates. K Represents the intrinsic parameter matrix. R Represents the rotation matrix.t Represents the translation vector. Represents homogeneous world coordinates.

[0066] S504: Based on the solution results, the final guiding pose is obtained.

[0067] In this embodiment of the invention, during the close-range phase where the distance between the UAV and the unmanned vessel is less than a preset value, the high-precision AprilTag algorithm is run on the CPU of the RK3588. By combining feature corner matching, homography matrix decomposition with the PnP algorithm, and incorporating inertial measurement unit (IMU) data for iterative optimization, it can output a high-precision six-degree-of-freedom (6-DoF) final guidance pose with millimeter-level accuracy. This not only makes up for the insufficient positioning accuracy of the YOLO11-OBB algorithm in the long-range phase, but also constructs a precise mapping relationship between pixel coordinates and three-dimensional spatial coordinates through the projection equation of the PnP algorithm, minimizing corner reprojection errors. At the same time, it adapts to the stringent requirements of high positioning accuracy for close-range landing, providing a key high-precision pose reference for the UAV to finally land smoothly and accurately on the deck of the unmanned vessel.

[0068] In one possible implementation, after S5, the following is also included: The entire process employs a Kalman filter to fuse inertial measurement unit data and visual data. Specifically, the visual data includes the initial and final guided poses.

[0069] The Kalman filter is a multi-source data fusion algorithm based on a "prediction-correction" closed loop. It integrates initial guidance pose (YOLO11-OBB algorithm output), final guidance pose (AprilTag algorithm output), and inertial measurement unit (IMU) data to generate smooth and stable UAV landing control commands. First, a state vector containing position, velocity, attitude quaternions, and zero biases in IMU acceleration and angular velocity is constructed. The current state and covariance matrix are predicted using the equation of motion combined with high-frequency IMU data. Then, the mapping relationship between the state vector and the visual pose is established using the observation Jacobian matrix, and the Kalman gain is calculated to balance the reliability of the predicted state (IMU data driven) and the visual observation (pose data driven). Finally, the predicted state is corrected using the observation residuals to obtain the optimal state estimate and update the covariance matrix. This effectively overcomes the limitations of a single sensor and ultimately outputs continuous and stable control commands, ensuring the UAV lands accurately and safely on the unmanned vessel deck.

[0070] It should be noted that using a Kalman filter to loosely or tightly couple visual data with inertial measurement unit (IMU) data can maintain the continuity of pose estimation during visual signal loss or algorithm switching intervals.

[0071] Kalman filters are specifically used for: Establish the state vector of the Kalman filter.

[0072] Specifically, the state vector of the Kalman filter is: in, express k The state vector at time t, express k Location at any given moment express k The speed of time express k The attitude quaternion at time, express k The acceleration at any given moment is zero bias. express k The angular velocity at time t is zero bias. T This indicates transpose.

[0073] It should be noted that the state vector of the Kalman filter is used to fully characterize the UAV in... k The motion state and sensor error state at any given time provide a unified state description benchmark for multi-source data fusion.

[0074] The state vector is predicted based on the data from the inertial measurement unit.

[0075] The specific formula for predicting the state vector is as follows: in, Indicates in k Under the optimal state estimation at time -1, k The predicted state vector at time t. Representing the equation of motion, express k The optimal state estimate at time -1. express k The sensor input vector at time t.

[0076] It should be noted that in the scenario of visually guided landing of UAVs, the predicted state vector is derived from the high-frequency motion data collected by the IMU and combined with the rigid body kinematics of the UAV. The predicted state at the current moment is derived from the historical best state, which provides a benchmark for visual correction in the subsequent measurement update stage. At the same time, it makes up for the defect of large inter-frame intervals in visual data and ensures the continuity of state estimation.

[0077] Calculate the Kalman gain.

[0078] Specifically, the Kalman gain is: in, expressk Kalman gain at time step Indicates in k Under the optimal state covariance matrix at time -1, k The predicted state covariance matrix at time t. express k The transpose of the observation Jacobian matrix at time t. express k The Jacobian matrix of the observation at time t, This represents the visual observation noise covariance matrix.

[0079] In this context, the Kalman gain is the core weight matrix of the Kalman filter during the measurement update phase. Its role is to dynamically balance the reliability of the IMU's predicted state with the credibility of the visual observation data, thereby determining the correction level for the predicted state. In the scenario of UAV vision-guided landing, the prediction covariance matrix in the Kalman gain formula... This represents the uncertainty of the IMU's predicted state; a larger value indicates a less reliable prediction. The visual observation noise covariance matrix... This represents the measurement error of visual pose (initial / final guided pose). A larger value indicates that the observation data is less reliable. When the IMU prediction drift is large or the visual observation is accurate, the Kalman gain will increase accordingly, making the filter rely more on visual observation to correct the prediction state. Conversely, the gain will decrease to preserve more of the stability of the IMU's high-frequency prediction, ultimately achieving optimal fusion of multi-source data and providing a smooth and reliable control basis for UAV landing.

[0080] Based on the Kalman gain, when the initial or final guided pose is a valid pose, the prediction result is updated to obtain the optimal state estimate.

[0081] Optionally, the optimal state estimate is calculated as follows: The prediction results are then fed into the nonlinear observation model to obtain the predicted observation values.

[0082] The optimal state estimate is calculated based on the effective pose, the observed predicted value, the Kalman gain, and the prediction result.

[0083] Specifically, the optimal state estimation is as follows: in, express k Optimal state estimation at time t. Indicates in k Under the optimal state estimation at time -1, k The predicted state vector at time t. express k Kalman gain at time step Indicates the valid pose. This represents a nonlinear observation model.

[0084] It should be noted that the optimal state estimation uses Kalman gain to weight and fuse the predicted state with the visual observation residuals, resulting in the final output. k The optimal state estimate at any given time. In the scenario of UAV vision-guided landing, the optimal state estimate, through the closed-loop logic of "prediction and correction", deeply fuses the pose output by the YOLO11-OBB algorithm and the AprilTag algorithm with the inertial measurement unit (IMU) data, effectively suppressing IMU drift and visual noise, and finally outputting a smooth and accurate UAV state, providing a reliable basis for the generation of landing control commands.

[0085] In this embodiment of the invention, a Kalman filter is introduced throughout the entire process to achieve deep fusion of inertial measurement unit (IMU) data and visual data (initial / final guided pose). Utilizing a closed-loop "prediction-correction" logic and comprehensive state vector representation, the continuity of IMU high-frequency predictions and the accuracy of visual pose are dynamically balanced. This not only compensates for the large inter-frame intervals and susceptibility to environmental interference in visual data through IMU data, but also suppresses IMU drift errors through visual pose correction. Simultaneously, during visual signal loss or algorithm switching intervals, the filter maintains the continuity of pose estimation, avoiding control jumps, and ultimately outputs a smooth, stable, and highly accurate optimal state estimate. This provides reliable control command support for the entire process of UAV approaching from a long distance to precise close-range landing, significantly improving the stability and safety of the landing process.

[0086] S6: Based on the final guided pose, drive the drone to land on the unmanned vessel.

[0087] Reference manual attached Figure 4 The diagram shows a structural schematic of a visually guided landing system for unmanned aerial vehicles (UAVs) to unmanned surface vessels provided in an embodiment of the present invention.

[0088] This invention provides a visually guided landing system 20 for unmanned aerial vehicles (UAVs) to unmanned surface vessels (USVs), comprising: a processor 201 and a memory 202; The memory 202 stores programs or instructions that can run on the processor 201. When the program or instructions are executed by the processor 201, they implement the steps of the above-described visual-guided landing method from UAV to unmanned vessel and achieve the same technical effect. To avoid repetition, the present invention will not elaborate further.

[0089] It should be understood that the processor 201 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0090] It should also be understood that the memory 202 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM).

[0091] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0092] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0093] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0095] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0096] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0097] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0098] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] This invention provides a readable storage medium that stores a program or instructions on the storage medium. When the program or instructions are executed by a processor, they implement the steps of the above-described visually guided landing method for unmanned aerial vehicles to unmanned ships, and achieve the same technical effect. To avoid repetition, this invention will not elaborate further.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A visually guided landing method for unmanned aerial vehicles (UAVs) to unmanned surface vessels (USVs), characterized in that, include: S1: Acquire images of the target landing area; S2: Based on the target landing area image, the YOLO11-OBB algorithm is used to identify the rotating bounding box information of the unmanned vessel and determine the initial guidance pose; S3: Based on the initial guidance pose, control the UAV to move towards the unmanned vessel and enter the switching threshold region; S4: In the switching threshold region, the UAV is controlled to move towards the unmanned vessel using a dual-model parallel detection and weighted decision model; S5: If the distance between the UAV and the unmanned vessel is less than a preset distance value, calculate the final guidance pose of the UAV relative to the unmanned vessel using the AprilTag algorithm; S6: Based on the final guided pose, drive the drone to land on the unmanned vessel.

2. The visually guided landing method for unmanned aerial vehicles (UAVs) to unmanned surface vessels according to claim 1, characterized in that, S2 specifically includes: S201: Input the target landing area image into the YOLO11-OBB model and output the rotation bounding box information of the unmanned vessel. The rotation bounding box information specifically includes: the center point pixel coordinates of the unmanned vessel, the bounding box length, the bounding box width, and the hull rotation angle. S202: By using the pinhole imaging model, the coordinates of the center point pixel are inversely projected to determine the position of the UAV in the camera coordinate system; S203: Determine the position of the UAV in the camera coordinate system and the hull rotation angle as the initial guiding pose.

3. The visually guided landing method for unmanned aerial vehicles (UAVs) to unmanned surface vessels according to claim 2, characterized in that, Also includes: S204: The hull rotation angle is encoded using an angle encoding method based on unit circle mapping to obtain the encoded vector of the hull rotation angle; S205: Based on the encoded vector, the angle prediction accuracy of the YOLO11-OBB model is optimized using the gradient correction loss function.

4. The visually guided landing method for unmanned aerial vehicles (UAVs) to unmanned surface vessels according to claim 1, characterized in that, The switching weight function in the weighted decision model is constructed as follows: The switching weight function is constructed based on the target detection confidence score output by the YOLO11-OBB algorithm and the Hamming distance decoding quality index output by the AprilTag algorithm.

5. The visually guided landing method for unmanned aerial vehicles (UAVs) to unmanned surface vessels according to claim 1, characterized in that, S5 specifically includes: S501: Extract the feature corner points of the target landing area image using the AprilTag algorithm; S502: Match the feature corner points with the geometric corner points marked by the AprilTag algorithm in the real physical world; S503: Based on the matching results, the pose of the UAV relative to the AprilTag algorithm marker is solved using the PnP algorithm; S504: Based on the solution results, the final guiding pose is obtained.

6. The visually guided landing method for unmanned aerial vehicles (UAVs) to unmanned surface vessels according to claim 5, characterized in that, The specific calculation method of the projection equation of the PnP algorithm is as follows: Construct the camera projection matrix; Obtain the homogeneous world coordinates of the unmanned vessel deck markers; Calculate the product of the camera projection matrix and the homogeneous world coordinates; Based on the product result and combined with the scale factor, the homogeneous pixel coordinates of the unmanned vessel deck marker points are obtained, and the projection transformation is completed.

7. The visually guided landing method for unmanned aerial vehicles (UAVs) to unmanned surface vessels according to claim 1, characterized in that, Following S5, it also includes: The entire process employs a Kalman filter to fuse inertial measurement unit data and visual data, wherein the visual data specifically includes the initial guidance pose and the final guidance pose; The Kalman filter is specifically used for: Establish the state vector of the Kalman filter; Based on the data from the inertial measurement unit, the state vector is predicted; Calculate the Kalman gain; Based on the Kalman gain, when the initial or final guided pose is a valid pose, the prediction result is updated to obtain the optimal state estimate.

8. The visually guided landing method for unmanned aerial vehicles (UAVs) to unmanned surface vessels according to claim 7, characterized in that, The calculation method for the optimal state estimate is as follows: The prediction results are then fed into the nonlinear observation model to obtain the predicted observation values. The optimal state estimate is calculated based on the effective pose, the observed predicted value, the Kalman gain, and the prediction result.

9. A vision-guided landing system for unmanned aerial vehicles (UAVs) to unmanned surface vessels (USVs), characterized in that, include: Processor and memory; The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the visually guided landing method for unmanned aerial vehicles to unmanned vessels as described in any one of claims 1 to 8.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the visually guided landing method for unmanned aerial vehicles to unmanned surface vessels as described in any one of claims 1 to 8.

Images