Unmanned aerial vehicle landing control method and system fusing visual beacon and UWB

Abstract

The application relates to the technical field of unmanned aerial vehicle landing control and multi-sensor fusion positioning, and discloses an unmanned aerial vehicle landing control method and system fusing a visual beacon and UWB, the method comprising the following steps: acquiring distance information of an airborne UWB tag and each ground UWB anchor point; processing a visual image, identifying a visual beacon, calculating a visual relative pose of the unmanned aerial vehicle relative to the visual beacon, and simultaneously calculating a visual confidence index; processing the distance information to generate a UWB validity flag; constructing an extended Kalman filter, dynamically adjusting a noise matrix, taking the visual relative pose and the distance information as observation, fusing and predicting a real-time pose state of the unmanned aerial vehicle, and outputting an unmanned aerial vehicle pose estimation state; and generating a landing control instruction based on the unmanned aerial vehicle pose estimation state. The application introduces an adaptive fusion mechanism, and completely changes the mode of traditional fixed-weight fusion or hard switching.

Description

Technical Field

[0001] This invention relates to the field of UAV landing control and multi-sensor fusion positioning technology, and more specifically, to a UAV landing control method and system that integrates visual beacons and UWB. Background Technology

[0002] Unmanned Aerial Vehicle (UAV) technology has shown broad application prospects in civilian and commercial fields such as logistics delivery, power line inspection, agricultural plant protection, aerial surveying, and emergency rescue. Currently, the dominant navigation method for UAVs is the Global Navigation Satellite System (GNSS), such as GPS. However, the limitations of GNSS become apparent during the final approach phase of autonomous landing. For example, satellite signals may be blocked or attenuated when approaching buildings, trees, or terrain, leading to loss of positioning information or a sharp decrease in accuracy. Signal reflections from surrounding objects can produce multipath effects, causing positioning errors of several meters or even greater, which can be fatal for safe landing. Furthermore, both unintentional electromagnetic interference and malicious deceptive interference can cause GNSS-based landing systems to fail.

[0003] Meanwhile, low-power wireless communication devices may exist near the landing platform, such as wireless sensor network nodes used for area security or IoT devices inside nearby buildings. The existing system's threshold for UWB non-line-of-sight (NLOS) identification is designed primarily to address significant ranging errors caused by typical occlusion or multipath effects. For occasional, low-amplitude ranging noise spikes caused by interference from external low-power wireless communication devices, these spikes may differ from typical NLOS measurement patterns. They may not cause a significant drop in signal strength or a significant time-of-arrival delay, and their amplitude is insufficient to trigger the existing NLOS identification threshold. Therefore, the system fails to mark them as invalid measurements, thus incorporating this slightly noisy UWB data into the subsequent fusion process. Therefore, designing an adaptive fusion algorithm that can accurately quantify and dynamically adjust the true uncertainties of visual and UWB measurements to eliminate persistent low-amplitude fluctuations in fusion pose estimation, thereby achieving centimeter-level smooth and accurate landing, is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a UAV landing control method and system that integrates visual beacons and UWB, aiming to solve the problem that in complex environments, visual and UWB fusion positioning results in continuous low-amplitude fluctuations in fused pose estimation due to local image quality degradation and occasional UWB noise spikes.

[0005] The technical solution of the present invention is as follows: In a first aspect, the present invention provides a landing control method for unmanned aerial vehicles (UAVs) that integrates visual beacons and UWB, comprising the following steps: After the UAV enters the landing procedure, visual images are acquired through the airborne image acquisition device, and the distance information between the airborne UWB tag and each ground UWB anchor point is obtained. The visual images are processed to identify the visual beacons deployed at the center of the preset landing platform and to calculate the visual relative pose of the UAV relative to the visual beacons. At the same time, a visual confidence index is calculated to evaluate the reliability of the visual images. The distance information is processed to generate a UWB validity flag used to evaluate the reliability of the distance information; An extended Kalman filter is constructed with the UAV pose state as the estimation target. The noise matrix of the extended Kalman filter is dynamically adjusted according to the visual confidence index and the UWB validity indicator. Visual relative pose and distance information are used as observations. The real-time pose state of the UAV is fused and predicted based on the adjusted noise matrix, and the UAV pose estimation state is output. Based on the UAV's pose estimation state, landing control commands are generated.

[0006] The above solution solves the problem in existing technologies where vision and UWB fusion positioning in complex environments causes continuous low-amplitude fluctuations in fused pose estimation due to local image quality degradation and occasional UWB noise spikes, achieving centimeter-level smooth and precise landing.

[0007] Furthermore, the present invention also proposes that the method further includes: Hardware triggering or timestamp alignment is used to ensure that visual images and distance information are synchronized in time.

[0008] The above scheme ensures precise temporal synchronization between visual images and distance information, thereby improving the accuracy of fusion positioning.

[0009] Furthermore, the present invention also proposes a step of processing visual images, identifying visual beacons deployed at the center of a preset landing platform, calculating the visual relative pose of the UAV relative to the visual beacons, and simultaneously calculating a visual confidence index for evaluating the reliability of the visual images, including: Visual beacons in visual images are identified using computer vision algorithms, and the pixel coordinates of multiple corner points of the visual beacons are extracted. Based on the intrinsic parameter matrix of the airborne image acquisition device, the known physical size of the visual beacon, and the pixel coordinates of multiple corner points, the six-degree-of-freedom pose of the UAV relative to the coordinate system centered on the visual beacon is calculated as the visual relative pose. The visual confidence index is obtained by weighted fusion calculation based on the pixel area of ​​the visual beacon in the image, the reprojection error of the pixel coordinates of multiple corner points, and the sharpness assessment value of the visual image.

[0010] Furthermore, the present invention also proposes a step of processing distance information to generate a UWB validity flag for evaluating the reliability of the distance information, comprising: The distance information of each ground UWB anchor point is processed by low-pass filtering or Kalman filtering. Based on the filtered distance information, machine learning or rule base methods are used to determine whether it is under non-line-of-sight conditions and generate UWB validity flags.

[0011] Furthermore, this invention also proposes a step for determining whether a condition is non-line-of-sight and generating a UWB validity flag using machine learning or rule-based methods, including: If the line-of-sight condition is determined, the generated UWB validity flag is valid; If the condition is determined to be outside of line-of-sight, the generated UWB validity flag will be invalid.

[0012] Furthermore, the present invention also proposes a step of dynamically adjusting the noise matrix of the extended Kalman filter based on the visual confidence index and the UWB validity indicator, comprising: The visual measurement noise matrix in the extended Kalman filter is dynamically adjusted based on the visual confidence index. The UWB measurement noise matrix in the extended Kalman filter is dynamically adjusted based on the UWB validity flag.

[0013] Furthermore, the present invention also proposes a step of dynamically adjusting the visual measurement noise matrix in the extended Kalman filter according to the visual confidence index, comprising: Based on the visual confidence index, the value of the visual measurement noise matrix is ​​determined according to the inverse correlation. The higher the visual confidence index, the smaller the value of the visual measurement noise matrix, and the greater the weight of the visual measurement noise matrix in the fusion prediction.

[0014] Furthermore, the present invention also proposes a step for dynamically adjusting the UWB measurement noise matrix based on the UWB validity flag, comprising: When the UWB validity flag is valid, a preset noise matrix value is assigned to the UWB measurement noise matrix; When the UWB validity flag is invalid, a preset maximum value is assigned to the UWB measurement noise matrix. The preset maximum value is greater than the preset noise matrix value, so as to reduce or ignore the weight of the UWB measurement noise matrix in the fusion prediction.

[0015] Furthermore, the present invention also proposes that the step of generating landing control commands based on the UAV pose estimation state includes: The estimated pose of the UAV is compared with the desired landing point location to generate landing control commands that include landing trajectory, throttle, pitch, roll, and yaw channel control.

[0016] Secondly, the present invention also proposes a UAV landing control system integrating visual beacons and UWB, for executing the aforementioned UAV landing control method integrating visual beacons and UWB, the system comprising: The data acquisition module is used to acquire visual images through the airborne image acquisition device after the UAV enters the landing procedure, and at the same time obtain the distance information between the airborne UWB tag and each ground UWB anchor point. The visual image processing module is used to process visual images, identify visual beacons deployed at the center of the preset landing platform, calculate the visual relative pose of the UAV relative to the visual beacons, and calculate the visual confidence index used to evaluate the reliability of the visual images. The distance information processing module is used to process distance information and generate UWB validity flags for evaluating the reliability of distance information; The fusion prediction module is used to construct an extended Kalman filter with the UAV pose state as the estimation target. It dynamically adjusts the noise matrix of the extended Kalman filter according to the visual confidence index and UWB validity indicator. It uses visual relative pose and distance information as observations and performs fusion prediction on the real-time pose state of the UAV based on the adjusted noise matrix, and outputs the UAV pose estimation state. The command generation module is used to generate landing control commands based on the UAV's pose estimation state.

[0017] The above scheme provides a system for implementing the above method, which has good engineering applicability.

[0018] In summary, this invention provides a method and system for unmanned aerial vehicle (UAV) landing control that integrates visual beacons and UWB. The method involves simultaneously acquiring visual images and UWB distance information after the UAV enters the landing procedure. The visual images are processed to calculate the visual relative pose and a visual confidence index. The UWB distance information is processed to generate a UWB validity flag. An extended Kalman filter is then constructed. The noise matrix of the filter is dynamically adjusted based on the visual confidence index and the UWB validity flag. The visual relative pose and distance information are fused and predicted as observations, outputting the estimated UAV pose state, and landing control commands are generated based on this. This scheme introduces a visual confidence index and a UWB validity indicator to achieve real-time assessment of the reliability of visual and UWB measurement data. Based on this, the noise matrix of the fusion filter is dynamically adjusted, thereby effectively reducing the impact of visual local image quality degradation and occasional UWB noise spikes on the fused pose estimation. It avoids the persistent low-amplitude fluctuations in the fused pose estimation, enabling UAVs to achieve smooth and precise landing at the centimeter level. It solves the problem in existing technologies where visual and UWB fusion positioning in complex environments causes continuous low-amplitude fluctuations in fused pose estimation due to local image quality degradation and occasional UWB noise spikes, achieving the advantage of smooth and precise landing at the centimeter level. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a method for unmanned aerial vehicle (UAV) landing control that integrates visual beacons and UWB, as provided in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of a UAV landing control system that integrates visual beacons and UWB, provided as an embodiment of the present invention.

[0021] Labeling Explanation: 210, Data Acquisition Module; 220, Visual Image Processing Module; 230, Distance Information Processing Module; 240, Fusion Prediction Module; 250, Instruction Generation Module. Detailed Implementation

[0022] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some, not all, of the embodiments of this invention. The components of this invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0023] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] In the fully automated operation process of unmanned aerial vehicles (UAVs), high-precision and high-reliability autonomous landing is the most technically challenging and crucial step. Especially in scenarios such as logistics delivery and power line inspection, the landing accuracy requirement reaches the centimeter level. Traditional global navigation satellite systems (GNSS) are susceptible to signal blockage, multipath effects, and electromagnetic interference during the final approach phase of landing, making it difficult to meet high-precision requirements. Therefore, using airborne sensors and ground-based auxiliary equipment for relative positioning has become a major research direction. Visual navigation provides rich information but is easily affected by environmental factors such as lighting and occlusion; ultra-wideband positioning offers high accuracy and strong penetration but may be affected by multipath effects.

[0025] Existing technologies often employ a switching or loosely coupled strategy between vision and ultra-wideband (UWB) navigation. This means that vision navigation is prioritized when available, and switched to UWB navigation when it fails. While simple to implement, this strategy is prone to abrupt changes in flight control commands during sensor switching, causing drone attitude jitter and affecting the smoothness and safety of the landing process. Furthermore, this either-or switching logic cannot fully utilize the complementary information when both sensors are simultaneously effective, nor can it provide a smooth response to gradual changes in sensor data quality, thus limiting the system's robustness and accuracy.

[0026] Firstly, please see Figure 1 The present invention provides a method for unmanned aerial vehicle (UAV) landing control that integrates visual beacons and ultra-wideband, comprising: S1. After controlling the UAV to enter the landing procedure, visual images are acquired through the airborne image acquisition device, and the distance information between the airborne UWB tag and each ground UWB anchor point is obtained at the same time. S2. Process the visual image, identify the visual beacon deployed at the center of the preset landing platform and calculate the visual relative pose of the UAV relative to the visual beacon, and calculate the visual confidence index used to evaluate the reliability of the visual image. S3. Process the distance information to generate a UWB validity flag for evaluating the reliability of the distance information; S4. Construct an extended Kalman filter with the UAV pose state as the estimation target. Dynamically adjust the noise matrix of the extended Kalman filter according to the visual confidence index and UWB validity flag. Use visual relative pose and distance information as observations. Based on the adjusted noise matrix, perform fusion prediction on the real-time pose state of the UAV and output the UAV pose estimation state. S5. Based on the UAV pose estimation state, generate landing control commands.

[0027] This invention proposes a deep fusion scheme, the core working principle of which is as follows: After the UAV enters the landing procedure, visual images are acquired through an onboard image acquisition device, and distance information between the onboard UWB tag and various ground UWB anchor points is obtained simultaneously. Next, the visual images are processed to identify the visual beacon deployed at the center of the pre-set landing platform and calculate the visual relative pose of the UAV relative to the visual beacon. Simultaneously, a visual confidence index is calculated to evaluate the reliability of the visual images. At the same time, the distance information is processed to generate an UWB validity flag to evaluate the reliability of the distance information. Then, an extended Kalman filter is constructed with the UAV pose state as the estimation target. The noise matrix of the extended Kalman filter is dynamically adjusted according to the visual confidence index and the UWB validity flag. Using the visual relative pose and distance information as observations, the real-time pose state of the UAV is fused and predicted based on the adjusted noise matrix, outputting the estimated UAV pose state. Finally, landing control commands are generated based on the estimated UAV pose state.

[0028] This method constructs an Extended Kalman Filter (EKF) as the core fusion framework, using UAV position, velocity, and attitude as state variables. The EKF is an efficient recursive filtering algorithm suitable for handling nonlinear systems, capable of optimally fusing measurement data from different sensors to estimate the system state. In this scheme, the EKF first predicts the state based on the UAV's motion model, i.e., predicts the UAV's pose at the current moment. Subsequently, it updates the state using measurement data from vision and ultra-wideband sensors, correcting the predicted values.

[0029] The key innovation of this scheme lies in the introduction of an adaptive fusion mechanism, which fundamentally changes the traditional fixed-weight fusion or hard-switching model. This mechanism no longer treats sensor data as equally reliable, but instead evaluates the quality of each sensor data source in real-time and quantitatively. Specifically, by calculating a visual confidence index and generating an ultra-wideband validity flag, sensor data quality is transformed from a vague concept into precise mathematical scalars. These scalars are then used as direct inputs to dynamically adjust the measurement noise matrix in the extended Kalman filter. The measurement noise matrix in the filter represents the degree of uncertainty of the measurement data, and its value directly determines the weight of that measurement data in the state update process. When the visual confidence index is high, it means that the visual pose calculation result is very reliable. In this case, the value of the visual measurement noise matrix is ​​correspondingly reduced, allowing the filter to trust the visual data more during fusion. Conversely, if the visual confidence index is low, the value of the visual measurement noise matrix is ​​increased, reducing its weight. Similarly, when the ultra-wideband validity flag is invalid, it means that the ultra-wideband ranging data has been severely interfered with. In this case, the corresponding measurement noise matrix is ​​set to a maximum value, making it equivalently ignored in this fusion update.

[0030] This shift from a switching-based to a continuously variable transmission (CVT) fusion system allows the entire system to smoothly and intelligently adjust the trust level of different data sources based on the real-time operating status of the sensors, thereby fundamentally avoiding control jitter caused by data source switching. Even in the event of a sensor performance degradation or temporary failure, the system can smoothly transition, relying more on another reliable sensor to ensure the continuity and stability of pose estimation. Ultimately, it outputs high-precision, high-smoothness UAV pose status, providing a solid foundation for achieving centimeter-level precision landing.

[0031] Furthermore, to ensure the accuracy and stability of the above fusion prediction, the method also includes: Hardware triggering or timestamp alignment is used to ensure that visual images and distance information are synchronized in time.

[0032] In high-frequency state estimation algorithms such as the Extended Kalman Filter (EKF), the temporal consistency of the input observations is crucial. If there is a significant discrepancy between the acquisition time of the visual image and the measurement time of the ultra-wideband distance information, then these two data points reflect the state of the UAV at different times. Directly fusing them will inevitably introduce errors and may even cause the filter to diverge. Therefore, measures must be taken to ensure precise temporal synchronization of the data.

[0033] In one specific embodiment, synchronization is achieved through hardware triggering. This method ensures the synchronization of data acquisition at a physical level. For example, the UAV's flight controller can be configured with a pulse width modulation output channel that periodically generates a rising edge signal. This signal is simultaneously connected to the external trigger input pin of the airborne image acquisition device and the synchronization input pin of the airborne ultra-wideband tag. When the rising edge signal arrives, the global shutter of the image acquisition device is triggered, starting the exposure to acquire one frame of image. At the same time, the ultra-wideband tag also begins ranging communication with all ground anchor points. Since the trigger signal is the same physical electrical signal, the start time of image exposure and the start time of ultra-wideband ranging are aligned on the nanosecond level, thus achieving highly accurate data synchronization.

[0034] In another specific embodiment, when hardware conditions do not allow for direct physical triggering, synchronization can be addressed at the software level using timestamp alignment. This approach requires the UAV's onboard computing platform to provide a high-precision, monotonically increasing system clock. When the onboard image acquisition device captures an image frame and transmits it to the computing platform, the image driver immediately reads the current system clock value and appends this high-precision timestamp to the image data packet. Similarly, when the onboard UWB tag completes a ranging operation and sends the distance information to the computing platform, the UWB driver also appends a timestamp of the current moment. Before performing the measurement update step, the extended Kalman filter compares the timestamps of the visual relative pose and the UWB distance information. Assuming the timestamp of the visual data is slightly earlier than that of the UWB data, the filter propagates the UAV's state from the visual data timestamp to the UWB data timestamp based on the UAV's motion model. This propagation process utilizes the UAV's velocity and angular velocity estimates from the previous moment to predict the UAV's pose change between the two timestamps. After state propagation, the visual observations and ultra-wideband observations are aligned to the same point in time. When the measurement is updated at this point, the error caused by time asynchrony is eliminated, ensuring the accuracy of the fusion.

[0035] Further, the steps of processing the visual images, identifying the visual beacons deployed at the center of the pre-set landing platform, calculating the visual relative pose of the UAV relative to the visual beacons, and calculating the visual confidence index used to evaluate the reliability of the visual images specifically include: Visual beacons in visual images are identified using computer vision algorithms, and the pixel coordinates of multiple corner points of the visual beacons are extracted. Based on the intrinsic parameter matrix of the airborne image acquisition device, the known physical size of the visual beacon, and the pixel coordinates of multiple corner points, the six-degree-of-freedom pose of the UAV relative to the coordinate system centered on the visual beacon is calculated as the visual relative pose. Based on the pixel area of ​​the visual beacon in the image, the reprojection error of the pixel coordinates of multiple corner points, and the sharpness evaluation value of the visual image, the visual confidence index is obtained through weighted fusion calculation.

[0036] This step details how to extract high-quality pose information from the original image and generate a reliable quantitative quality assessment. First, visual beacons are identified using computer vision algorithms. A visual beacon can be a marker with specific geometric patterns and coded information, such as an AprilTag or ArUco code. The identification process typically involves grayscale and binarization of the image, followed by contour detection and topological analysis to find quadrilateral regions that match the beacon's features. Once the beacon is identified, several key feature points can be extracted, typically the precise pixel coordinates of its four interior corners.

[0037] Next, based on the extracted corner pixel coordinates, the UAV's visual relative pose is calculated. This is a typical problem of recovering three-dimensional spatial pose from two-dimensional image information, usually solved using the Perspective-n-Point algorithm. The inputs to this algorithm include: the intrinsic parameter matrix of the onboard image acquisition device, which describes the camera's focal length, principal point coordinates, and other internal optical parameters, usually obtained through pre-calibration of the camera; the known physical dimensions of the visual beacon, such as the side length of the beacon square; and the pixel coordinates of multiple corner points detected in the image. By solving the perspective projection equations, the rotation matrix and translation vector of the UAV's camera coordinate system relative to the coordinate system centered on the visual beacon can be calculated. These two together constitute the UAV's six-degree-of-freedom pose, namely, three-dimensional position and three-dimensional attitude.

[0038] Most importantly, to provide a basis for subsequent adaptive fusion, a visual confidence index that comprehensively reflects the reliability of current visual measurements needs to be calculated. A single evaluation dimension often has limitations; therefore, this solution adopts a multi-factor weighted fusion approach. Specifically, it integrates information from the following three dimensions: The first dimension is the pixel area of ​​the visual beacon in the image. This metric intuitively reflects the distance between the UAV and the landing platform. When the UAV is closer, the beacon occupies a larger pixel area in the image, resulting in richer image details and generally higher accuracy and stability in corner extraction and pose calculation.

[0039] The second dimension is the reprojection error of multiple corner pixel coordinates. After calculating the six-DOF pose of the UAV using the PnP algorithm, the corner coordinates of the beacon in three-dimensional space can be reprojected back onto the two-dimensional image plane based on the calculated pose, resulting in a set of reprojected corner pixel coordinates. Comparing these coordinates with the original corner pixel coordinates detected in the image, the Euclidean distance between them is the reprojection error. This error directly measures the degree of agreement between the pose calculation result and the actual image observation; the smaller the error, the more accurate and reliable the pose calculation result.

[0040] The third dimension is the visual image sharpness assessment value. Image sharpness is affected by various factors such as motion blur, focus inaccuracy, and lighting conditions. A blurry image will lead to a decrease in the accuracy of corner detection, thus affecting the accuracy of pose calculation. Sharpness can be evaluated by calculating the variance of the Laplacian operator of the image. For an image, the larger the variance of its Laplacian operator, the richer the edge and detail information, and the sharper the image.

[0041] Finally, the three normalized indicators are weighted and fused to calculate the final visual confidence index. For example, a linear weighted summation can be used: Visual Confidence Index = Weight Coefficient 1 * Normalized Pixel Area + Weight Coefficient 2 * (1 - Normalized Reprojection Error) + Weight Coefficient 3 * Normalized Sharpness. Here, weight coefficients 1, 2, and 3 are empirical parameters optimized from a large amount of experimental data, reflecting the relative importance of each factor in assessing the reliability of visual data. For example, pixel area may be given a high weight because it is directly related to the drone's state during the terminal landing phase; reprojection error, as a direct measure of solution accuracy, also has a high weight; and sharpness serves as an auxiliary correction term. In this way, a more comprehensive and robust visual confidence index is obtained than a single indicator.

[0042] Further, the steps of processing the distance information to generate an ultra-wideband validity flag for evaluating the reliability of the distance information specifically include: The distance information of each ground ultra-wideband anchor point is processed by low-pass filtering or Kalman filtering. Based on the filtered distance information, machine learning or rule base methods are used to determine whether it is under non-line-of-sight conditions and generate an ultra-wideband validity flag.

[0043] This step aims to identify and flag unreliable measurements from the raw UWB ranging data caused by factors such as non-line-of-sight propagation. First, the raw distance information is filtered. Raw UWB ranging values ​​may contain Gaussian white noise or occasional outliers. Low-pass filtering can effectively remove high-frequency noise, making the distance data sequence smoother. Alternatively, a simple one-dimensional Kalman filter can be built for the distance measurement at each anchor point, utilizing the assumption of motion continuity to optimally estimate the measurement values, further improving the stability of the data.

[0044] After filtering, the core task is to determine whether the current measurement conditions are non-line-of-sight (NFS). NFS refers to a situation where the ultra-wideband (UWS) signal does not have a direct straight-line propagation path between the transmitter and receiver, but arrives through reflection, diffraction, etc. This leads to a measured distance value that is significantly greater than the actual distance, and is a major source of error in UWS positioning. Various methods can be used to determine NFS conditions.

[0045] In one specific embodiment, a rule-based approach is used for judgment. This method establishes a set of logical rules based on an understanding of the physical characteristics of non-line-of-sight signals. For example, channel impulse response data provided by an ultra-wideband chip can be used. Under line-of-sight conditions, the channel impulse response typically exhibits a sharp main peak with concentrated energy. However, under non-line-of-sight conditions, due to multipath effects, the main peak energy attenuates, and multiple delayed secondary peaks appear. Therefore, a rule can be set: if the received signal strength indication value is below a certain threshold, or if the ratio of the power of the first path signal in the channel impulse response to the total received power is less than a certain threshold, then it is judged as a non-line-of-sight condition.

[0046] In another, more complex embodiment, machine learning is employed for the determination. This method learns complex patterns in non-line-of-sight (NLS) signals in a data-driven manner. First, a large amount of ultra-wideband signal data with either NLS or NLS labels is collected as a training set. A series of features are extracted from each signal, such as received signal strength, first-path signal power, signal rise time, and root mean square delay spread. Then, these features are used to train a classifier, such as a support vector machine, decision tree, or small neural network. After training, these features are extracted in real time and input into the trained model during actual UAV operation. The model then outputs a classification result, indicating whether the current measurement is under NLS or NLS conditions.

[0047] Based on the result of any of the above methods, an UWB validity flag is generated. If the condition is determined to be line-of-sight, the generated UWB validity flag is valid; if the condition is determined to be non-line-of-sight, the generated UWB validity flag is invalid. This binary flag will serve as the direct input to the subsequent adaptive fusion algorithm, used to determine whether to accept the current UWB measurement data.

[0048] Furthermore, the steps for dynamically adjusting the noise matrix of the extended Kalman filter based on the visual confidence index and the ultra-wideband effectiveness indicator specifically include: The visual measurement noise matrix in the extended Kalman filter is dynamically adjusted based on the visual confidence index. The ultra-wideband measurement noise matrix in the extended Kalman filter is dynamically adjusted based on the ultra-wideband effectiveness flag.

[0049] This step is the core of adaptive fusion, transforming the aforementioned evaluation of sensor data quality into direct control of the behavior of the extended Kalman filter. In the extended Kalman filter, the measurement noise covariance matrix is ​​typically represented as an R matrix, whose diagonal elements represent the uncertainty or noise variance of each observation. The value of the R matrix directly affects the calculation of the Kalman gain, thus determining the degree of trust the filter has in the predicted and measured values ​​during state updates. This scheme achieves fine-grained control of the fusion weights by dynamically adjusting the R matrix portions corresponding to visual and ultra-wideband measurements respectively.

[0050] Specifically, the steps for dynamically adjusting the visual measurement noise matrix based on the visual confidence index include: Based on the visual confidence index, the value of the visual measurement noise matrix is ​​determined according to the inverse correlation. The higher the visual confidence index, the smaller the value of the visual measurement noise matrix, and the greater the weight of the visual measurement noise matrix in the fusion prediction.

[0051] This inverse correlation is intuitive and effective. When the visual confidence index is close to 1, it indicates that the visual pose calculation result is highly reliable with very low uncertainty, and therefore a small noise variance value should be assigned to it. Conversely, when the visual confidence index is close to 0, it indicates that the visual data quality is poor with high uncertainty, and a large noise variance value should be assigned to it. A specific implementation function can be: Visual Measurement Noise Variance = (1 - Visual Confidence Index) * Dynamic Range Coefficient + Minimum Noise Value. Here, the minimum noise value is used to prevent the noise variance from becoming zero, which would lead to filter instability, while the dynamic range coefficient is used to adjust the magnitude of the noise variance change with confidence. In this way, the weight of the visual measurement changes smoothly with its quality, achieving stepless adjustment.

[0052] Meanwhile, the steps for dynamically adjusting the ultra-wideband measurement noise matrix based on the ultra-wideband validity indicator include: When the UWB validity flag is valid, a preset noise matrix value is assigned to the UWB measurement noise matrix; When the UWB validity flag is invalid, a preset maximum value is assigned to the UWB measurement noise matrix. The preset maximum value is greater than the preset noise matrix value, so as to reduce or ignore the weight of the UWB measurement noise matrix in the fusion prediction.

[0053] This adjustment method is a judgment-based on-off adjustment. When the UWB validity flag is valid, it means the UWB ranging data is reliable. In this case, a preset noise variance value, obtained from the sensor manual or experimental calibration, is assigned, for example, 0.01 square meters, representing a standard deviation of 0.1 meters. When the UWB validity flag is invalid, it means the data is severely contaminated by non-line-of-sight factors and is completely unreliable. In this case, a preset maximum value is assigned, for example, 10,000 square meters. In the Kalman filter update formula, a very large measurement noise value will cause the Kalman gain to approach zero, making the contribution of this measurement to the state estimation update negligible, equivalent to ignoring this unreliable UWB data at the current moment.

[0054] By independently and dynamically adjusting the visual and ultra-wideband measurement noise matrices as described above, the extended Kalman filter can intelligently and adaptively fuse data from both sensors, prioritizing the use of the higher-quality data source at all times, thereby outputting a continuously stable and highly accurate pose estimate.

[0055] Furthermore, after outputting a high-precision, smooth UAV pose estimation state, the specific steps for generating landing control commands based on the UAV pose estimation state include: The estimated pose of the UAV is compared with the desired landing point location to generate landing control commands that include landing trajectory, throttle, pitch, roll, and yaw channel control.

[0056] This step translates the precise pose estimation into the actual flight maneuvers of the UAV. First, the real-time pose estimation state of the UAV, output by the fusion algorithm, including its 3D spatial position and attitude, is compared with a preset desired landing point pose. The desired landing point is typically located at a certain height directly above the center of the visual beacon. This comparison yields an error vector, which describes the deviation between the UAV's current position and attitude and the target position and attitude.

[0057] Then, a landing controller, such as a cascaded proportional-integral-derivative (PI-DE) controller, generates specific control commands based on this error vector. The position controller calculates the desired pitch and roll angles based on the horizontal position error to drive the UAV horizontally toward the target point. The altitude controller calculates the desired throttle command based on the altitude error to control the UAV's vertical descent speed. The attitude controller calculates specific pitch, roll, and yaw channel control values ​​based on the attitude error to stabilize the UAV's attitude and orient it correctly. These control values ​​are ultimately sent to the UAV's underlying flight controller, driving the motors to produce corresponding speed changes, thereby guiding the UAV along a smooth trajectory and landing precisely and smoothly on the pre-designed landing platform.

[0058] In a preferred embodiment, to address more complex and dynamically changing real-world environments, this invention also proposes a context-aware multimodal uncertainty quantification framework. This framework aims to solve some subtle but accuracy-affecting problems that may arise during long-term operation.

[0059] In a specific application scenario, the ground visual beacons of a drone landing system deployed in an outdoor logistics park are exposed for extended periods, and their surfaces become uneven due to dust, pollen, and other deposits. Under varying natural light at different times and angles throughout the day, this uneven surface can cause localized, slight glare or decreased contrast in the beacon image. In this situation, relying solely on global image sharpness, such as Laplacian variance, to assess visual confidence may result in an inflated confidence score because most areas of the image remain sharp, failing to accurately reflect the actual difficulty of beacon corner point extraction and leading to misjudgments of the visual data's reliability. Simultaneously, low-power wireless devices near the landing platform may intermittently interfere with the ultra-wideband signal, manifesting as small noise spikes in the ranging data. Existing non-line-of-sight recognition thresholds primarily target significant errors caused by physical occlusion and may not be effective in filtering out such weak interference. The combination of these two factors leads to the extended Kalman filter slightly over-trusting visual data with local distortion on the one hand, and intermittently incorporating ultrawideband data with slight noise on the other. Ultimately, this results in continuous, low-amplitude, minute fluctuations in the fused pose estimation, hindering smooth and accurate landing at the centimeter level.

[0060] To address this issue, a context-aware framework was introduced. The core of this framework lies in proactively sensing and integrating contextual information from the external environment, rather than relying solely on sensor data, to more accurately quantify measurement uncertainties.

[0061] First, during the information collection phase, in addition to the airborne camera and ultra-wideband tag, the drone is also equipped with an ambient light sensor to measure the illuminance of the current environment and the uniformity of light distribution in real time.

[0062] Secondly, in the visual uncertainty quantification stage, instead of simply calculating global sharpness, a localized analysis is performed on the identified beacon regions. For example, a small image patch is extracted centered on each detected corner point, and the local gradient variance and local contrast are calculated within this patch. If the local metrics of a corner point region are significantly lower than the global average, the visual quality of that region is considered degraded. Simultaneously, by incorporating readings from the ambient light sensor, if the current light intensity is too high or the light distribution is extremely uneven, the initial confidence level of the visual data is correspondingly lowered. By integrating a prior model of beacon surface contamination, real-time lighting information, and local image features, a more accurate estimate of visual uncertainty is generated.

[0063] Furthermore, in the ultra-wideband uncertainty quantification stage, time-series analysis based on ranging residuals is introduced. By maintaining a simplified motion model of the UAV, the theoretical ranging value at each moment can be predicted. The ranging residual is obtained by subtracting the actual measured value from the predicted value. By analyzing the residual sequence through a sliding window, if a residual spike with an extremely short duration but abnormal amplitude is detected, and the received signal strength does not decrease significantly at this time, it can be determined that the measurement was subjected to weak external interference. By statistically analyzing the frequency of such interference spikes and combining them with health indicators such as the long-term average signal strength, a more refined ultra-wideband uncertainty estimate is generated.

[0064] Finally, these more accurate visual and ultrawideband uncertainty estimates, corrected for contextual factors, are used to dynamically adjust the measurement noise matrix in the extended Kalman filter. This approach enables the filter to more intelligently understand the true uncertainty behind the data, thereby effectively suppressing minor fluctuations in pose estimation caused by subtle environmental changes. This ensures that the UAV can achieve smooth, reliable, and centimeter-level precision landings even in complex and variable environments.

[0065] Secondly, see Figure 2 The present invention also provides a UAV landing control system that integrates visual beacons and ultra-wideband technology, the system being used to execute any of the foregoing methods. Specifically, the system includes: The data acquisition module 210 is used to acquire visual images through the airborne image acquisition device after the UAV enters the landing procedure, and at the same time obtain the distance information between the airborne UWB tag and each ground UWB anchor point. The visual image processing module 220 is used to process visual images, identify visual beacons deployed at the center of the preset landing platform and calculate the visual relative pose of the UAV relative to the visual beacons, and at the same time calculate the visual confidence index used to evaluate the reliability of the visual images. The distance information processing module 230 is used to process the distance information and generate a UWB validity flag for evaluating the reliability of the distance information; The fusion prediction module 240 is used to construct an extended Kalman filter with the UAV pose state as the estimation target. It dynamically adjusts the noise matrix of the extended Kalman filter according to the visual confidence index and UWB validity indicator. It uses visual relative pose and distance information as observations and performs fusion prediction on the real-time pose state of the UAV based on the adjusted noise matrix to output the UAV pose estimation state. The instruction generation module 250 is used to generate landing control instructions based on the UAV pose estimation state.

[0066] The UAV landing control system fusion of visual beacons and ultra-wideband (UWB) provided by this invention significantly improves the accuracy, reliability, and environmental adaptability of the landing process through innovative multi-sensor fusion and dynamic adaptive mechanisms. This system achieves complementary advantages by fusing visual beacon and UWB technologies. Visual guidance technology can provide high-precision relative pose information, but it is susceptible to environmental influences such as lighting and occlusion. UWB technology can provide absolute distance information with strong signal penetration, but its vertical positioning accuracy may be insufficient. This system uses an extended Kalman filter to fuse visual relative pose and UWB distance information as observations for prediction, effectively overcoming the limitations of a single sensor and outputting a more stable and accurate real-time UAV pose state estimate. This fusion scheme helps achieve centimeter-level accuracy landing. The system also introduces a visual confidence index and a UWB validity flag to evaluate the reliability of sensor data and dynamically adjust the noise matrix of the extended Kalman filter accordingly. When the visual image quality is high (e.g., good lighting and clear beacons), the system relies more on visual data to obtain high-precision pose. When visual images are disturbed (such as sudden changes in light or brief occlusion) leading to a decrease in confidence, or when the UWB signal is abnormal, the system can automatically reduce the weight of unreliable data in the filtering process and increase the reliance on UWB or other valid data.

[0067] This dynamic adaptive mechanism enhances the system's robustness in complex landing environments (such as changes in lighting, slight obstructions, and radio interference), preventing landing failures due to temporary malfunctions of a single sensor. The high-precision pose estimation output by the fusion prediction module 240 provides a solid foundation for the decision-making of the command generation module 250. Based on this, the system can generate smoother and more precise landing control commands, guiding the UAV to descend steadily along the optimal trajectory. This intelligent decision-making and control capability reduces reliance on human intervention and improves the success rate and safety of autonomous landing of UAVs on complex terrain or mobile platforms. These modules work together to fully realize the adaptive fusion control method, thus providing UAVs with a high-precision and highly robust autonomous landing solution. The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for unmanned aerial vehicle (UAV) landing control integrating visual beacons and UWB, characterized in that, include: After the UAV enters the landing procedure, visual images are acquired through the airborne image acquisition device, and the distance information between the airborne UWB tag and each ground UWB anchor point is obtained. The visual image is processed to identify the visual beacon deployed at the center of the preset landing platform and calculate the visual relative pose of the UAV relative to the visual beacon. At the same time, a visual confidence index is calculated to evaluate the reliability of the visual image. The distance information is processed to generate a UWB validity flag for evaluating the reliability of the distance information; An extended Kalman filter is constructed with the UAV pose state as the estimation target. The noise matrix of the extended Kalman filter is dynamically adjusted according to the visual confidence index and the UWB validity flag. The visual relative pose and the distance information are used as observations. The real-time pose state of the UAV is fused and predicted based on the adjusted noise matrix, and the UAV pose estimation state is output. Based on the estimated pose of the UAV, landing control commands are generated.

2. The UAV landing control method integrating visual beacons and UWB according to claim 1, characterized in that, The method further includes: Hardware triggering or timestamp alignment is used to ensure that the visual image and the distance information are synchronized in time.

3. The UAV landing control method integrating visual beacons and UWB according to claim 1, characterized in that, The steps of processing the visual image, identifying the visual beacon deployed at the center of the preset landing platform and calculating the visual relative pose of the UAV relative to the visual beacon, and simultaneously calculating the visual confidence index used to evaluate the reliability of the visual image include: The computer vision algorithm is used to identify visual beacons in the visual image and extract the pixel coordinates of multiple corner points of the visual beacons. Based on the intrinsic parameter matrix of the airborne image acquisition device, the known physical size of the visual beacon, and the pixel coordinates of the multiple corner points, the six-degree-of-freedom pose of the UAV relative to the coordinate system centered on the visual beacon is calculated, which is used as the visual relative pose. Based on the pixel area of ​​the visual beacon in the image, the reprojection error of the pixel coordinates of the multiple corner points, and the sharpness assessment value of the visual image, a visual confidence index is obtained through weighted fusion calculation.

4. The UAV landing control method integrating visual beacons and UWB according to claim 1, characterized in that, The step of processing the distance information to generate a UWB validity flag for evaluating the reliability of the distance information includes: The distance information of each ground UWB anchor point is processed by low-pass filtering or Kalman filtering. Based on the filtered distance information, machine learning or rule base methods are used to determine whether it is under non-line-of-sight conditions, and the UWB validity flag is generated.

5. The UAV landing control method integrating visual beacons and UWB according to claim 4, characterized in that, The step of using machine learning or rule base methods to determine whether the condition is non-line-of-sight and generating the UWB validity flag includes: If it is determined that the line-of-sight condition is met, then the generated UWB validity flag is valid; If it is determined that the condition is not at line of sight, the generated UWB validity flag is invalid.

6. The UAV landing control method integrating visual beacons and UWB according to claim 1, characterized in that, The step of dynamically adjusting the noise matrix of the extended Kalman filter based on the visual confidence index and the UWB validity flag includes: The visual measurement noise matrix in the extended Kalman filter is dynamically adjusted according to the visual confidence index. The UWB measurement noise matrix in the extended Kalman filter is dynamically adjusted based on the UWB validity flag.

7. The UAV landing control method integrating visual beacons and UWB according to claim 6, characterized in that, The step of dynamically adjusting the visual measurement noise matrix in the extended Kalman filter according to the visual confidence index includes: Based on the visual confidence index, the value of the visual measurement noise matrix is ​​determined according to the inverse correlation. The higher the visual confidence index, the smaller the value of the visual measurement noise matrix, and the greater the weight of the visual measurement noise matrix in the fusion prediction.

8. The UAV landing control method integrating visual beacons and UWB according to claim 6, characterized in that, The step of dynamically adjusting the UWB measurement noise matrix according to the UWB validity flag includes: When the UWB validity flag is valid, a preset noise matrix value is assigned to the UWB measurement noise matrix; When the UWB validity flag is invalid, a preset maximum value is assigned to the UWB measurement noise matrix. The preset maximum value is greater than the preset noise matrix value, so as to reduce or ignore the weight of the UWB measurement noise matrix in the fusion prediction.

9. The UAV landing control method integrating visual beacons and UWB according to claim 1, characterized in that, The step of generating landing control commands based on the UAV pose estimation state includes: The estimated pose state of the UAV is compared with the desired landing point position to generate the landing control command, which includes landing trajectory, throttle, pitch, roll and yaw channel control.

10. A UAV landing control system integrating visual beacons and UWB, used to execute a UAV landing control method integrating visual beacons and UWB as described in any one of claims 1 to 9, characterized in that, The system includes: The data acquisition module is used to acquire visual images through the airborne image acquisition device after the UAV enters the landing procedure, and at the same time obtain the distance information between the airborne UWB tag and each ground UWB anchor point. The visual image processing module is used to process the visual image, identify the visual beacon deployed at the center of the preset landing platform and calculate the visual relative pose of the UAV relative to the visual beacon, and at the same time calculate the visual confidence index used to evaluate the reliability of the visual image. The distance information processing module is used to process the distance information and generate a UWB validity flag for evaluating the reliability of the distance information; The fusion prediction module is used to construct an extended Kalman filter with the UAV pose state as the estimation target, dynamically adjust the noise matrix of the extended Kalman filter according to the visual confidence index and the UWB validity flag, use the visual relative pose and the distance information as observations, and perform fusion prediction on the real-time pose state of the UAV based on the adjusted noise matrix to output the UAV pose estimation state; the command generation module is used to generate landing control commands based on the UAV pose estimation state.

Images