Unmanned aerial vehicle accurate take-off and landing control system based on RTK and vision fusion

Abstract

The invention discloses an unmanned aerial vehicle accurate take-off and landing control system based on RTK and vision fusion, and relates to the technical field of unmanned aerial vehicle navigation control, the system comprises an RTK positioning module, a vision positioning module, an environment perception module, a weight decision maker, a data fusion module and a degradation strategy management module, the RTK positioning module and the vision positioning module respectively obtain unmanned aerial vehicle position data; the environment sensing module evaluates RTK and visual environment factors in real time; the weight decision-making device outputs credibility coefficients of the two positioning modes based on the fuzzy logic rule set; the data fusion module adjusts a Kalman filtering measurement noise covariance matrix according to the credibility coefficient to realize data dynamic fusion; according to the invention, through a dynamic weight distribution and degradation guarantee mechanism, the positioning precision, robustness and safety of the unmanned aerial vehicle in taking-off and landing in a complex environment are improved, and the method is suitable for accurate taking-off and landing scenes of the unmanned aerial vehicle in the fields of logistics, surveying and mapping, emergency rescue and the like.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) navigation and control technology, specifically to a UAV precision take-off and landing control system based on RTK and vision fusion. Background Technology

[0002] With the widespread application of drones in logistics, surveying, agriculture, and emergency rescue, the requirements for precise take-off and landing control are increasing. Especially in complex environments such as take-off and landing within drone nests, drones need to achieve millimeter-level positioning accuracy to ensure operational safety and reliability. While Global Navigation Satellite Systems (GNSS) and their enhanced real-time dynamic differential (RTK) technology can provide centimeter- to millimeter-level global positioning, RTK signals are susceptible to multipath effects and obstruction interference in complex environments such as areas with tall buildings, near high-voltage power towers, or in canyons, leading to a significant decrease in positioning accuracy or even loss of lock. Although visual sensors can provide relative positioning information by recognizing ground markings, their performance is easily affected by factors such as changes in lighting, motion blur, and texture loss. Relying solely on either positioning method cannot stably maintain high accuracy in complex take-off and landing scenarios. Therefore, integrating the advantages of RTK and visual sensors has become a key technological direction for improving drone take-off and landing accuracy.

[0003] In existing technologies, a multi-rotor UAV autonomous tracking and landing control system and method (patent publication number CN115237158B) proposes to achieve autonomous tracking and landing of the UAV by recognizing two layers of QR code landmarks using a monocular camera. This method relies solely on visual information. While it solves the landing problem without human intervention to some extent, it does not integrate RTK technology. When the visual sensor is affected by lighting, occlusion, or damaged markers, the system lacks global positioning compensation, making it difficult to maintain high accuracy in GNSS-denied environments. Another patent publication number (CN106708066B) proposes a vision / inertial navigation-based autonomous landing method for UAVs. This method solves the UAV pose using a visual navigation algorithm, uses inertial navigation to help eliminate mismatched point pairs, and then achieves combined navigation through unscented Kalman filtering. Although this method introduces multi-sensor fusion, it does not involve deep fusion of RTK and vision, and its weight allocation method is fixed, making it unable to dynamically adjust sensor reliability according to the environment. This leads to increased error accumulation when the RTK signal is interfered with or visual performance deteriorates. While the visually enhanced RTK2 positioning sensor proposed by FPNavigation Technology achieves tight coupling of GNSS, vision and IMU, its fusion engine focuses on data integration at the hardware level and lacks an intelligent weight decision-making mechanism for dynamic environmental changes during take-off and landing, making it difficult to guarantee millimeter-level accuracy in high-interference scenarios near the aircraft hull.

[0004] The aforementioned existing technologies have not solved the core problem of UAVs in the final stage of takeoff and landing: how to enable the system to intelligently determine when to prioritize visual information and when to rely on RTK data to achieve millimeter-level stable landing. Specifically, in the specific scenario of the UAV pod, RTK signals are affected by local multipath effects, resulting in drift errors, while visual sensors may fail due to sudden changes in illumination, marker occlusion, or motion blur. Fixed-weight fusion methods cannot adapt to dynamic environmental changes, leading to divergent localization solutions or excessive landing point deviations. Therefore, there is an urgent need for a control system that can evaluate the environmental reliability of RTK and visual sensors in real time and dynamically adjust the fusion weights accordingly to simulate the decision-making process of experienced pilots and maintain accurate and robust localization capabilities in complex takeoff and landing environments.

[0005] In summary, existing UAV takeoff and landing control systems mostly employ static or post-processing strategies for sensor fusion, lacking a feedforward weight allocation mechanism based on environmental perception. Especially in highly disruptive nest environments, the system cannot autonomously avoid the risk of localized sensor failures, making it difficult to meet the accuracy and reliability requirements for millimeter-level landing. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a precise take-off and landing control system for UAVs based on RTK and vision fusion. By introducing intelligent environmental perception and dynamic reliability assessment, the weights of RTK and vision fusion are optimized in real time, thereby ensuring precise control of the UAV in the final stage of take-off and landing.

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: a precision take-off and landing control system for unmanned aerial vehicles (UAVs) based on RTK and vision fusion, comprising an RTK positioning module, a vision positioning module, an environmental perception module, a weighted decision-maker, and a data fusion module. The RTK positioning module is used to acquire the RTK location data of the UAV; The visual positioning module is used to acquire the visual position data of the UAV; The environment perception module includes an RTK environment perception submodule and a visual environment perception submodule. The RTK environment perception submodule is used to evaluate RTK environmental factors in real time, and the visual environment perception submodule is used to evaluate visual environmental factors in real time. The weighted decision-maker is connected to the environment perception module and is used to output the RTK confidence coefficient and the visual confidence coefficient through a fuzzy logic rule set based on the RTK environment factors and the visual environment factors. The data fusion module connects the RTK positioning module, the visual positioning module, and the weight decision-maker. It is used to adjust the weights of the RTK position data and the visual position data according to the RTK confidence coefficient and the visual confidence coefficient, and to perform data fusion through the Kalman filter algorithm to output the accurate position data of the UAV.

[0008] Furthermore, the RTK environmental factors include sky view factor, signal quality factor, and multipath risk factor; The visual environment factors include illumination robustness factor, image sharpness factor, and feature abundance factor; The sky view factor is calculated based on the distribution geometry and number of visible satellites; the signal quality factor is calculated based on the average and variance of the signal-to-noise ratio of visible satellites; and the multipath risk factor is calculated based on the relative relationship between a pre-stored simplified map of obstacles around the drone nest and the real-time position of the drone. The illumination robustness factor is calculated based on the image brightness distribution and contrast, the image sharpness factor is calculated based on the Laplacian variance, and the feature abundance factor is calculated based on the number of corner points or feature points in the image. The weighted decision-maker updates the RTK credibility coefficient and the visual credibility coefficient with a period of 50mm. The data fusion module uses the RTK confidence coefficient and the visual confidence coefficient as the basis for adjusting the measurement noise covariance matrix in the Kalman filtering algorithm. The RTK confidence coefficient is used to adjust the measurement noise covariance of the RTK position data, and the visual confidence coefficient is used to adjust the measurement noise covariance of the visual position data.

[0009] Furthermore, the specific methods used by the RTK environment perception submodule to evaluate the sky view factor, signal quality factor, and multipath risk factor include: The sky view factor is obtained by calculating the geometric distribution accuracy factor of visible satellites and the number of visible satellites. The geometric distribution accuracy factor is the accuracy attenuation factor caused by the geometric distribution of satellites in space, and the number of visible satellites is the total number of satellites that can be received at the current moment. The signal quality factor is obtained by calculating the average signal-to-noise ratio (SNR) and the SNR variance of all visible satellites. The average SNR is the arithmetic mean of the SNR of each satellite, and the SNR variance is the sum of squared deviations of the SNR of each satellite from the average. The multipath risk factor is obtained by querying a pre-stored simplified map of obstacles around the nest and calculating the closest distance and relative height difference between the UAV's real-time position and the obstacles. The simplified map of obstacles is a three-dimensional obstacle model around the nest obtained through pre-surveyed mapping. The closest distance is the minimum Euclidean distance between the UAV and the obstacle surface in the horizontal direction, and the relative height difference is the height difference between the UAV and the top of the obstacle in the vertical direction. The RTK environment perception submodule transmits the sky view factor, signal quality factor, and multipath risk factor as inputs to the weight decision-maker.

[0010] Furthermore, the specific methods for the visual environment perception submodule to evaluate the illumination robustness factor, image sharpness factor, and feature abundance factor include: The illumination robustness factor is obtained by calculating the overall brightness mean and brightness standard deviation of the image. The overall brightness mean is the average value of the gray values ​​of all pixels in the image, and the brightness standard deviation is the square root of the sum of squares of the deviations of the gray values ​​of all pixels from the mean. The image sharpness factor is obtained by calculating the Laplacian variance of the image, which is the variance value of the image obtained after performing Laplacian operator convolution on the image. The feature abundance factor is obtained by calculating the total number and distribution density of corner points or feature points in the image. The corner points or feature points are extracted by the Harris corner detection algorithm, and the distribution density is the ratio of the number of corner points or feature points to the total number of pixels in the image. The visual environment perception submodule transmits the illumination robustness factor, image clarity factor, and feature abundance factor as inputs to the weight decision-maker.

[0011] Furthermore, the weighted decision maker uses a fuzzy logic rule set to process the RTK environmental factors and visual environmental factors to output the RTK confidence coefficient and the visual confidence coefficient. The fuzzy logic rule set includes multiple fuzzy rules. Each fuzzy rule contains a premise part and a conclusion part. The premise part is a combination of fuzzy linguistic variables of RTK environmental factors and visual environmental factors. The conclusion part is a fuzzy linguistic variable of RTK credibility coefficient and visual credibility coefficient. The fuzzy linguistic variables include three levels: poor, medium, and excellent. The weighted decision-maker fuzzifies the input environmental factors through a fuzzy inference engine and matches them with fuzzy rules. Then, through a defuzzification operation, it converts the fuzzy linguistic variables in the conclusion part into precise RTK credibility coefficients and visual credibility coefficients. The fuzzy inference engine adopts the Mamdani fuzzy model, and the defuzzification operation uses the centroid method. The RTK credibility coefficient and visual credibility coefficient output by the weighted decision maker are real numbers between 0 and 1, where 0 represents completely untrustworthy and 1 represents completely trustworthy.

[0012] Furthermore, the data fusion module uses the Kalman filter algorithm for data fusion and uses the RTK confidence coefficient and visual confidence coefficient as the basis for adjusting the measurement noise covariance matrix; The data fusion module adjusts the measurement noise covariance matrix of the RTK location data based on the RTK confidence coefficient, and adjusts the measurement noise covariance matrix of the visual location data based on the visual confidence coefficient. The adjustment formula is as follows: in The adjusted RTK measurement noise covariance matrix, This is the adjusted visual measurement noise covariance matrix. Initialize the noise covariance matrix for RTK measurements. The initial values ​​are for the visual measurement noise covariance matrix. This represents the RTK credibility coefficient. Visual credibility coefficient; The data fusion module uses the adjusted measurement noise covariance matrix to perform Kalman filtering prediction and update steps, fusing RTK position data and visual position data to output the UAV's accurate position data.

[0013] Furthermore, the weighted decision-maker also includes an environmental interference index calculation unit, used to generate a unified environmental interference index by integrating RTK environmental factors and visual environmental factors. The environmental interference index is calculated using the following formula: in The environmental disturbance index. The normalized comprehensive score for RTK environmental factors. The normalized comprehensive score for visual environmental factors, and The weighting coefficients are and satisfy the following conditions: ; The normalized comprehensive score of the RTK environmental factors The normalized comprehensive score of the visual environment factors is calculated by weighting and summing the sky view factor, signal quality factor, and multipath risk factor. The calculation is performed by weighting and summing the illumination robustness factor, image sharpness factor, and feature abundance factor. The environmental interference index is used to assist the weight decision-maker in adjusting the rule triggering threshold in the fuzzy logic rule set. When the environmental interference index is higher than the preset threshold, the weight decision-maker will prioritize the use of rules that reduce the RTK credibility coefficient.

[0014] Furthermore, the system also includes a degradation strategy management module, which connects the weight decision-maker and the data fusion module, and is used to activate the degradation strategy when both the RTK confidence coefficient and the visual confidence coefficient are lower than a preset threshold. The degradation strategies include pure visual navigation mode, pure RTK navigation mode, and inertial navigation descent mode. The degradation strategy management module selects the optimal degradation strategy based on the specific values ​​of RTK environmental factors and visual environmental factors. When the overall visual environmental factors are better than the RTK environmental factors, the pure visual navigation mode is activated. When the overall RTK environmental factors are better than the visual environmental factors, the pure RTK navigation mode is activated. When both are worse, the inertial navigation descent mode is activated. After the degradation strategy is activated, the degradation strategy management module continuously monitors changes in environmental factors and automatically switches back to normal fusion mode when the environmental factors return to the normal range.

[0015] Furthermore, the system is implemented through embedded hardware, which includes a flight control computer and sensor interfaces; The flight control computer runs the algorithms of the environmental perception module, weight decision-maker, and data fusion module, and the sensor interface connects to the RTK receiver and the vision sensor. The flight control computer adopts a multi-threaded architecture, in which the environmental perception module and weight decision-maker run in a high-priority thread, and the data fusion module runs in a medium-priority thread to ensure real-time performance. The system pre-stores a simplified diagram of obstacles around the drone nest and a set of fuzzy logic rules. The simplified diagram of obstacles is generated from the mapping data during the first landing of the drone, and the set of fuzzy logic rules is obtained by optimizing the machine learning algorithm during the training phase. The system performs calculations on all modules at a frequency of 20Hz or higher during the take-off and landing of the UAV.

[0016] Compared with existing technologies, this RTK and vision-based precision take-off and landing control system for drones has the following advantages: I. This invention sets up an environmental perception module to evaluate RTK environmental factors and visual environmental factors in real time. The weight decision-maker processes the above environmental factors based on a fuzzy logic rule set and dynamically outputs the RTK confidence coefficient and the visual confidence coefficient. The data fusion module adjusts the measurement noise covariance matrix in the Kalman filter algorithm based on the confidence coefficient, thereby realizing the dynamic weight fusion of RTK position data and visual position data. This enables the positioning system to adapt to the dynamic changes in positioning conditions in complex take-off and landing environments in real time, effectively ensuring the accuracy and reliability of positioning data during the take-off and landing phase of the UAV.

[0017] Second, by adding a degradation strategy management module, this invention adaptively selects the optimal degradation navigation mode based on the specific evaluation results of the two types of environmental factors when both the RTK confidence coefficient and the visual confidence coefficient are below a preset threshold. At the same time, the system adopts an embedded hardware multi-threaded architecture and high-frequency computing design to ensure the real-time response of core modules such as environmental perception, weight decision-making and data fusion. Thus, it can still ensure the safety and continuity of the UAV take-off and landing process in extremely complex environments, further improving the robustness and engineering practicality of the system.

[0018] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0020] Figure 1 This is a schematic diagram of the system module interaction of the present invention; Figure 2 This is a schematic diagram of the reliability adaptive fusion logic of the present invention; Figure 3 This is a flowchart illustrating the operation of the present invention. Detailed Implementation

[0021] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0022] Example 1 like Figure 1 and Figure 2As shown, this embodiment discloses a specific implementation of a UAV precision take-off and landing control system based on RTK and vision fusion. By building an embedded hardware platform, an RTK positioning module, a vision positioning module, an environmental perception module, a weighted decision-maker, a data fusion module, and a degradation strategy management module are deployed to achieve coordinated operation of each module. The system uses the environmental perception module to evaluate the environmental adaptability of RTK and vision positioning in real time. The weighted decision-maker dynamically outputs a confidence coefficient based on a fuzzy logic rule set. The data fusion module uses this coefficient to adjust the measurement noise covariance matrix of the Kalman filter, achieving precise fusion of the two types of positioning data. Simultaneously, a degradation strategy is employed to ensure take-off and landing reliability in complex environments.

[0023] The UAV precision take-off and landing control system based on RTK and vision fusion in this embodiment is implemented through embedded hardware. The core hardware includes a flight control computer and sensor interfaces. The deployment and functional implementation of each hardware component are as follows: As the core computing unit of the system, the flight control computer must possess multi-threaded concurrent processing capabilities and high-efficiency data processing performance to support the real-time operation of core algorithms such as environmental perception, weighted decision-making, and data fusion. The flight control computer adopts a multi-threaded architecture, allocating the computational tasks of the environmental perception module and the weighted decision-maker to high-priority threads to ensure real-time response in environmental factor assessment and confidence coefficient updates; allocating the computational tasks of the data fusion module to medium-priority threads to ensure the continuity and timeliness of positioning data fusion; and allocating other auxiliary function tasks to low-priority threads to avoid resource contention affecting core functions.

[0024] The sensor interface is used to establish communication connections between the flight control computer and external sensors, specifically connecting the RTK receiver and the vision sensor. The sensor interface must support mainstream communication protocols to ensure that the raw data acquired by the RTK receiver and vision sensor can be stably and efficiently transmitted to the flight control computer, providing a foundation for subsequent data processing.

[0025] The system pre-stores two types of key data: a simplified obstacle map of the nest surrounding the drone and a fuzzy logic rule set. The simplified obstacle map is generated during the mapping process before the drone's first takeoff and landing mission. The drone's onboard mapping sensors scan the area around the nest to acquire 3D spatial information and construct an obstacle model. This model is used by the RTK environmental perception submodule to assess multipath risk factors. The fuzzy logic rule set is optimized during the training phase using machine learning algorithms. Based on extensive takeoff and landing experimental data from various environmental scenarios, the preconditions and conclusions of the rules are iteratively adjusted to ensure the rule set can adapt to various complex environments.

[0026] The system performs calculations on all modules at a frequency of 20Hz or higher during the take-off and landing of the drone. This frequency setting is based on the dynamic response requirements of the drone during take-off and landing, and can capture environmental changes and drone position dynamics in real time, avoiding a decrease in positioning accuracy due to calculation delays.

[0027] The RTK positioning module is implemented through an RTK receiver, which connects to the sensor interface and continuously receives satellite navigation signals during UAV takeoff and landing. It then calculates the UAV's RTK position data using real-time dynamic differential technology. This RTK position data includes the UAV's coordinates in three-dimensional space, providing a global positioning reference for the system. The RTK receiver's operation must adapt to the UAV's takeoff and landing altitude range and motion state to ensure stable reception of satellite signals and continuous position data output even during low-altitude takeoff and landing phases.

[0028] The visual positioning module utilizes a visual sensor, which is installed at the UAV's takeoff and landing observation location and connected to a sensor interface. During takeoff and landing, it continuously acquires images of the ground scene. After the image data acquired by the visual sensor is transmitted to the flight control computer, a preset image processing algorithm extracts ground feature information, thereby calculating the UAV's visual position data and providing a relative positioning reference for the system. The visual sensor's acquisition frequency is adapted to the overall system processing frequency to ensure the real-time nature of the image data.

[0029] The environment perception module includes an RTK environment perception submodule and a visual environment perception submodule, which are used to evaluate RTK environmental factors and visual environmental factors in real time, respectively, and provide input data for the weighted decision-maker.

[0030] The core task of the RTK environment awareness submodule is to evaluate the sky view factor, signal quality factor, and multipath risk factor. The specific evaluation process is as follows: The sky view factor is calculated using the geometric distribution accuracy factor of visible satellites and the number of visible satellites. The geometric distribution accuracy factor reflects the impact of the satellite's spatial distribution geometry on positioning accuracy; the more uniform the distribution, the smaller the accuracy attenuation. This factor is directly obtained from the solution data of the satellite navigation system. The number of visible satellites is the total number of satellites whose signals the RTK receiver can stably receive at the current moment. The calculation of the sky view factor comprehensively considers the above two indicators. The better both indicators are, the better the sky view factor reflects the environmental adaptability of RTK positioning.

[0031] The signal quality factor is calculated using the average signal-to-noise ratio (SNR) and variance of all visible satellites. The average SNR is the arithmetic mean of the SNR of each visible satellite, reflecting the overall strength of the satellite signal. The SNR variance is the sum of squared deviations of the SNR of each satellite from the average, reflecting the consistency of signal quality across all satellites. The signal quality factor is calculated by combining these two indicators; higher overall strength and better consistency result in a better signal quality factor.

[0032] The multipath risk factor is calculated by querying a pre-stored simplified obstacle map around the drone's nest and combining it with the drone's real-time position. The calculation process first determines the relative positional relationship between the drone's real-time position and each obstacle in the simplified obstacle map. Then, it calculates the minimum Euclidean distance between the drone and the obstacle surface in the horizontal direction, and the vertical height difference between the drone and the top of the obstacle. These two calculations together reflect the degree of risk of multipath interference from obstacles affecting the drone's current position. The greater the minimum Euclidean distance and the larger the relative height difference, the better the multipath risk factor.

[0033] The RTK environment perception submodule transmits the calculated sky view factor, signal quality factor, and multipath risk factor to the weight decision-maker in real time.

[0034] The core task of the visual environment perception submodule is to evaluate the illumination robustness factor, image sharpness factor, and feature abundance factor. The specific evaluation process is as follows: The illumination robustness factor is calculated using the overall mean brightness and standard deviation of the image. The overall mean brightness is the average of the gray values ​​of all pixels in the image, reflecting the overall illumination intensity. The standard deviation brightness is the square root of the sum of squares of the deviations of all pixel gray values ​​from the mean, reflecting the uniformity of the illumination distribution. The illumination robustness factor combines these two indicators; a better factor indicates a moderate and uniform overall illumination intensity.

[0035] The image sharpness factor is obtained by calculating the Laplacian variance of the image. The specific process is as follows: the original image acquired by the vision sensor is convolved using the Laplacian operator to obtain an enhanced image; then, the variance of this enhanced image is calculated, which is the Laplacian variance. A larger Laplacian variance indicates richer edge details, higher image sharpness, and a better image sharpness factor.

[0036] The feature abundance factor is obtained by calculating the total number and distribution density of corner points or feature points in an image. Corner points or feature points are extracted using the Harris corner detection algorithm, which can effectively identify corner features in an image; the distribution density is the ratio of the number of extracted corner points or feature points to the total number of pixels in the image. The calculation of the feature abundance factor combines the total number of feature points and the distribution density. The higher the total number and the more uniform the distribution density, the better the feature abundance factor, which is more conducive to the accurate solution of visual localization.

[0037] The visual environment perception submodule transmits the calculated illumination robustness factor, image sharpness factor, and feature abundance factor to the weight decision-maker in real time.

[0038] The weighted decision-maker connects to the environment perception module, receives RTK environmental factors and visual environmental factors, processes them through a fuzzy logic rule set, and outputs RTK confidence coefficients and visual confidence coefficients. At the same time, it calculates the environmental interference index to assist in the rule-triggered threshold adjustment.

[0039] The fuzzy logic rule set contains multiple fuzzy rules, each consisting of a premise and a conclusion. The premise is a combination of fuzzy linguistic variables representing RTK environmental factors and visual environmental factors, while the conclusion is a combination of fuzzy linguistic variables representing RTK confidence coefficients and visual confidence coefficients. All fuzzy linguistic variables are categorized into three levels: poor, moderate, and excellent. This categorization is based on a range determined from extensive experimental data, objectively reflecting the actual levels of each factor and its confidence coefficient.

[0040] The following is an example of how fuzzy rules are constructed: If both the RTK environmental factor and the visual environmental factor are excellent, then the RTK confidence coefficient is excellent and the visual confidence coefficient is excellent; if both the RTK environmental factor and the visual environmental factor are moderate, then the RTK confidence coefficient is excellent and the visual confidence coefficient is moderate; if both the RTK environmental factor and the visual environmental factor are poor, then the RTK confidence coefficient is poor and the visual confidence coefficient is excellent; all other combinations correspond to the corresponding fuzzy rules to ensure coverage of various environmental scenarios.

[0041] The operation of the weighted decision-maker includes three steps: fuzzification, fuzzy inference, and defuzzification. Fuzzification process: The precise environmental factors input by the environmental perception module are converted into corresponding fuzzy linguistic variable levels. The membership function is used to determine the membership degree of each precise value to different fuzzy levels, thus completing the conversion from precise to fuzzy quantities.

[0042] Fuzzy reasoning process: Using the Mamdani fuzzy model, the fuzzified environmental factor linguistic variables are matched with the premise part of the fuzzy rule set to trigger the corresponding fuzzy rules. The membership degree of the conclusion part of each triggered rule is calculated by the minimax method to obtain the fuzzy set of credibility coefficients.

[0043] Defuzzification process: The centroid method is used to perform integral operations on the fuzzy set of the confidence coefficients, and the precise value corresponding to the centroid of the fuzzy set is obtained. This precise value is the final output result of the RTK confidence coefficient and the visual confidence coefficient. The output confidence coefficient is a real number between 0 and 1, where 0 indicates that the data of the corresponding positioning module is completely unreliable, and 1 indicates that the data of the corresponding positioning module is completely reliable. The larger the value, the higher the data reliability.

[0044] The weighted decision maker updates the RTK credibility coefficient and visual credibility coefficient at fixed intervals to ensure that the coefficients can track environmental changes in real time and provide a dynamic adjustment basis for data fusion.

[0045] The weighted decision maker has a built-in environmental interference index calculation unit, which is used to generate a unified environmental interference index by combining RTK environmental factors and visual environmental factors. The calculation formula is as follows: In the formula: This is the environmental interference index, used to reflect the overall interference level of the current takeoff and landing environment on RTK positioning and visual positioning. The higher the value, the more severe the environmental interference; The normalized comprehensive score for RTK environmental factors is obtained by weighting and calculating the sky view factor, signal quality factor, and multipath risk factor. The weighting coefficients are preset according to the degree of influence of each factor on RTK positioning accuracy. The value ranges from 0 to 1, and the closer it is to 1, the better the RTK environment is; The normalized comprehensive score for visual environment factors is obtained by weighting and calculating the factors of illumination robustness, image sharpness, and feature abundance. The weighting coefficients are preset according to the degree of influence of each factor on visual positioning accuracy. The value ranges from 0 to 1, with a value closer to 1 indicating a better visual environment. and Let be the weighting coefficient, satisfying Its value is preset based on the degree of dependence of the take-off and landing scenario on RTK positioning and visual positioning. For example, it can be set in an open environment. Larger size, can be set in environments with severe obstruction. Relatively large.

[0046] The environmental interference index is used to assist the weight decision-maker in adjusting the rule triggering threshold of the fuzzy logic rule set. When the environmental interference index is higher than the preset threshold, it indicates that the current environment is seriously interfering with the positioning system. The weight decision-maker will prioritize the use of rules that reduce the RTK confidence coefficient or visual confidence coefficient to avoid unreliable positioning data from affecting the fusion accuracy. When the environmental interference index is lower than the preset threshold, the original rule triggering threshold remains unchanged.

[0047] The data fusion module connects the RTK positioning module, the visual positioning module, and the weight decision unit. It receives RTK location data, visual location data, and two types of confidence coefficients, and achieves data fusion through the Kalman filter algorithm to output the precise location data of the UAV.

[0048] The core of the data fusion module is to adjust the measurement noise covariance matrix of the Kalman filter based on the confidence coefficient. The adjustment formula is as follows: In the formula: The adjusted RTK measurement noise covariance matrix reflects the noise level of the RTK location data, and the matrix dimensions are consistent with the dimensions of the location data. The initial value of the noise covariance matrix for RTK measurement is preset based on the performance parameters of the RTK receiver and determined through experimental calibration, reflecting the inherent noise level of RTK positioning in an ideal environment; This is the RTK confidence coefficient, output by the weighted decision maker, with a value ranging from 0 to 1. The larger the value, the more reliable the RTK location data. The smaller the value, the higher the confidence level of the RTK location data in the Kalman filter; The adjusted visual measurement noise covariance matrix reflects the noise level of the visual location data, and the matrix dimensions are consistent with the dimensions of the location data. The initial value of the visual measurement noise covariance matrix is ​​preset based on the performance parameters of the visual sensor and determined through experimental calibration, reflecting the inherent noise level of visual positioning in an ideal environment; The visual credibility coefficient is output by the weighted decision maker and ranges from 0 to 1. The larger the value, the more reliable the visual position data. The smaller the value, the higher the confidence level of the visual position data in the Kalman filter.

[0049] The Kalman filter algorithm consists of two steps: prediction and update. Prediction Steps: Based on the UAV's motion model, and utilizing the fused position data and motion state from the previous moment, the current UAV position and error covariance matrix are predicted. The motion model is preset according to the UAV's dynamic characteristics, reflecting the UAV's position change patterns.

[0050] Update steps: Adjust the... and Substituting the values ​​into the Kalman filter update equation, the Kalman gains of the RTK location data and the visual location data are calculated separately. Then, based on these Kalman gains, the predicted location data is fused with the two types of measured location data to update and obtain the precise location data and error covariance matrix for the current moment. During the fusion process, the higher the confidence coefficient of the positioning data, the larger the corresponding Kalman gain, and the greater its contribution to the fusion result.

[0051] The data fusion module outputs the fused precise location data in real time, providing a positioning basis for the take-off and landing control of the drone.

[0052] The degradation strategy management module connects the weight decision-maker and the data fusion module to monitor the RTK reliability coefficient and the visual reliability coefficient in real time. When both coefficients are lower than the preset threshold, it indicates that the reliability of RTK positioning and visual positioning is insufficient in the current environment, and the system starts the degradation strategy.

[0053] Degradation strategies include pure visual navigation mode, pure RTK navigation mode, and inertial navigation descent mode. The mode selection is based on the specific values ​​of RTK environmental factors and visual environmental factors. When the overall visual environment factors are better than the RTK environment factors, it indicates that the visual positioning has a relatively stronger environmental adaptability. The system starts the pure visual navigation mode and performs navigation control based solely on visual position data. When the overall RTK environmental factors are better than the visual environmental factors, it indicates that the RTK positioning has a relatively stronger environmental adaptability. The system starts the pure RTK navigation mode and performs navigation control based solely on RTK location data. When both RTK and visual environmental factors are poor, it indicates that the reliability of both positioning methods is insufficient. The system then activates the inertial navigation descent mode, which uses data from the UAV's built-in inertial sensors and historical fusion location data to achieve a slow descent and ensure safe takeoff and landing.

[0054] After the degradation strategy is activated, the degradation strategy management module continuously monitors the changes in RTK environmental factors and visual environmental factors. When the two types of factors return to the normal range and the corresponding confidence coefficient rises above the preset threshold, it automatically switches back to the normal fusion mode and restores the fusion positioning of RTK and vision.

[0055] In this embodiment, after the UAV initiates the takeoff and landing procedure, the various modules of the system operate collaboratively according to the following process: 1. Upon power-up of the embedded hardware system, the flight control computer loads a pre-stored simplified diagram of obstacles around the avionics nest and a set of fuzzy logic rules. The sensor interface establishes communication connections with the RTK receiver and the vision sensor. 2. The RTK positioning module and the visual positioning module start synchronously, continuously collecting and transmitting RTK location data and visual location data; 3. The environmental perception module runs in real time. The RTK environmental perception submodule and the visual environmental perception submodule calculate various environmental factors and transmit them to the weight decision-maker. 4. The weighted decision-maker receives environmental factors, calculates the environmental interference index, adjusts the fuzzy rule trigger threshold, and outputs the RTK credibility coefficient and visual credibility coefficient through the fuzzy logic rule set; 5. The data fusion module receives location data and confidence coefficients, adjusts the measurement noise covariance matrix, fuses the data using a Kalman filter algorithm, and outputs accurate location data; 6. The degradation strategy management module monitors the credibility coefficient and determines whether to start the degradation strategy based on the threshold. If it needs to be started, the optimal degradation mode is selected, and the normal fusion mode is switched back after the environment is restored. 7. The UAV completes take-off and landing control based on the precise location data output by the data fusion module. After take-off and landing are completed, the system stops the core module operation.

[0056] Example 2 like Figure 3 As shown in Example 1, this example details the complete workflow of the UAV precision take-off and landing control system based on RTK and vision fusion during UAV take-off and landing missions. This workflow describes each stage of the system from startup to shutdown, as well as the execution sequence and logical relationships of each functional module over time.

[0057] Specific steps of the system workflow: 1. System startup and initialization After the drone is powered on, the embedded hardware platform starts up.

[0058] The flight control computer completes a self-test and loads two types of key data pre-stored in non-volatile memory: a simplified map of obstacles around the nacelle and a fuzzy logic rule set. The obstacle map provides three-dimensional spatial information about obstacles around the nacelle, while the fuzzy logic rule set contains decision rules for evaluating sensor reliability.

[0059] Sensor interface initialization: establish a stable communication link with the RTK receiver and vision sensor, and complete the configuration of sensor parameters.

[0060] All software modules of the system have completed loading and initialization and have entered the ready state.

[0061] 2. Synchronous collection of positioning data The RTK positioning module continuously receives satellite navigation signals through an RTK receiver, calculates the RTK position data of the UAV using real-time dynamic differential technology, and outputs the RTK position data of the UAV.

[0062] The visual positioning module continuously acquires ground scene images through airborne visual sensors and calculates the drone's visual position data using preset image processing algorithms.

[0063] RTK location data and visual location data are simultaneously timestamped and transmitted to the data fusion module, while also providing raw input to the environmental perception module.

[0064] 3. Real-time assessment of environmental factors The environmental perception module has started operating periodically.

[0065] The RTK environment awareness submodule calculates three RTK environment factors based on the current output data of the RTK receiver: Sky Visibility Factor: An assessment based on the number of currently visible satellites and their geometric distribution in the sky.

[0066] Signal quality factor: Evaluated based on the average signal-to-noise ratio of all visible satellite signals and their consistency.

[0067] Multipath risk factor: By querying a pre-stored obstacle map and combining it with the real-time location of the drone, the proximity of the drone to nearby obstacles is calculated to assess the risk of multipath reflection of signals.

[0068] The visual environment perception submodule calculates three visual environment factors based on the images currently acquired by the visual sensor: Illumination robustness factor: evaluated based on the overall brightness level and uniformity of the image.

[0069] Image sharpness factor: Evaluated based on the edge sharpness and detail richness of an image.

[0070] Feature abundance factor: Evaluated based on the number and distribution of corner points or feature points that can be used for localization in the image.

[0071] The calculated RTK environmental factors and visual environmental factors are transmitted to the weighted decision-maker in real time.

[0072] 4. Dynamic calculation of credibility coefficient The weighted decision-maker receives real-time environmental factors from the environmental perception module.

[0073] The weighted decision maker first calculates a comprehensive environmental interference index, which reflects the overall adverse impact of the current environment on the positioning system.

[0074] The weighted decision-maker converts precise environmental factor input values ​​into fuzzy language descriptions such as "poor," "medium," and "excellent," and then applies a pre-stored set of fuzzy logic rules for reasoning.

[0075] Through fuzzy inference and defuzzification calculations, the weighted decision maker outputs two precise confidence coefficients: the RTK confidence coefficient and the visual confidence coefficient. These two coefficients are real numbers between 0 and 1; the higher the value, the more reliable the data from the corresponding sensor in the current environment.

[0076] The environmental interference index can be used to fine-tune the triggering conditions of fuzzy rules, for example, to make the RTK credibility assessment more conservative when interference is severe.

[0077] 5. Degradation strategy condition judgment The degradation strategy management module monitors the RTK credibility coefficient and visual credibility coefficient output by the weight decision-maker in real time.

[0078] This module determines whether both coefficients are simultaneously below a preset reliability threshold. This threshold is preset based on system security redundancy requirements.

[0079] If the judgment result is "yes", meaning that the reliability of both sensors is insufficient, the process proceeds to step 6.

[0080] If the judgment result is "no", that is, at least one sensor is still reliable, then the process proceeds to step 7.

[0081] 6. Implementation and monitoring of degradation strategies When the degradation condition is triggered, the degradation strategy management module selects and activates one of the three degradation strategies based on the specific scores of the current RTK environment factors and visual environment factors: Pure vision navigation mode: Activated when the overall visual environmental factors are significantly better than the RTK environmental factors. The system ignores RTK position data and relies solely on visual position data for navigation and control.

[0082] Pure RTK navigation mode: Activated when RTK environmental factors are significantly better than visual environmental factors overall. The system ignores visual position data and relies solely on RTK position data for navigation and control.

[0083] Inertial navigation descent mode: Activated when both RTK and visual environmental factors are poor. The system mainly relies on data from the onboard inertial measurement unit and combines it with historical fusion positions for dead reckoning, controlling the UAV to descend slowly and cautiously at a higher speed and attitude, prioritizing safety.

[0084] During degraded mode operation, the degraded strategy management module continuously monitors changes in environmental factors. Once environmental factors improve, causing both RTK and visual confidence coefficients to rise above the threshold, the module automatically instructs the system to switch back to normal fusion mode. In degraded mode, the system directly outputs the location data of the selected degraded mode for control.

[0085] 7. Adaptive fusion of multi-source data When the system is in normal fusion mode, the data fusion module performs the following operations: The module receives timestamped location data from the RTK positioning module and the visual positioning module.

[0086] Simultaneously, it receives real-time RTK confidence coefficients and visual confidence coefficients from the weighted decision-maker.

[0087] The data fusion module uses the Kalman filter algorithm as its core. Crucially, it dynamically adjusts the measurement noise covariance matrix corresponding to RTK and visual positions in the filtering algorithm using a confidence coefficient. Higher confidence results in a greater weight allocation to the sensor data during the fusion process.

[0088] After Kalman filtering prediction and update steps, the data fusion module outputs an optimized, more accurate and reliable UAV position estimate.

[0089] 8. Flight control execution The UAV flight control system receives precise location data from the data fusion module or degraded location data from the degraded strategy management module.

[0090] The flight control system compares this location information with the desired takeoff and landing trajectory, generates control commands, drives the drone's actuators, and achieves precise takeoff or landing maneuvers.

[0091] 9. Task Loop and End Steps 2 to 8 are executed cyclically at a frequency of no less than 20Hz throughout the entire take-off and landing process of the drone to ensure that the system can respond quickly to changes in the environment and the drone's status.

[0092] Real-time performance is ensured through the multi-threaded architecture of the flight control computer, with environmental perception and weighted decision-making running in high-priority threads and data fusion running in medium-priority threads.

[0093] Once the drone has successfully taken off or landed and stabilized at the target location, the system can receive instructions to stop the core positioning fusion loop or enter a low-power standby state, thus ending the workflow.

[0094] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A precision take-off and landing control system for unmanned aerial vehicles (UAVs) based on RTK and vision fusion, characterized in that, It includes an RTK positioning module, a visual positioning module, an environmental perception module, a weighted decision-maker, and a data fusion module. The RTK positioning module is used to acquire the RTK location data of the UAV; The visual positioning module is used to acquire the visual position data of the UAV; The environment perception module includes an RTK environment perception submodule and a visual environment perception submodule. The RTK environment perception submodule is used to evaluate RTK environmental factors in real time, and the visual environment perception submodule is used to evaluate visual environmental factors in real time. The weighted decision-maker is connected to the environment perception module and is used to output the RTK confidence coefficient and the visual confidence coefficient through a fuzzy logic rule set based on the RTK environment factors and the visual environment factors. The data fusion module connects the RTK positioning module, the visual positioning module, and the weight decision-maker. It is used to adjust the weights of the RTK position data and the visual position data according to the RTK confidence coefficient and the visual confidence coefficient, and to perform data fusion through the Kalman filter algorithm to output the accurate position data of the UAV.

2. The UAV precision take-off and landing control system based on RTK and vision fusion as described in claim 1, characterized in that, The RTK environmental factors include sky view factor, signal quality factor, and multipath risk factor; The visual environment factors include illumination robustness factor, image sharpness factor, and feature abundance factor; The sky view factor is calculated based on the distribution geometry and number of visible satellites; the signal quality factor is calculated based on the average and variance of the signal-to-noise ratio of visible satellites; and the multipath risk factor is calculated based on the relative relationship between a pre-stored simplified map of obstacles around the drone nest and the real-time position of the drone. The illumination robustness factor is calculated based on the image brightness distribution and contrast, the image sharpness factor is calculated based on the Laplacian variance, and the feature abundance factor is calculated based on the number of corner points or feature points in the image. The weighted decision-maker updates the RTK credibility coefficient and the visual credibility coefficient with a period of 50mm. The data fusion module uses the RTK confidence coefficient and the visual confidence coefficient as the basis for adjusting the measurement noise covariance matrix in the Kalman filtering algorithm. The RTK confidence coefficient is used to adjust the measurement noise covariance of the RTK position data, and the visual confidence coefficient is used to adjust the measurement noise covariance of the visual position data.

3. The UAV precision take-off and landing control system based on RTK and vision fusion as described in claim 2, characterized in that, The specific methods used by the RTK environmental perception submodule to evaluate the sky view factor, signal quality factor, and multipath risk factor include: The sky view factor is obtained by calculating the geometric distribution accuracy factor of visible satellites and the number of visible satellites. The geometric distribution accuracy factor is the accuracy attenuation factor caused by the geometric distribution of satellites in space, and the number of visible satellites is the total number of satellites that can be received at the current moment. The signal quality factor is obtained by calculating the average signal-to-noise ratio (SNR) and the SNR variance of all visible satellites. The average SNR is the arithmetic mean of the SNR of each satellite, and the SNR variance is the sum of squared deviations of the SNR of each satellite from the average. The multipath risk factor is obtained by querying a pre-stored simplified map of obstacles around the nest and calculating the closest distance and relative height difference between the UAV's real-time position and the obstacles. The simplified map of obstacles is a three-dimensional obstacle model around the nest obtained through pre-surveyed mapping. The closest distance is the minimum Euclidean distance between the UAV and the obstacle surface in the horizontal direction, and the relative height difference is the height difference between the UAV and the top of the obstacle in the vertical direction. The RTK environment perception submodule transmits the sky view factor, signal quality factor, and multipath risk factor as inputs to the weight decision-maker.

4. The UAV precision take-off and landing control system based on RTK and vision fusion as described in claim 2, characterized in that, The specific methods used by the visual environment perception submodule to evaluate the illumination robustness factor, image sharpness factor, and feature abundance factor include: The illumination robustness factor is obtained by calculating the overall brightness mean and brightness standard deviation of the image. The overall brightness mean is the average value of the gray values ​​of all pixels in the image, and the brightness standard deviation is the square root of the sum of squares of the deviations of the gray values ​​of all pixels from the mean. The image sharpness factor is obtained by calculating the Laplacian variance of the image, which is the variance value of the image obtained after performing Laplacian operator convolution on the image. The feature abundance factor is obtained by calculating the total number and distribution density of corner points or feature points in the image. The corner points or feature points are extracted by the Harris corner detection algorithm, and the distribution density is the ratio of the number of corner points or feature points to the total number of pixels in the image. The visual environment perception submodule transmits the illumination robustness factor, image clarity factor, and feature abundance factor as inputs to the weight decision-maker.

5. The UAV precision take-off and landing control system based on RTK and vision fusion as described in claim 1, characterized in that, The weighted decision-maker uses a fuzzy logic rule set to process the RTK environmental factors and visual environmental factors to output the RTK confidence coefficient and the visual confidence coefficient. The fuzzy logic rule set includes multiple fuzzy rules. Each fuzzy rule contains a premise part and a conclusion part. The premise part is a combination of fuzzy linguistic variables of RTK environmental factors and visual environmental factors. The conclusion part is a fuzzy linguistic variable of RTK credibility coefficient and visual credibility coefficient. The fuzzy linguistic variables include three levels: poor, medium, and excellent. The weighted decision-maker fuzzifies the input environmental factors through a fuzzy inference engine and matches them with fuzzy rules. Then, through a defuzzification operation, it converts the fuzzy linguistic variables in the conclusion part into precise RTK credibility coefficients and visual credibility coefficients. The fuzzy inference engine adopts the Mamdani fuzzy model, and the defuzzification operation uses the centroid method. The RTK credibility coefficient and visual credibility coefficient output by the weighted decision maker are real numbers between 0 and 1, where 0 represents completely untrustworthy and 1 represents completely trustworthy.

6. The UAV precision take-off and landing control system based on RTK and vision fusion according to claim 1, characterized in that, The data fusion module uses the Kalman filter algorithm for data fusion and uses the RTK confidence coefficient and visual confidence coefficient as the basis for adjusting the measurement noise covariance matrix. The data fusion module adjusts the measurement noise covariance matrix of the RTK location data based on the RTK confidence coefficient, and adjusts the measurement noise covariance matrix of the visual location data based on the visual confidence coefficient. The adjustment formula is as follows: in The adjusted RTK measurement noise covariance matrix, This is the adjusted visual measurement noise covariance matrix. Initialize the noise covariance matrix for RTK measurements. The initial values ​​are for the visual measurement noise covariance matrix. This represents the RTK credibility coefficient. Visual credibility coefficient; The data fusion module uses the adjusted measurement noise covariance matrix to perform Kalman filtering prediction and update steps, fusing RTK position data and visual position data to output the UAV's accurate position data.

7. The UAV precision take-off and landing control system based on RTK and vision fusion as described in claim 1, characterized in that, The weighted decision-maker also includes an environmental interference index calculation unit, which is used to generate a unified environmental interference index by integrating RTK environmental factors and visual environmental factors. The environmental interference index is calculated using the following formula: in The environmental disturbance index. The normalized comprehensive score for RTK environmental factors. The normalized comprehensive score for visual environmental factors, and The weighting coefficients are and satisfy the following conditions: ; The normalized comprehensive score of the RTK environmental factors The normalized comprehensive score of the visual environment factors is calculated by weighting and summing the sky view factor, signal quality factor, and multipath risk factor. The calculation is performed by weighting and summing the illumination robustness factor, image sharpness factor, and feature abundance factor. The environmental interference index is used to assist the weight decision-maker in adjusting the rule triggering threshold in the fuzzy logic rule set. When the environmental interference index is higher than the preset threshold, the weight decision-maker will prioritize the use of rules that reduce the RTK credibility coefficient.

8. The UAV precision take-off and landing control system based on RTK and vision fusion as described in claim 1, characterized in that, The system also includes a degradation strategy management module, which is connected to the weight decision-maker and the data fusion module, and is used to activate the degradation strategy when the RTK confidence coefficient and the visual confidence coefficient are both lower than a preset threshold. The degradation strategies include pure visual navigation mode, pure RTK navigation mode, and inertial navigation descent mode. The degradation strategy management module selects the optimal degradation strategy based on the specific values ​​of RTK environmental factors and visual environmental factors. When the overall visual environmental factors are better than the RTK environmental factors, the pure visual navigation mode is activated. When the overall RTK environmental factors are better than the visual environmental factors, the pure RTK navigation mode is activated. When both are worse, the inertial navigation descent mode is activated. After the degradation strategy is activated, the degradation strategy management module continuously monitors changes in environmental factors and automatically switches back to normal fusion mode when the environmental factors return to the normal range.

9. The UAV precision take-off and landing control system based on RTK and vision fusion according to claim 1, characterized in that, The system is implemented through embedded hardware, which includes a flight control computer and sensor interfaces. The flight control computer runs the algorithms of the environmental perception module, weight decision-maker, and data fusion module, and the sensor interface connects to the RTK receiver and the vision sensor. The flight control computer adopts a multi-threaded architecture, in which the environmental perception module and weight decision-maker run in a high-priority thread, and the data fusion module runs in a medium-priority thread; The system pre-stores a simplified diagram of obstacles around the drone nest and a set of fuzzy logic rules. The simplified diagram of obstacles is generated from the mapping data during the drone's first landing, and the set of fuzzy logic rules is obtained by optimizing the machine learning algorithm during the training phase.

Images