Unmanned aerial vehicle inspection precision positioning method and device fusing vision and satellite navigation

Abstract

The application provides a kind of unmanned aerial vehicle inspection precision positioning method and device of fusion vision and satellite navigation, it is related to space information technical field.The application extracts satellite signal quality features reflecting space information reliability and multi-dimensional features such as visual geometric texture reflecting ground environment characteristics, and deeply perceives environment state;Further, based on real-time dynamic generation of fusion strategy adapting to current environment, intelligently adjust data weight and algorithm model, break through the limitation of traditional fixed parameter fusion mode;Through visual recognition, understand the specific inspection stage and target object of unmanned aerial vehicle, and load the corresponding prior task constraint model, so that the positioning process closely matches the accuracy and safety requirements of actual operation.Finally, under the joint guidance of dynamic strategy and task constraint, fusion calculation is carried out, the problem of inaccurate positioning of unmanned aerial vehicle in complex environment is solved, and the environmental adaptability, robustness and task-oriented accuracy of the positioning system are improved.

Description

Technical Field

[0001] This invention relates to the field of aerospace information technology, and in particular to a method and device for precise positioning of unmanned aerial vehicle (UAV) inspections that integrates vision and satellite navigation. Background Technology

[0002] With the rapid development and widespread application of aerospace information technology, the aerospace information system, centered on satellite navigation, remote sensing, and communication, has become a crucial space infrastructure for modern society. In the field of unmanned aerial vehicle (UAV) inspection, aerospace information, especially Global Navigation Satellite System (GNSS), provides UAVs with an indispensable global coverage, all-weather absolute positioning reference, serving as a key technological support for achieving wide-area operations and unified spatial coordinates. However, positioning technologies relying on a single aerospace information source face a significant "last mile" bottleneck when dealing with complex ground application scenarios.

[0003] On the one hand, GNSS signals in aerospace information are susceptible to interference from complex electromagnetic and physical environments such as urban high-rise building clusters, dense forests, and high-voltage transmission corridors, leading to signal attenuation, multipath reflection, or even complete blockage. This causes positioning accuracy to deteriorate from the meter level to the ten-meter level or even fail, making it difficult to meet the stringent requirements for decimeter-level and centimeter-level positioning accuracy in applications such as precise inspection of power facilities and precision agricultural monitoring. On the other hand, current methods for fusing aerospace information with ground-based sensor information are mostly still at a relatively rudimentary static integration level, generally employing loosely coupled or parameter-fixed filtering algorithms. This fails to fully analyze and utilize the multi-dimensional state characteristics carried by the aerospace information itself—such as signal strength, carrier-to-noise ratio, and spatial distribution of visible satellites—which reflect environmental quality. Once aerospace information is interfered with or interrupted, the overall robustness of the system will significantly decrease, severely impacting the positioning accuracy and operational continuity of UAVs in complex environments. Summary of the Invention

[0004] This invention provides a method and device for precise positioning of UAVs during inspections that integrates vision and satellite navigation, solving the technical problem of inaccurate UAV positioning in complex environments.

[0005] In a first aspect, the present invention provides a method for precise positioning of unmanned aerial vehicles (UAVs) during inspections that integrates vision and satellite navigation. The method includes: acquiring satellite navigation data, visual sensor data, and inertial measurement data of the UAV; extracting multi-dimensional features based on the satellite navigation data, visual sensor data, and inertial measurement data, wherein the multi-dimensional features include satellite signal quality features, visual image texture features, geometric structure features, and real-time motion state features; dynamically generating a fusion positioning strategy matching the current environment based on the multi-dimensional features, wherein the fusion positioning strategy includes fusion weights among various types of data, a state estimation algorithm, and an interference source error compensation model; identifying the current inspection stage and target inspection object of the UAV based on the visual sensor data, and determining a priori task constraint model corresponding to the current inspection stage and target inspection object; and performing fusion calculations on the satellite navigation data and visual sensor data based on the fusion positioning strategy and using the priori task constraint model as optimization constraints to generate real-time positioning information of the UAV.

[0006] Secondly, embodiments of the present invention provide a precise positioning device for UAV inspection that integrates vision and satellite navigation. This device includes a communication module and a processing module. The communication module is used to acquire satellite navigation data, visual sensor data, and inertial measurement data from the UAV. The processing module is used to extract multi-dimensional features based on the satellite navigation data, visual sensor data, and inertial measurement data. These multi-dimensional features include satellite signal quality features, visual image texture features, geometric structure features, and real-time motion state features. Based on these multi-dimensional features, a fusion positioning strategy matching the current environment is dynamically generated. The fusion positioning strategy includes fusion weights among various types of data, a state estimation algorithm, and an interference source error compensation model. Based on the visual sensor data, the current inspection stage and target inspection object of the UAV are identified, and a priori task constraint model corresponding to the current inspection stage and target inspection object is determined. Based on the fusion positioning strategy, and using the priori task constraint model as an optimization constraint, the satellite navigation data and visual sensor data are fused and calculated to generate real-time positioning information for the UAV.

[0007] Thirdly, embodiments of the present invention provide an electronic device including a memory and a processor. The memory stores a computer program, and the processor is used to call and run the computer program stored in the memory to perform the steps of the method as described in the first aspect and any possible implementation thereof.

[0008] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, characterized in that, when executed by a processor, the computer program implements the steps of the method as described in the first aspect and any possible implementation thereof.

[0009] This invention provides a method and apparatus for precise positioning of unmanned aerial vehicles (UAVs) during inspections, integrating vision and satellite navigation. The invention jointly extracts multi-dimensional features, such as satellite signal quality characteristics reflecting the reliability of space-air information and visual geometric textures reflecting ground environmental characteristics, to deeply perceive the environmental state. Then, based on real-time dynamic generation of a fusion strategy adapted to the current environment, it intelligently adjusts data weights and algorithm models, overcoming the limitations of traditional fixed-parameter fusion modes. Through visual recognition, it understands the specific inspection stage and target object of the UAV and loads corresponding prior task constraint models, ensuring that the positioning process closely aligns with the accuracy and safety requirements of actual operations. Finally, under the joint guidance of dynamic strategies and task constraints, fusion calculations are performed to solve the problem of inaccurate UAV positioning in complex environments, improving the environmental adaptability, robustness, and task-oriented accuracy of the positioning system. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating a precise positioning method for UAV inspection that integrates vision and satellite navigation, provided by an embodiment of the present invention.

[0012] Figure 2 This is a schematic diagram of the structure of a drone inspection precision positioning device that integrates vision and satellite navigation, provided in an embodiment of the present invention;

[0013] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0014] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0015] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0016] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the steps or modules listed, but may optionally include other steps or modules not listed, or may optionally include other steps or modules inherent to such process, method, product, or device.

[0017] To make the objectives, technical solutions, and advantages of the present invention clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0018] As described in the background section, traditional single-sensor positioning solutions have significant limitations in complex environments. Pure GNSS positioning is susceptible to multipath effects in densely built-up areas or forested areas; when signal obstruction results in fewer than 4 visible satellites, the probability of positioning interruption reaches 100%. In urban canyon scenarios, multipath errors can reach 3-5 meters, which is completely unacceptable for inspection accuracy requirements. While pure visual SLAM can work in environments without GNSS, it suffers from trajectory drift in textureless areas such as deserts and water surfaces due to feature loss. Experimental data shows that after 10 minutes of cruising in textureless areas, the cumulative error can reach 8-10 meters, and the error increases linearly over time (at a rate of approximately 0.15 meters per minute).

[0019] Fusion positioning technology achieves complementary advantages through a collaborative mechanism of absolute coordinate anchoring and relative motion constraints. In areas with good GNSS signal (≥5 visible satellites), the system uses GNSS / IMU data as a reference, and visual features are used to calibrate multipath errors, improving planar positioning accuracy from 1.2 meters to 0.15 meters. When entering areas with weak GNSS signal (2-3 visible satellites), the visual module maintains positioning accuracy at the 0.5-meter level by matching with the initialized feature library. Even if GNSS signal is completely lost, the system can still maintain decimeter-level positioning capability for up to 30 seconds by relying on visual loop closure detection and short-term IMU prediction. This layered fault-tolerant mechanism significantly improves the robustness of the positioning system in complex environments.

[0020] like Figure 1 As shown, this invention provides a precise positioning method for UAV inspection that integrates vision and satellite navigation. The method includes steps S101-S105.

[0021] S101. Acquire satellite navigation data, visual sensor data, and inertial measurement data of the UAV.

[0022] In some embodiments, satellite navigation data includes three-dimensional coordinates (longitude, latitude, and elevation) and velocity information from a GNSS module, with a typical sampling frequency of 1-10 Hz; visual sensor data comes from monocular / binocular cameras or RGB-D cameras, providing image sequences and relative pose information, with a sampling frequency of 20-30 Hz. Inertial measurement data is real-time data acquired by the UAV's inertial measurement unit.

[0023] S102. Extract multi-dimensional features based on satellite navigation data, visual sensor data, and inertial measurement data.

[0024] In this embodiment of the application, the multi-dimensional features include satellite signal quality features, visual image texture features, geometric structure features, and real-time motion state features.

[0025] In some embodiments, satellite signal quality features include the number of visible satellites, signal-to-noise ratio, carrier phase continuity, satellite spatial geometric distribution, and position accuracy factor; visual image texture features include the number of image key points, distribution uniformity, and local descriptors; geometric structure features include edge detection, line segment detection, and linear structures, corners, and geometric contours identified by matching with known models; and real-time motion state features include real-time linear velocity and angular velocity estimated by filtering based on the three-dimensional acceleration and angular velocity output by the IMU.

[0026] As one possible implementation, step S102 can be specifically implemented as steps S1021-S1028.

[0027] S1021. Analyze the satellite navigation data and calculate the number of visible satellites at the current time, the signal-to-noise ratio of each satellite, the carrier phase continuity index, and the spatial geometric distribution of the satellites.

[0028] S1022. Calculate the position accuracy factor based on the spatial geometric distribution of satellites.

[0029] S1023. Based on the number of visible satellites at the current time, the signal-to-noise ratio of each satellite, the carrier phase continuity index, the spatial geometric distribution of satellites, and the position accuracy factor, generate satellite signal quality characteristics that characterize signal reliability.

[0030] In some embodiments, this invention can parse the number of visible satellites, the signal-to-noise ratio of each satellite, and the carrier phase continuity index in real time from the raw observation data output by the GNSS module. Simultaneously, based on the satellite distribution configuration in the sky, satellite spatial geometric distribution parameters are calculated, and the position accuracy factor is solved accordingly. Combining these indicators, a satellite signal quality feature vector characterizing the reliability and geometric strength of the GNSS signal at the current moment is constructed to evaluate the availability and confidence of absolute positioning.

[0031] S1024. Based on visual sensor data, key points and local descriptors in each frame of the image are extracted through a convolutional neural network, and the quantity and spatial distribution uniformity are statistically analyzed to obtain visual image texture features.

[0032] In some embodiments, for each frame of image captured by the camera, the present invention can detect key points in the image and calculate their local descriptors using feature extraction algorithms such as SIFT or SURF. The number of key points extracted in a single frame is counted, and the spatial distribution uniformity of these key points on the image plane is evaluated. The number of key points and their distribution uniformity together constitute the visual image texture features, used to reflect the visual richness of the current scene and the potential success rate of feature matching.

[0033] S1025. Based on visual sensor data, significant linear structures and corner points in each frame of the image are identified through edge detection and line segment detection algorithms, and matched with known models to infer planes, edges, and geometric contours, generating geometric structure features.

[0034] In some embodiments, the present invention can perform edge detection and line segment detection on images to identify significant linear structures and corners. The identified geometric elements are then matched with a pre-loaded known inspection target model to infer planes, edges, and specific geometric contours in the scene. For example, in power line inspection, triangular or quadrilateral structures of poles are matched; in bridge inspection, linear edges of beams and columns are matched. Successfully matched geometric elements and their parameters constitute geometric structural features, providing constraints for model-based localization.

[0035] S1026. Based on inertial measurement data, the three-dimensional acceleration and three-dimensional angular velocity of the UAV in the body coordinate system are calculated, and the current instantaneous linear velocity and angular velocity are estimated by using a filtering algorithm to generate instantaneous motion state characteristics.

[0036] In some embodiments, the present invention can receive raw three-dimensional acceleration and three-dimensional angular velocity data from an IMU. Kalman filtering or complementary filtering algorithms are used to fuse and denoise the raw data, and the instantaneous linear velocity and angular velocity of the UAV in the body coordinate system are estimated in real time. This instantaneous motion state characteristic reflects the short-term, high-frequency dynamic changes of the UAV, providing kinematic constraints for the fusion algorithm.

[0037] S1027. Using a spatiotemporal synchronization mechanism, time alignment is performed on satellite signal quality characteristics, visual image texture characteristics, geometric structure characteristics, and instantaneous motion state characteristics to obtain time-aligned features.

[0038] In some embodiments, the present invention may employ hardware-triggered or software timestamp calibration methods based on GPS second pulses to ensure microsecond-level time synchronization of GNSS data, image frames, and IMU data acquisition times. Simultaneously, using the rotation matrix R and translation vector T between the camera and GNSS antenna, pre-obtained through hand-eye calibration, the features extracted by the visual sensor are uniformly transformed to the GNSS global coordinate system. Finally, the time-aligned and spatially unified satellite signal quality features, visual image texture features, geometric structure features, and real-time motion state features are combined into a structured multi-dimensional feature vector for subsequent environmental perception and fusion decision-making.

[0039] S1028. Based on time alignment features and combined with the initial pose value of the UAV, visual and geometric features are mapped to the global reference coordinate system to generate structured multi-dimensional features.

[0040] S103. Based on multi-dimensional features, dynamically generate a fusion positioning strategy that matches the current environment.

[0041] In this embodiment of the application, the fusion positioning strategy includes fusion weights among various types of data, a state estimation algorithm, and an interference source error compensation model.

[0042] As one possible implementation, step S103 can be specifically implemented as steps S1031-S1034.

[0043] S1031. Based on multi-dimensional features, a lightweight classification network is used to make real-time judgments to determine the current environmental pattern, environmental confidence level, and key influencing factor vectors.

[0044] In some embodiments, the current environment mode includes open mode, semi-occluded mode, fully occluded mode, or dynamic interference mode.

[0045] For example, in this embodiment of the invention, a structured multi-dimensional feature vector can be input into a lightweight environmental classification neural network for real-time inference, outputting the classification result of the current environmental mode and its corresponding environmental confidence score. The current environmental mode includes at least: open mode, partially occluded mode, fully occluded mode, and dynamic interference mode. Simultaneously, the network extracts a key influencing factor vector, which quantifies the state of key dimensions such as the mean GNSS signal-to-noise ratio, the number of visual feature points, the geometric structure matching degree, and the intensity of motion, to characterize the main sources of challenge in the current environment.

[0046] S1032. Dynamically determine the state estimation algorithm based on the current environment pattern and environment confidence.

[0047] In the open mode, a Kalman filter based on GNSS is used; in the semi-occluded mode, a vision-based particle filter and a loosely coupled GNSS filter are activated; in the fully occluded mode, a deep learning localization model based on CNN-LSTM is used; and in the dynamic interference mode, a dedicated anti-interference filtering algorithm is enabled.

[0048] For example, embodiments of the present invention may execute the following strategies based on the determined environmental pattern and confidence score:

[0049] Open mode: When the confidence level is higher than a preset threshold (e.g., 0.85), GNSS-dominated Kalman filtering is enabled. The system uses the tightly coupled GNSS / IMU solution results as the absolute observation, and visual data as auxiliary verification and smoothing constraints.

[0050] Partial Occlusion Mode: Activates vision-led particle filtering with loose coupling assistance from GNSS. In this mode, GNSS data is primarily used for updating particle weights and preventing visual odometry divergence, rather than as the primary observation.

[0051] Full Occlusion Mode: Switches to a deep learning fusion localization model based on CNN-LSTM. This model uses the aforementioned multi-dimensional features as input and directly predicts the location compensation value through a pre-trained network, maintaining localization even when GNSS signals are completely lost.

[0052] Dynamic interference mode: Enables a dedicated anti-interference filtering algorithm, which introduces adaptive robust estimation based on the innovation sequence into the standard Kalman filter framework to suppress outliers in GNSS data caused by multipath or electromagnetic interference.

[0053] S1033. Based on multi-dimensional features, automatically adjust the key parameters of the state estimation algorithm.

[0054] In some embodiments, key parameters include the process noise covariance Q and observation noise covariance R of the Kalman filter, the number of particles and resampling threshold of the particle filter, and the attention weights of the deep learning model.

[0055] For example, embodiments of the present invention can optimize the core parameters of the selected state estimation algorithm online based on real-time calculated multi-dimensional feature values ​​and key influencing factor vectors:

[0056] For Kalman filtering: the observation noise covariance matrix R is dynamically adjusted based on the GNSS signal-to-noise ratio and satellite geometry; the process noise covariance matrix Q is adaptively adjusted based on the motion state measured by the IMU.

[0057] For particle filtering: dynamically adjust the number of particles (e.g., reduce the number of particles when the features are abundant and increase the number of particles when the features are sparse) and the resampling threshold according to the number and distribution uniformity of visual feature points.

[0058] For deep learning models: A lightweight attention mechanism sub-network is used to dynamically adjust the fusion weights of the output features of the CNN and LSTM branches based on the characteristics of the current input features (such as image blur or GNSS signal discontinuity).

[0059] S1034. Based on the vector of key influencing factors, analyze and identify the current dominant interference source type, and determine the interference source error compensation model.

[0060] For example, step S1034 can be specifically implemented as steps A1-A4.

[0061] A1. Based on the vector of key influencing factors, analyze and identify the type of interference currently dominating.

[0062] In some embodiments, the types of interference sources include GNSS multipath effects, strong electromagnetic field interference, visual image motion blur, or sudden changes in illumination.

[0063] For example, embodiments of the present invention can perform real-time analysis of key influencing factor vectors. When the dimensions characterizing satellite signal quality in the vector (such as low average signal-to-noise ratio, frequent carrier phase jumps) show significant degradation, and do not decrease synchronously with the stability of visual features (number of feature points and matching rate), the dominant interference source is determined to be GNSS multipath effect or strong electromagnetic field interference; when the dimensions of visual feature quality (such as low image gradient amplitude, sharp reduction in the number of feature points) decrease significantly, while the satellite signal is relatively stable, further analysis is performed: if the motion state characteristics show that the UAV is in high-speed maneuvering, it is determined to be visual image motion blur; if the overall brightness or contrast of the image changes abruptly, it is determined to be illumination change.

[0064] A2. Based on the currently dominant interference source type, a compensation algorithm is obtained by matching from the pre-established error compensation model library.

[0065] In some embodiments, the error compensation model library includes a multipath error suppression model, an anti-electromagnetic interference filtering model, and an image deblurring and illumination invariance enhancement model.

[0066] For example, embodiments of the present invention can maintain a pre-established error compensation model library, and perform matching and invocation based on the interference source type identified in the previous step:

[0067] To address GNSS multipath effects: a multipath error suppression model is employed. This model uses an algorithm that weights signal-to-noise ratio (SNR) and satellite elevation angle to correct the raw GNSS pseudorange observations. Satellites with lower SNR and smaller elevation angles have their observation weights reduced accordingly, thereby suppressing the influence of multipath signals reflected from the ground or buildings.

[0068] For strong electromagnetic interference: an anti-electromagnetic interference filtering model is invoked. This model employs an IMU-assisted tightly coupled robust filtering architecture. When an abnormal increase in GNSS observation residuals is detected, the model automatically enhances the relevant terms in the process noise covariance Q and introduces an M estimator (such as the Huber loss function) to robustly process observation updates, reducing the impact of abnormal observations on state estimation.

[0069] For motion blur in visual images: an image deblurring and illumination invariance enhancement model is invoked. This model first detects blurred frames using a sharpness evaluation function (such as the Laplacian gradient), and then employs a non-uniform blind deconvolution algorithm based on inertial data for image restoration. The algorithm uses angular velocity information provided by the IMU to estimate the camera's point spread function (PSF) during exposure, and then performs deconvolution to recover image details.

[0070] For sudden changes in illumination: The illumination invariance enhancement module from the same model library is invoked. This module uses a combination of adaptive histogram equalization (CLAHE) and Retinex color constancy theory to preprocess the input image, enhance the details of shadow and highlight areas, and improve the robustness of feature extraction under different illumination conditions.

[0071] A3. Based on the satellite signal quality characteristics in the multi-dimensional features, the key parameters for the compensation algorithm are initialized.

[0072] For example, embodiments of the present invention can utilize multi-dimensional features at the current moment, particularly satellite signal quality features (such as the real-time signal-to-noise ratio and elevation angle of each satellite) and visual image texture features (such as the overall gradient statistics of the image), to dynamically initialize key operating parameters for the aforementioned invoked compensation algorithm. For instance, a signal-to-noise ratio threshold can be set for the weighting function in the multipath error suppression model; and an initial value for the blur kernel size based on IMU angular velocity estimation can be set for the image deblurring algorithm.

[0073] A4. Based on the compensation algorithm and key parameters, determine the error compensation model for the interference source.

[0074] For example, embodiments of the present invention can integrate the matched compensation algorithm, initialized key parameters, and raw data streams from the sensors to perform real-time error compensation calculations. This process generates a specific interference source error compensation model instance, which can output corrected or enhanced sensor data (such as corrected GNSS pseudoranges or deblurred image frames) and seamlessly integrate it into the main process of the fusion positioning strategy, ensuring that subsequent state estimation is based on cleaner and more reliable observation inputs.

[0075] S104. Based on visual sensor data, identify the current inspection stage and target inspection object of the UAV, and determine the prior task constraint model corresponding to the current inspection stage and target inspection object.

[0076] As one possible implementation, step S104 can be specifically implemented as steps S1041-S1044.

[0077] S1041. Perform semantic segmentation and instance segmentation on each frame of the image data from the visual sensor to extract the scene category and preliminarily detect potential inspection targets.

[0078] In some embodiments, the scene categories include farmland, tower clusters, or ruins, and potential inspection targets include insulators, cracks, or crops.

[0079] For example, embodiments of the present invention can process real-time image sequences acquired by a visual sensor in parallel using a semantic segmentation network and an instance segmentation network based on deep learning. The semantic segmentation network (such as DeepLabV3+) performs pixel-level classification of the entire frame image and outputs scene category labels (such as farmland, tower clusters, urban ruins, bridge surfaces). At the same time, the instance segmentation network (such as Mask R-CNN) identifies and segments independent entities in the image as potential inspection targets, such as insulators and bolts in power line inspections, cracks and leaks in bridge inspections, or diseased areas and crop plants in agricultural monitoring.

[0080] S1042. Based on visual sensor data, determine the current inspection stage of the drone through a temporal convolutional network.

[0081] In some embodiments, the current inspection phase includes approach, circling, linear cruising, or fixed-point detailed inspection.

[0082] For example, embodiments of the present invention can input visual features (such as optical flow fields and feature point trajectories) from multiple consecutive frames of images, as well as the real-time motion state features of the UAV (from the IMU), into a temporal convolutional network for analysis. This network learns typical flight patterns for different inspection tasks and outputs a judgment on the current inspection stage. The inspection stages include: the approach stage (high-speed approach to the target point, with rapidly increasing feature scale), the circling stage (circling the target, with a ring-shaped distribution of the optical flow field), the linear cruise stage (flying level along a preset route, with gradual feature changes), and the fixed-point detailed inspection stage (hovering or making slight movements at specific locations for multi-angle observation).

[0083] S1043. Based on scene categories and inspection stages, use graph attention networks to establish spatial, functional, and task logic relationships between potential inspection targets and deduce the target inspection objects.

[0084] For example, embodiments of the present invention can input the identified scene category, the set of potential inspection targets, and the current inspection stage into a graph attention network. This network constructs a graph structure with potential targets as nodes and their spatial adjacency, functional association, and task logic relationships as edges. For instance, in a power pole scenario, the identified insulator nodes and crossarm nodes are spatially connected and logically associated with the insulator self-explosion detection in the task plan. The GAT (Graph Attention Network) aggregates the information of nodes and edges through an attention mechanism to deduce the most likely specific target inspection object (such as the second insulator of phase C on tower #123) to be focused on in the current stage and scenario.

[0085] S1044. Based on the current inspection stage and the target inspection object, and combined with the satellite positioning point type, IMU motion mode and mission plan, perform multi-source verification to determine the current inspection stage and target inspection object that are consistent with the verification.

[0086] For example, embodiments of the present invention can perform multi-source verification between the current inspection stage and the target inspection object obtained from the above visual perception and reasoning and information from other information sources:

[0087] Satellite positioning point type verification: The real-time GNSS position of the UAV is compared with the key waypoint types (such as tower center point, photo point) in the preset route to confirm whether it conforms to the mission plan.

[0088] IMU Motion Pattern Verification: The consistency between the phase determined by the TCN and the motion pattern directly measured by the IMU (such as acceleration and angular velocity patterns) is checked. If they are consistent, the IMU motion pattern verification passes.

[0089] Mission plan logic verification: Compare with the pre-loaded mission sequence plan in the flight management system to ensure that the identified target object conforms to the current mission logic.

[0090] If the satellite positioning point type verification conforms to the mission plan, the IMU motion mode verification passes, and the mission plan logic verification conforms to the current mission logic, then the final confirmed current inspection stage and target inspection object are output. If any verification fails, a confidence assessment and error correction mechanism is triggered, or the system reverts to a more conservative identification state.

[0091] As one possible implementation, step S104 can also be implemented as steps S1045-S1048.

[0092] S1045. Based on the current inspection stage and the target inspection object, retrieve the matching set of constraint rules from the pre-built task-scenario-constraint knowledge graph.

[0093] In some embodiments, the set of constraint rules includes upper limits for positioning accuracy, trajectory smoothness requirements, safe obstacle avoidance distance, observation point dwell time, and data acquisition integrity indicators.

[0094] For example, embodiments of the present invention can query a pre-built task-scenario-constraint knowledge graph based on the current inspection stage (e.g., approach, surround, fixed-point detailed inspection) and the target inspection object (e.g., C-phase insulator of tower #123, main beam crack). This graph stores the core operational requirements of different inspection tasks (e.g., power line inspection, bridge inspection, agricultural monitoring) in different scenarios and environments in a structured form. The query returns a set of constraint rules strongly associated with the task stage and the target object. This set includes, but is not limited to: upper limit of positioning accuracy (e.g., power bolt inspection requirement ≤ 0.5 meters, bridge crack positioning requirement ≤ 0.3 meters), trajectory smoothness requirement (expressed as the maximum allowable angular acceleration or centripetal acceleration), safe obstacle avoidance distance (minimum three-dimensional interval with the target or obstacle), observation point dwell time (shortest hovering time to ensure image acquisition quality), and data acquisition integrity indicators (e.g., angle requirements for target coverage, image overlap rate).

[0095] S1046. Combining multi-dimensional features, dynamically adjust the parameters of the constraint rule set to obtain the adjusted constraint rule set.

[0096] The upper limit of positioning accuracy is set according to the criticality of the target object, and the trajectory smoothness coefficient is optimized online based on the current wind speed and flight speed.

[0097] For example, embodiments of the present invention can combine the set of constraint rules with currently calculated multi-dimensional features (especially environmental patterns and motion state features) to perform online, adaptive adjustments to the constraint parameters:

[0098] Dynamic grading of positioning accuracy upper limit: The grading is set according to the criticality of the target object (such as main load-bearing components vs. auxiliary facilities), and when the environment deteriorates (such as the GNSS signal-to-noise ratio decreases and visual features are reduced), the accuracy requirements of non-critical targets are relaxed according to the preset degradation strategy to maintain system stability.

[0099] Online optimization of trajectory smoothness coefficient: Based on the current real-time estimated wind speed (which can be deduced from the airspeed indicator or state estimation) and flight speed, the trajectory smoothness coefficient in the control algorithm is dynamically adjusted. For example, under high wind speeds, the smoothness requirement can be appropriately relaxed to prioritize positioning and obstacle avoidance safety.

[0100] Adaptive scaling of safe distance: During the close-up fine observation phase, the distance is dynamically calculated and reduced to the optimal observation distance based on the target's geometry and the camera's field of view, while activating the vision-based close-range collision avoidance module.

[0101] S1047. Based on the adjusted set of constraint rules, perform a consistency check. If there is a constraint conflict, automatically redistribute the weights according to the predefined task priorities and security criteria to generate a feasible constraint set.

[0102] For example, embodiments of the present invention can perform consistency checks on all dynamically adjusted constraint rules. When a constraint conflict is detected (e.g., under strong wind conditions, the hovering time required to meet high-precision positioning conflicts with the minimum flight speed required to meet trajectory smoothness), the system automatically reallocates constraint weights or relaxes some constraints based on predefined task priority rules (usually safety > data quality > efficiency) and global safety criteria, generating a set of executable feasible constraints that do not contain mutual exclusions.

[0103] S1048. Based on the feasible constraint set and the preset data model, it is integrated into the state estimation process to obtain the prior task constraint model.

[0104] In some embodiments, the data model includes a set of inequality constraints, Lyapunov functions, or factor graph nodes.

[0105] For example, embodiments of the present invention can transform a set of feasible constraints into a form that can be integrated into a state estimation algorithm through mathematical modeling, forming a prior task constraint model. Specifically, this model will be transformed into:

[0106] Inequality constraint set: For example, transforming safe obstacle avoidance distance into inequality constraints in state space (location).

[0107] Lyapunov functional constraints: used to ensure the stability and convergence of trajectory tracking at the control layer.

[0108] Nodes and edges in the factor graph model: In the backend optimization, positioning accuracy requirements and closed-loop detection conditions are transformed into prior factors or constraint edges in the factor graph. These factors, along with GNSS observation factors, visual reprojection factors, and IMU pre-integration factors, participate in the optimization, thereby forcibly satisfying task constraints in the global trajectory. This model is injected into the fusion calculation process in real time, guiding the system to output the optimal state estimate that satisfies both high-precision positioning requirements and conforms to the specific inspection task procedures.

[0109] S105. Based on the fusion positioning strategy, and using the prior task constraint model as the optimization constraint, satellite navigation data and visual sensor data are fused and calculated to generate real-time positioning information of the UAV.

[0110] As one possible implementation, step S105 can be specifically implemented as steps S1051-S1054.

[0111] S1051. According to the fusion positioning strategy, configure and start the corresponding data fusion calculation pipeline to complete the timestamp alignment and preprocessing of multi-source data to obtain fused data.

[0112] For example, embodiments of the present invention can dynamically configure a tightly coupled data fusion computation pipeline based on a fusion positioning strategy (including a selected state estimation algorithm, data weights, and an error compensation model). This pipeline first performs timestamp alignment of multi-source data, using hardware PPS signals or software interpolation methods to ensure that the raw GNSS observations, preprocessed image features, and IMU data are strictly synchronized to a unified system time base. Simultaneously, it invokes an interference source error compensation model to perform real-time compensation and enhancement on the input data, resulting in high-quality fused data.

[0113] S1052. Based on the fused data and combined with the state estimation algorithm, optimization iteration is performed, and the prior task constraint model is injected as the optimization constraint to solve for the optimal state estimate of the UAV that satisfies the constraints.

[0114] For example, embodiments of the present invention can use the aforementioned fused data as input to initiate the core optimization iteration process of a state estimation algorithm (such as Kalman filtering, particle filtering, or variants thereof). During this process, prior task constraint models (such as sets of inequality constraints, Lyapunov functions, or factor graph prior factors) are formally injected into the objective function or update step of the state estimation as hard constraints or weighted soft constraints. By solving this constrained optimization problem, an optimal state estimate of the UAV that satisfies all task and safety constraints is output in real time. This estimate contains high-precision three-dimensional position, velocity, and attitude information.

[0115] S1053. Based on the optimal state estimation of the UAV, evaluate the credibility of each sensor and the fusion result.

[0116] For example, an embodiment of the present invention can run a confidence assessment module in parallel. This module calculates a dynamic confidence score for the current GNSS observation, visual observation, and final fusion result based on innovation sequence analysis (for filtering algorithms), particle weight distribution entropy (for particle filtering), or confidence scores from neural network outputs (for deep learning models), combined with real-time multi-dimensional features (such as GNSS signal-to-noise ratio and visual feature matching inlier rate). This score quantifies the reliability of each information source in the current location estimation.

[0117] S1054. Based on the confidence level, perform local relocation and trajectory correction until the confidence level reaches the confidence threshold to obtain the real-time positioning information of the UAV.

[0118] For example, embodiments of the present invention can continuously monitor the aforementioned confidence score. When the confidence score of the fusion result is lower than a preset confidence threshold for the current environment mode (e.g., >0.9 for open scenes, >0.7 for urban blocks), the system triggers a local relocation and correction mechanism:

[0119] Local relocalization: The keyframe relocalization technology or visual loop closure detection function of the vision module is invoked to match the historical feature map and quickly correct pose jumps caused by short-term interference.

[0120] Trajectory Correction: The state estimate for the current low-confidence period is marked as needing optimization, and its observation data is cached. When computational resources permit, a lightweight local sliding window optimization is immediately initiated, fusing multi-frame observation data and constraints within the window to smooth the trajectory segment.

[0121] This invention iteratively executes the aforementioned evaluation-correction loop until the reliability of the output positioning information stably reaches or exceeds a preset threshold, ultimately outputting stable and reliable real-time positioning information for the UAV. This information will be directly used for UAV flight control, geocoding of inspection targets (embedded in image EXIF), and real-time trajectory display.

[0122] This invention provides a precise positioning method for UAV inspections that integrates vision and satellite navigation. It jointly extracts multi-dimensional features, such as satellite signal quality characteristics reflecting the reliability of space-air information and visual geometric textures reflecting ground environmental characteristics, to deeply perceive the environmental state. Then, based on real-time dynamic generation of a fusion strategy adapted to the current environment, it intelligently adjusts data weights and algorithm models, overcoming the limitations of traditional fixed-parameter fusion modes. Through visual recognition, it understands the specific inspection stage and target object of the UAV and loads corresponding prior task constraint models, ensuring that the positioning process closely matches the accuracy and safety requirements of actual operations. Finally, under the joint guidance of dynamic strategies and task constraints, fusion calculations are performed to solve the problem of inaccurate UAV positioning in complex environments, improving the environmental adaptability, robustness, and task-oriented accuracy of the positioning system.

[0123] Optionally, the UAV inspection precision positioning method integrating vision and satellite navigation provided in this embodiment of the invention further includes steps S201-S204.

[0124] S201. During the takeoff initialization phase, a visual feature map is constructed and bound to the absolute coordinates of the satellite as a global positioning reference.

[0125] For example, after the drone takes off, it enters the initialization phase. This phase executes two core tasks in parallel to build a global baseline:

[0126] GNSS Absolute Reference Establishment: The GNSS module continuously acquires satellite signals until it stably tracks at least 4 or more visible satellites with a signal strength > 45 dBHz, and calculates reliable global absolute coordinates as the space anchor point for the entire mission.

[0127] Visual Feature Map Construction: Visual sensors acquire multi-angle data about the environment around the takeoff point. Image features are extracted and matched using SIFT or SURF algorithms to construct a local visual feature map containing over 1000 stable feature points. Subsequently, through a spatial synchronization mechanism, this visual feature map is precisely bound to the absolute coordinates calculated by GNSS, forming a feature-coordinate-associated global positioning reference. This process typically completes within 30 seconds and must ensure a feature point matching success rate >90% to provide a reference for subsequent cruise positioning.

[0128] S202. During the cruise phase, a tightly coupled fusion mechanism is adopted, using satellite navigation data and inertial measurement data as absolute references and visual sensor data as relative motion constraints to perform UAV positioning.

[0129] For example, after entering the cruise phase, a tightly coupled fusion mechanism is used for real-time positioning:

[0130] Dominant fusion: The output of GNSS / IMU integrated navigation is used as the absolute pose reference (providing an update of about 10 Hz), while the absolute reference is corrected and smoothed by relative motion constraints generated by visual SLAM through inter-frame feature matching (processing 200-300 feature points per frame) (providing an update of about 50 Hz).

[0131] Mode switching: Real-time monitoring of GNSS signal status. When the number of visible GNSS satellites drops to at least 4, it is determined that a weak signal or obstructed environment has been entered, and the fusion algorithm automatically switches from GNSS-dominated to vision-dominated mode. In this mode, relying on keyframe relocalization technology, the current visual observation is matched with the feature map built during the initialization phase or cruise to maintain the continuity of positioning, ensuring that the pose update frequency is not less than 20 Hz.

[0132] S203. When the satellite signal is weakened or interrupted, switch to vision-dominated mode and maintain the continuity of UAV positioning through keyframe repositioning.

[0133] S204. At the end of the task, the backend graph optimization algorithm is activated to integrate global observation data to correct trajectory errors and achieve full-process adaptive positioning.

[0134] For example, after the inspection task is completed or at a critical task node, the trajectory optimization phase begins, and the backend graph optimization algorithm is activated to correct the cumulative error throughout the process:

[0135] Factor graph construction: Construct a factor graph model that includes GNSS observation factors, visual reprojection error factors, IMU pre-integration factors, and task constraint prior factors.

[0136] Global optimization solution: The g2o (general graph optimization) framework is used to solve and optimize the sparse matrix of the above factor graph, minimizing the residuals of all constraints. This process can effectively utilize global information (such as loop closure detection) to make overall adjustments to the pose sequence during the cruise phase.

[0137] Error Correction: This optimization systematically corrects the sub-meter cumulative error generated during the cruise phase to the centimeter level. For example, in actual testing, the root mean square error (RMSE) of a 10-kilometer inspection trajectory can be reduced from 0.8 meters before optimization to 0.08 meters after optimization, achieving a high-precision positioning closed loop with full-process adaptive capability.

[0138] Thus, this invention, through a closed-loop design encompassing initial mapping, cruise fusion, and backend optimization, ensures high-precision and robust positioning of the UAV throughout the entire inspection process. This enables the system to maintain stable and continuous positioning capabilities in complex environments such as open spaces and obstructed areas, and systematically corrects the final trajectory accumulation error to the centimeter level, effectively meeting the stringent requirements of global consistency and reliability for high-precision inspection operations in areas such as power lines and bridges.

[0139] Optionally, the UAV inspection precision positioning method integrating vision and satellite navigation provided in this embodiment of the invention further includes steps S301-S303.

[0140] S301. When multiple drones perform collaborative inspection tasks, each drone is independently located to obtain its location information.

[0141] For example, in a collaborative inspection mission, each UAV can independently run the fusion positioning method. Each UAV utilizes its own GNSS module, visual sensor, and IMU to calculate and output its independent three-dimensional position, velocity, and attitude information in real time through a tightly coupled fusion mechanism of "absolute coordinate anchoring - relative motion correction," which serves as the basis for subsequent collaborative optimization.

[0142] S302. Based on satellite time reference and the relative positional relationships between each UAV, construct UAV swarm collaborative positioning constraints.

[0143] For example, all participating UAVs achieve microsecond-level time synchronization based on UTC time and pulse-of-seconds (PPS) signals provided by GNSS, ensuring that all observation data have a unified time reference. Using inter-UAV communication links, each UAV periodically broadcasts its own positioning status and uncertainty. Based on this, the system constructs a cooperative positioning constraint network, where constraints mainly include two types: first, absolute constraints based on shared satellite observation (each UAV observes a common satellite, and their clock errors and atmospheric errors are correlated); second, inter-UAV relative position and attitude constraints obtained based on relative measurement techniques such as vision or ultra-wideband (UWB), these measurements are added to the network as strong constraint edges.

[0144] S303. Based on the cooperative positioning constraints of the UAV swarm, cross-validate and jointly optimize the positioning information of each UAV to determine the verified positioning information of each UAV.

[0145] For example, a central node (such as a ground station or a lead drone) or a distributed algorithm can perform centralized or distributed joint optimization of the cooperative positioning constraint network. The optimization process uses the independent positioning results of each drone as initial values, with inter-drone relative observation constraints and redundant observations from shared satellites as strong constraints. Cross-validation is performed: outliers caused by local interference (such as single-drone multipath effects) from individual drones are checked and eliminated; joint state estimation is achieved by minimizing the overall residual between the states and observations of all drones. This process not only significantly improves the absolute positioning accuracy and robustness of each drone (especially those in obstructed areas), but also ensures that the entire fleet has a highly consistent relative positioning relationship in the global coordinate system, providing a solid foundation for accurate cooperative track maintenance, data association, and task allocation.

[0146] Thus, this invention establishes a multi-drone collaborative positioning constraint network and utilizes satellite common-view and inter-drone relative measurements to jointly optimize and cross-validate the independent positioning results of each UAV. This significantly improves the overall positioning accuracy and consistency of UAV swarms in complex environments, enhances the positioning robustness of individual UAVs in signal-blocked areas, and provides a reliable technical foundation for achieving high-precision formation, collaborative data acquisition, and task allocation.

[0147] Furthermore, the UAV inspection precision positioning method provided by this invention has a workflow divided into three core stages, with each stage dynamically optimizing positioning accuracy through data closure. In the takeoff initialization stage, the system simultaneously completes dual-source data preparation: the GNSS module captures signals from at least four visible satellites to establish an absolute coordinate reference, while the visual sensor acquires ground feature images and constructs a local feature library containing over 1000 feature points using the SIFT / SURF algorithm, providing a reference benchmark for subsequent visual matching. This stage takes approximately 30 seconds and requires ensuring a GNSS signal strength > 45 dBHz and a feature point matching pass rate > 90%, laying the foundation for the cruise phase.

[0148] During the real-time positioning phase of the cruise phase, the system employs a tightly coupled fusion strategy: the GNSS / IMU combination provides 10Hz absolute pose updates, while visual SLAM achieves 50Hz relative motion constraints through inter-frame feature matching (processing 200-300 feature points per frame). When the number of visible GNSS satellites is less than 4, the fusion algorithm automatically switches to a vision-dominated mode, maintaining positioning continuity through keyframe relocalization technology (time complexity O(n), where n is the number of feature points), ensuring a pose update frequency of no less than 20Hz. In complex scenarios such as high-voltage tower clusters, the vision module can control the relative positioning error within 0.5 meters through geometric feature matching of the tower structure.

[0149] At the end of the mission, the trajectory optimization phase utilizes a backend graph optimization algorithm to correct global errors. The system constructs a factor graph model incorporating GNSS observations, visual reprojection errors, and IMU pre-integration constraints. A sparse matrix solution is then performed using the g2o framework, reducing the accumulated error from sub-meter to centimeter levels. Actual testing shows that after optimization, the root mean square error (RMSE) of the 10km inspection trajectory decreased from 0.8 meters to 0.08 meters, and the trajectory closure accuracy improved by 90%.

[0150] The integrated positioning technology in this invention exhibits significant advantages in anti-interference capability, positioning accuracy, and environmental adaptability. Through multi-source data redundancy verification, the system reduces GNSS multipath error by more than 60%. In high-voltage transmission line inspections, even with strong electromagnetic interference of level 5 or higher, the positioning error can still be controlled within 0.3 meters, while the error of traditional GNSS solutions can expand to 2.5 meters. The planar positioning accuracy is improved from sub-meter level (0.8-1.2 meters) of conventional positioning to centimeter level (0.05-0.15 meters), and the elevation direction accuracy reaches 0.2 meters, meeting the positioning requirements for identifying tower bolt-level defects in power line inspections.

[0151] Environmental adaptability tests show that the system can operate stably in various extreme scenarios: maintaining 0.5-meter positioning accuracy within 30 seconds of GNSS lock-off; maintaining a feature matching success rate of >85% within a light intensity range of 500-100,000 lux (from dawn to noon); and exhibiting no performance degradation of the hardware modules at operating temperatures ranging from -20℃ to 60℃. Field measurements during power line inspections in mountainous areas show that the fusion positioning success rate reaches 98.5%, a 36.8% improvement over conventional positioning solutions (72%), with a 40% increase in effective data acquisition per mission, significantly reducing the need for re-flight.

[0152] The positioning device provided by this invention adopts a modular design, with its core consisting of a sensor integration module and an embedded processing unit. The sensor module employs a tightly coupled GNSS and IMU architecture. The GNSS chip is a domestically produced chip supporting BeiDou-3 B1I / B2a dual-frequency signals, with a cold start positioning time of <25 seconds and a warm start time of <1 second. The IMU uses a six-axis (3-axis accelerometer + 3-axis gyroscope) MEMS sensor with a zero-bias stability of ≤0.1° / h and a sampling rate of 200Hz. Both sensors achieve hardware time synchronization via an SPI interface, with a synchronization error of <1ms, providing high-precision raw data for the tightly coupled algorithm.

[0153] The embedded processing unit adopts an FPGA+ARM heterogeneous computing platform: the FPGA is responsible for parallel acceleration of sensor data preprocessing (such as GNSS pseudorange calculation and IMU noise filtering) and visual feature extraction, and can process 50 frames of 1920×1080 images per second for feature point detection; the ARM Cortex-A53 quad-core processor runs a Linux system and carries out fusion algorithm and trajectory optimization tasks. The computing power allocation ratio is 60% for FPGA (responsible for computationally intensive tasks) and 40% for ARM (responsible for logic control and data interaction). This hardware architecture consumes less than 15W of power and weighs less than 200 grams, which can meet the payload requirements of small UAVs.

[0154] The precise positioning method for UAV inspection provided by this invention can be applied to intelligent inspection of power facilities. The application of integrated positioning technology in the field of power inspection significantly improves inspection accuracy and efficiency. The core requirements of power inspection are reflected in two aspects: for defects such as missing bolts and spontaneously exploding insulators on transmission towers (e.g., cat-head towers, goblet towers), precise positioning to specific locations is required (e.g., the lower phase insulator on the small side of tower #123 on a 220kV line), with a positioning error requirement of less than 0.5 meters; while the detection of hot spots on substation equipment (e.g., transformers, circuit breakers) requires associating with equipment coordinates, with an accuracy requirement controlled within 1 meter.

[0155] In terms of implementation, the system imports tower coordinates from the power GIS system into flight path planning software to generate a circumferential inspection flight path including GNSS anchor points, with 3-5 observation points set for each tower. When the UAV flies along the preset flight path, the fusion positioning system outputs pose data in real time, simultaneously triggering the camera to take pictures and embedding the positioning coordinates into the image's EXIF ​​information. The backend uses image recognition algorithms such as YOLOv8 to detect defects, ultimately generating a "defect location-image-coordinates" correlated report.

[0156] The UAV inspection precision positioning method provided by this invention can be applied to agricultural production monitoring and management. Fusion positioning technology demonstrates significant advantages in agricultural production monitoring and management, its core value lying in solving the key problem of "inaccurate positioning leading to zoning errors" in traditional inspections. The unique characteristics of agricultural scenarios pose severe challenges to positioning technology: during the crop growth cycle, tall crops such as corn and sugarcane significantly block GNSS signals after the jointing stage, resulting in a 40% reduction in the number of visible satellites; simultaneously, the undulating terrain of farmland (slope <15°) also interferes with relative height measurements. To address these issues, fusion positioning technology employs an innovative scheme of "visual ground feature matching + sparse GNSS anchor points," successfully maintaining positioning accuracy at the 0.5-1 meter level, laying a solid foundation for precision agricultural monitoring.

[0157] At the practical application level, fusion positioning technology has proven effective in monitoring various crops. Taking wheat stripe rust as an example, a drone equipped with a 5-band multispectral camera conducts a zigzag flight inspection. The fusion positioning system records coordinate and spectral data every 2 seconds, accurately identifies diseased areas through NDVI (Normalized Difference Vegetation Index) threshold segmentation, and generates a pest and disease distribution map based on the positioning data. The positioning error of the area boundary can be controlled within 0.8 meters. This accurate monitoring result provides strong guidance for precision pesticide application, reducing pesticide usage by 20% in practical applications.

[0158] In assessing rice growth during the tillering stage, integrated positioning technology also played a crucial role. It ensured stable flight of drones at a fixed 3-meter row spacing, and the collected images were stitched together to generate a digital surface model (DSM) of the field, thereby accurately calculating plant height distribution with an error controlled within 3 centimeters. Combined with yield models, the accuracy of yield prediction per acre improved by 15%, providing a scientific basis for rice production management.

[0159] The UAV inspection precision positioning method provided by this invention can be applied to disaster emergency rescue and assessment. In the field of disaster emergency rescue and assessment, UAV positioning technology that integrates vision and satellite navigation demonstrates irreplaceable value in extreme environments. When natural disasters occur, traditional GNSS positioning systems often fail due to base station damage or signal blockage. For example, urban building collapses caused by earthquakes can significantly exacerbate the "urban canyon effect," leading to a GNSS positioning interruption rate exceeding 50%. At this time, fusion positioning technology, with its independent operation capability of visual SLAM, becomes the core technical support for maintaining positioning continuity.

[0160] In earthquake rubble rescue scenarios, drones equipped with infrared cameras can quickly search for trapped personnel. Even in environments without GNSS signals, such as "tunnels" formed by collapsed buildings, the fusion positioning system can still maintain a positioning accuracy of 2 meters and transmit personnel location coordinates back to the command center in real time with a delay of less than 2 seconds. Compared with traditional manual search and rescue (which takes several hours per person), this technology improves positioning efficiency by more than 10 times and significantly shortens the time for life detection.

[0161] For emergency assessment of flood-inundated areas, the integration of positioning technology with water level sensor data can accurately map water level changes, with positioning errors controlled within 0.5 meters. By generating dynamic change maps of the inundated area, it provides precise evacuation route planning data for the rescue command center. A flood rescue case in a certain river basin shows that after adopting this technology, the disaster assessment time was reduced from the traditional 3 days to 12 hours, significantly improving emergency response efficiency.

[0162] In forest fire prevention and control, the fusion positioning system can accurately calculate the fire spread rate (error <0.2m / s) by overlaying real-time drone trajectories with infrared images of the fire site. This data supports command departments in formulating scientific deployment plans for firefighting teams, reducing fire control time by an average of 25% and effectively minimizing disaster losses.

[0163] The continuous and stable operation of fusion positioning technology during the critical 72-hour window for disaster relief directly determines the success rate of rescue operations. Its advantages in positioning continuity and accuracy provide crucial technical support for emergency rescue in various disaster scenarios, becoming an important technological means to enhance disaster response capabilities.

[0164] The UAV inspection precision positioning method provided by this invention can be applied to the inspection of large-scale infrastructure projects. Fusion positioning technology demonstrates significant application value in the full life-cycle health monitoring of large-scale infrastructure projects, with its core advantage being its ability to meet the stringent positioning accuracy requirements of infrastructure inspection. Bridge crack detection requires precise positioning down to specific beam segments, with lateral errors controlled within 0.3 meters; tunnel deformation monitoring requires sub-millimeter relative accuracy. To meet these high-precision requirements, fusion positioning technology adopts an innovative combination of absolute control point positioning and relative positioning of the inspection trajectory: GNSS control points (with static positioning accuracy reaching 0.1 mm) are preset at key parts of the bridge; during UAV inspection, the trajectory is calibrated in real time through visual recognition of the control points, achieving a relative positioning accuracy of 0.5 mm, fully meeting the technical indicators for deformation monitoring.

[0165] In specific applications, a fusion positioning drone equipped with a 20-megapixel high-resolution camera can simultaneously record the location (x, y coordinates) and width of cracks (measurement accuracy 0.01mm). Compared to traditional manual inspection (using suspended platforms, a single inspection takes 3 days), this technology increases inspection efficiency by 20 times while completely eliminating the risks of working at heights. In the field of subway tunnel monitoring, fusion positioning technology can achieve precise measurement of the three-dimensional coordinates of the tunnel's central axis (with a measuring point set every 5 meters), and by comparing data from multiple periods, it can capture settlement or convergence changes at the 0.1mm level.

[0166] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0167] Figure 2 This diagram illustrates the structure of a UAV inspection precision positioning device integrating vision and satellite navigation, according to an embodiment of the present invention. The positioning device 400 includes a communication module 401 and a processing module 402.

[0168] The communication module 401 is used to acquire satellite navigation data, visual sensor data and inertial measurement data of the UAV.

[0169] The processing module 402 is used to extract multi-dimensional features based on satellite navigation data, visual sensor data, and inertial measurement data. These multi-dimensional features include satellite signal quality features, visual image texture features, geometric structure features, and real-time motion state features. Based on these multi-dimensional features, it dynamically generates a fusion positioning strategy that matches the current environment. This fusion positioning strategy includes fusion weights among various data types, a state estimation algorithm, and an interference source error compensation model. Based on the visual sensor data, it identifies the current inspection stage and target inspection object of the UAV and determines the prior task constraint model corresponding to the current inspection stage and target inspection object. Based on the fusion positioning strategy, and using the prior task constraint model as an optimization constraint, it performs fusion calculations on the satellite navigation data and visual sensor data to generate real-time positioning information for the UAV.

[0170] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 500 includes: a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the computer program 503, it implements the steps in the above-described method embodiments. Alternatively, when the processor 501 executes the computer program 503, it implements the functions of each module / unit in the above-described device embodiments.

[0171] For example, the computer program 503 may be divided into one or more modules / units, which are stored in the memory 502 and executed by the processor 501 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 503 in the electronic device 500.

[0172] The processor 501 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0173] The memory 502 can be an internal storage unit of the electronic device 500, such as a hard disk or memory of the electronic device 500. The memory 502 can also be an external storage device of the electronic device 500, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 500. Furthermore, the memory 502 can include both internal and external storage units of the electronic device 500. The memory 502 is used to store the computer program and other programs and data required by the terminal. The memory 502 can also be used to temporarily store data that has been output or will be output.

[0174] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A precise positioning method for unmanned aerial vehicle (UAV) inspection integrating vision and satellite navigation, characterized in that, include: Acquire satellite navigation data, visual sensor data, and inertial measurement data from the drone; Based on satellite navigation data, visual sensor data, and inertial measurement data, multi-dimensional features are extracted, including satellite signal quality features, visual image texture features, geometric structure features, and real-time motion state features. Based on the aforementioned multi-dimensional features, a fusion positioning strategy that matches the current environment is dynamically generated. The fusion positioning strategy includes fusion weights among various types of data, a state estimation algorithm, and an interference source error compensation model. Based on the visual sensor data, the current inspection stage and target inspection object of the UAV are identified, and the prior task constraint model corresponding to the current inspection stage and target inspection object is determined. Based on the fusion positioning strategy, and using the prior task constraint model as the optimization constraint, the satellite navigation data and visual sensor data are fused and calculated to generate the real-time positioning information of the UAV. The method of dynamically generating a fusion positioning strategy that matches the current environment based on the multi-dimensional features includes: determining the current environment mode, environment confidence, and key influencing factor vectors in real time using a lightweight classification network based on the multi-dimensional features; the current environment mode includes open mode, partial occlusion mode, full occlusion mode, or dynamic interference mode; and dynamically determining the state estimation algorithm based on the current environment mode and environment confidence: wherein, in open mode, a GNSS-dominant Kalman filter is used; in partial occlusion mode, a vision-dominant particle filter with loosely coupled GNSS is activated; in full occlusion mode, a deep learning positioning model based on CNN-LSTM is used; and in dynamic interference mode, a dedicated anti-interference filtering algorithm is enabled; the dedicated anti-interference filtering algorithm introduces adaptive robust estimation based on innovation sequences into the standard Kalman filter framework to suppress interference caused by multipath or electromagnetic interference in GNSS data. The generated outliers are analyzed. Based on the multi-dimensional features, the key parameters of the state estimation algorithm are automatically adjusted. These key parameters include the process noise covariance Q and observation noise covariance R of the Kalman filter, the number of particles and resampling threshold of the particle filter, and the attention weights of the deep learning model. Based on the key influencing factor vector, the currently dominant interference source type is analyzed and identified. The interference source type includes GNSS multipath effect, strong electromagnetic field interference, visual image motion blur, or sudden illumination change. Based on the currently dominant interference source type, a compensation algorithm is matched from a pre-established error compensation model library. The error compensation model library includes a multipath error suppression model, an anti-electromagnetic interference filtering model, and an image deblurring and illumination invariance enhancement model. Based on the satellite signal quality features in the multi-dimensional features, the key parameters of the compensation algorithm are initialized. Based on the compensation algorithm and the key parameters, the interference source error compensation model is determined.

2. The method for precise positioning of unmanned aerial vehicle (UAV) inspection based on the fusion of vision and satellite navigation as described in claim 1, characterized in that, The extraction of multi-dimensional features based on satellite navigation data, visual sensor data, and inertial measurement data includes: The satellite navigation data is analyzed to calculate the number of visible satellites at the current moment, the signal-to-noise ratio of each satellite, the carrier phase continuity index, and the spatial geometric distribution of the satellites. Calculate the position accuracy factor based on the spatial geometric distribution of satellites; Based on the number of visible satellites at the current moment, the signal-to-noise ratio of each satellite, the carrier phase continuity index, the spatial geometric distribution of satellites, and the position accuracy factor, satellite signal quality characteristics that characterize signal reliability are generated. Based on the visual sensor data, key points and local descriptors in each frame of the image are extracted by a convolutional neural network, and the quantity and spatial distribution uniformity are statistically analyzed to obtain visual image texture features. Based on the visual sensor data, significant linear structures and corners in each frame of the image are identified by edge detection and line segment detection algorithms, and matched with known models to infer planes, edges and geometric contours, and generate geometric structure features. Based on the inertial measurement data of the UAV, the three-dimensional acceleration and three-dimensional angular velocity of the UAV in the body coordinate system are calculated, and the current instantaneous linear velocity and angular velocity are estimated by the filtering algorithm to generate instantaneous motion state characteristics. By using a spatiotemporal synchronization mechanism, satellite signal quality features, visual image texture features, geometric structure features, and instantaneous motion state features are time-aligned to obtain time-aligned features. Based on the aforementioned time alignment features, combined with the initial pose value of the UAV, visual and geometric features are mapped to the global reference coordinate system to generate structured multi-dimensional features.

3. The method for precise positioning of unmanned aerial vehicle (UAV) inspection based on the fusion of vision and satellite navigation as described in claim 1, characterized in that, The process of identifying the current inspection stage and target inspection object of the drone based on the visual sensor data includes: Semantic segmentation and instance segmentation are performed on each frame of the image data from the visual sensor to extract scene categories and preliminarily detect potential inspection targets. The scene categories include farmland, tower groups or ruins, and the potential inspection targets include insulators, cracks or crops. Based on the visual sensor data, the current inspection stage of the UAV is determined by a temporal convolutional network. The current inspection stage includes approach, circling, linear cruise, or fixed-point detailed inspection. Based on the aforementioned scenario categories and inspection stages, a graph attention network is used to establish spatial, functional, and task-related logical relationships between potential inspection targets, thereby deducing the target inspection objects. Based on the current inspection stage and the target inspection object, multi-source verification is performed in conjunction with satellite positioning point type, IMU motion mode and mission plan to determine the current inspection stage and target inspection object that are consistent with the verification.

4. The method for precise positioning of UAV inspection based on the fusion of vision and satellite navigation as described in claim 1, characterized in that, The method for determining the prior task constraint model corresponding to the current inspection stage and the target inspection object includes: Based on the current inspection stage and the target inspection object, a matching set of constraint rules is retrieved from the pre-constructed task-scenario-constraint knowledge graph; the set of constraint rules includes the upper limit of positioning accuracy, trajectory smoothness requirements, safe obstacle avoidance distance, observation point dwell time and data collection integrity indicators; Combining the aforementioned multi-dimensional features, the constraint rule set is dynamically adjusted to obtain an adjusted constraint rule set; wherein, the upper limit of positioning accuracy is set hierarchically according to the criticality of the target object, and the trajectory smoothness coefficient is optimized online based on the current wind speed and flight speed; Based on the adjusted set of constraint rules, a consistency check is performed. If there are constraint conflicts, the weights are automatically redistributed according to the predefined task priorities and security criteria to generate a feasible set of constraints. Based on the feasible constraint set and the preset data model, they are integrated into the state estimation process to obtain the prior task constraint model. The data model includes a set of inequality constraints, Lyapunov functions, or factor graph nodes.

5. The method for precise positioning of unmanned aerial vehicle (UAV) inspection based on the fusion of vision and satellite navigation as described in claim 1, characterized in that, The step of fusing satellite navigation data and visual sensor data based on the fusion positioning strategy, using the prior task constraint model as the optimization constraint, to generate real-time positioning information for the UAV includes: According to the fusion positioning strategy, configure and start the corresponding data fusion calculation pipeline to complete the timestamp alignment and preprocessing of multi-source data to obtain fused data; Based on the fused data, combined with the state estimation algorithm, optimization iteration is performed, and the prior task constraint model is injected as the optimization constraint to solve for the optimal state estimate of the UAV that satisfies the constraints. Based on the optimal state estimation of the UAV, the reliability of each sensor and the fusion result is evaluated; Based on the stated confidence level, local relocation and trajectory correction are performed until the confidence level reaches a confidence threshold, thereby obtaining the real-time positioning information of the UAV.

6. The method for precise positioning of unmanned aerial vehicle (UAV) inspection based on the fusion of vision and satellite navigation as described in claim 1, characterized in that, The method further includes: During the takeoff initialization phase, a visual feature map is constructed and bound to satellite absolute coordinates as a global positioning reference. During the cruise phase, a tightly coupled fusion mechanism is adopted, using satellite navigation data and inertial measurement data as absolute references and visual sensor data as relative motion constraints for UAV positioning; When satellite signals are weakened or interrupted, switch to vision-dominated mode and maintain the continuity of UAV positioning through keyframe repositioning; At the end of the mission, the backend graph optimization algorithm is activated to integrate global observation data for trajectory error correction, achieving full-process adaptive positioning.

7. The method for precise positioning of unmanned aerial vehicle (UAV) inspection based on the fusion of vision and satellite navigation as described in claim 1, characterized in that, The method further includes: When multiple drones perform collaborative inspection tasks, each drone is independently located to obtain its location information. Based on satellite time reference and the relative positional relationships among the drones, a collaborative positioning constraint for drone swarm is constructed. Based on the aforementioned drone swarm collaborative positioning constraints, the positioning information of each drone is cross-validated and jointly optimized to determine the verified positioning information of each drone.

8. A precise positioning device for unmanned aerial vehicle (UAV) inspection integrating vision and satellite navigation, characterized in that, include: The communication module is used to acquire satellite navigation data, visual sensor data, and inertial measurement data of the UAV. The processing module is used to extract multi-dimensional features based on satellite navigation data, visual sensor data, and inertial measurement data. The multi-dimensional features include satellite signal quality features, visual image texture features, geometric structure features, and real-time motion state features. Based on the multi-dimensional features, a fusion positioning strategy matching the current environment is dynamically generated. The fusion positioning strategy includes fusion weights among various types of data, a state estimation algorithm, and an interference source error compensation model. Based on the visual sensor data, the current inspection stage and target inspection object of the UAV are identified, and a prior task constraint model corresponding to the current inspection stage and target inspection object is determined. Based on the fusion positioning strategy, the satellite navigation data and visual sensor data are fused and calculated using the prior task constraint model as optimization constraints to generate real-time positioning information of the UAV. The processing module is specifically used to determine the current environment mode, environment confidence, and key influencing factor vectors in real time based on the multi-dimensional features using a lightweight classification network. The current environment mode includes open mode, partial occlusion mode, full occlusion mode, or dynamic interference mode. Based on the current environment mode and environment confidence, the module dynamically determines the state estimation algorithm: In open mode, a GNSS-dominated Kalman filter is used; in partial occlusion mode, a vision-dominated particle filter with loosely coupled GNSS is activated; in full occlusion mode, a CNN-LSTM-based deep learning localization model is used; and in dynamic interference mode, a dedicated anti-interference filtering algorithm is enabled. This anti-interference filtering algorithm, within the standard Kalman filter framework, introduces adaptive robust estimation based on the innovation sequence to suppress outliers in GNSS data caused by multipath or electromagnetic interference. The system automatically adjusts the key parameters of the state estimation algorithm based on dimensional features. These key parameters include the process noise covariance Q and observation noise covariance R of the Kalman filter, the number of particles and resampling threshold of the particle filter, and the attention weights of the deep learning model. Based on the key influencing factor vector, it analyzes and identifies the currently dominant interference source type, which includes GNSS multipath effects, strong electromagnetic field interference, visual image motion blur, or sudden illumination changes. Based on the currently dominant interference source type, it matches a compensation algorithm from a pre-established error compensation model library. This error compensation model library includes multipath error suppression models, anti-electromagnetic interference filtering models, and image deblurring and illumination invariance enhancement models. Based on the satellite signal quality features in the multidimensional features, it initializes the key parameters for the compensation algorithm. Based on the compensation algorithm and the key parameters, it determines the interference source error compensation model.

Images