A method for dynamic inspection of unmanned aerial vehicles in GNSS denial environment based on multi-source state estimation and local gradient
By employing multi-source state estimation and local gradient methods, the positioning instability and missed detection problems of UAV inspection systems in GNSS denied environments were solved, enabling UAVs to perform adaptive dynamic inspections in complex environments and improving positioning accuracy and safety.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- KUNMING UNIV OF SCI & TECH
- Filing Date
- 2026-06-24
- Publication Date
- 2026-07-21
AI Technical Summary
In complex environments where GNSS signals are missing or limited, UAV inspection systems are prone to unstable positioning due to the loss of features from a single sensor, making it impossible to adaptively adjust flight paths to approach and confirm new diseases, resulting in a high rate of missed detections and high costs for re-flight.
By employing a multi-source state estimation and local gradient method, a joint error cost function is constructed through tight coupling of laser, vision, and inertial sensors to optimize the UAV pose in real time. The trajectory is dynamically adjusted using local gradients, and a collision-free obstacle avoidance trajectory is generated by combining 3D laser point cloud and visual image back projection, thus achieving dynamic reconstruction and adaptive close-up observation.
It improves positioning accuracy and stability in GNSS denied environments, reduces the missed detection rate, enhances flight safety and inspection efficiency, and enables UAVs to perform adaptive inspections in complex environments.
Smart Images
Figure CN122431398A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a dynamic inspection method for unmanned aerial vehicles (UAVs) based on multi-source state estimation and local gradient in GNSS denied environments, and belongs to the field of UAV dynamic inspection technology. Background Technology
[0002] Drones have become a mainstream inspection tool due to their mobility. However, traditional drone inspections heavily rely on GNSS signals for route planning and positioning. Currently, for complex environments such as under bridges, tunnels, and deep mountain canyons where GNSS signals are severely blocked or even blocked, some studies have proposed drone inspection schemes based on pure vision or pure laser SLAM (Simultaneous Localization and Mapping), or utilizing airborne edge computing devices for data backhaul and analysis. For example, some patents disclose an inspection architecture based on GNSS signal-based RTK (Real-Time Kinematic) technology, which combines preset waypoints with cloud-based identification.
[0003] While the aforementioned solutions address the drone inspection problem to some extent, significant shortcomings remain: First, environments such as under-bridge structures, tunnels, and deep mountain canyons often involve severe weak textures, abrupt changes in lighting, and dynamic interference, making single-sensor solutions highly susceptible to feature loss, leading to drone malfunctions. Second, existing inspection trajectories are typically fixed, unable to be adjusted flexibly for newly discovered defects. Edge computing is only used for data filtering or post-processing, failing to achieve reverse control of the underlying flight path based on perception results. When a drone detects minor defects at the edge, it cannot dynamically change its original trajectory to approach and confirm, resulting in a high rate of missed detections and significant re-flight costs. Summary of the Invention
[0004] The purpose of this invention is to provide a dynamic inspection method for unmanned aerial vehicles (UAVs) based on multi-source state estimation and local gradient in GNSS denied environments, aiming to solve the following technical problems:
[0005] First, in scenarios where GNSS signals are missing or limited, and in low-texture conditions, drone inspection systems equipped with a single sensor are prone to losing environmental features, having their positioning easily diverge, and being easily damaged or out of control of the drone.
[0006] Second, traditional manual waypoints only contain three-dimensional position information and lack strict constraints on the observation attitude of UAVs, resulting in a high rate of missed detection of new or minor defects.
[0007] Third, existing UAV inspection trajectories are usually static and fixed, which cannot be adjusted flexibly to address newly discovered minor defects during inspections. They also lack the dynamic reconstruction capability to adaptively deviate from the flight path and make close-up confirmations, resulting in high re-flight costs.
[0008] To achieve the above objectives, the technical solution of this invention is: a dynamic inspection method for unmanned aerial vehicles (UAVs) based on multi-source state estimation and local gradient in GNSS denied environments, comprising the following steps:
[0009] Step 1: Acquire multi-source data of the spatial environment based on a time reference. Using the relative zero point of the system power-on or a preset physical node as a reference, construct a joint error cost function containing the residuals of the multi-source data. Then, use the error state to perform nonlinear optimization to continuously calculate the six-degree-of-freedom pose of the UAV and establish a local physical coordinate system. The multi-source data includes three-dimensional laser point clouds, visual images, and inertial measurement data.
[0010] Step 2: The UAV performs automated waypoint navigation within the local physical coordinate system and performs onboard real-time inference on the visual image to obtain the two-dimensional features of the disease. Combining the real-time depth information provided by the three-dimensional laser point cloud with the camera intrinsic parameter matrix, the two-dimensional features of the disease are reverse-projected, and the three-dimensional absolute spatial coordinates of the disease under the local physical coordinate system are obtained using the six degrees of freedom pose of the UAV.
[0011] Step 3: Using the three-dimensional absolute spatial coordinates as the gravity source, construct a gravity penalty term for the target disease, and use the spatial distance field constructed by the three-dimensional laser point cloud as a collision repulsion penalty term. Introduce the gravity penalty term and the collision repulsion penalty term into the comprehensive objective function of local trajectory optimization. Solve the local gradient by minimizing the comprehensive objective function, dynamically adjust the curve control points, and generate a collision-free continuous obstacle avoidance trajectory in real time from the current position of the UAV to the observation area of the disease target.
[0012] Step 4: Discretize the collision-free continuous obstacle avoidance trajectory into setpoint commands according to the set spatiotemporal sampling parameters, and send them to the UAV's built-in low-level flight control system. Drive the UAV to adaptively deviate from the original cruise route and perform close-range observation operations to get close to structural surface defects. At the same time, build a data stream isolation mechanism to complete the UAV's dynamic inspection.
[0013] Optionally, the joint error cost function is specifically:
[0014]
[0015] in, Let the joint error cost function be... Let be the system state vector. For the index variable of the feature points of the laser point cloud, The set of all valid laser point cloud feature points. For index variables of feature points in visual images, The set of all valid visual image feature points. The information matrix of the IMU pre-integrated residuals. The information matrix of the laser point cloud residual. The information matrix of visual residuals, For dynamic constraints based on inertial measurement data, Geometric constraints for matching laser point clouds with points and surfaces or points and lines. Constraints for visual reprojection photometric residuals.
[0016] Optionally, the automated waypoint navigation specifically includes:
[0017] Using a local physical coordinate system as a reference, multiple six-degree-of-freedom pose nodes containing three-dimensional translation coordinates and three-dimensional spatial attitude angles are pre-recorded to construct a local inspection waypoint map containing sensor observation line-of-sight constraints. The local inspection waypoint map is then optimized offline to obtain the optimal route, and the UAV performs automated flight according to the optimal route.
[0018] Optionally, the step of performing global waypoint offline optimization on the local inspection waypoint map to obtain the optimal route specifically involves:
[0019] The recorded six-degree-of-freedom pose nodes are transformed into a directed weighted graph model, and a comprehensive edge weight cost is constructed for offline solution. Redundant routes are eliminated, and a globally optimal topology sequence that traverses all preset observation nodes is generated. The globally optimal topology sequence is then used as the best route.
[0020] Optionally, obtaining the three-dimensional absolute spatial coordinates of the disease in the local physical coordinate system specifically involves:
[0021]
[0022] in, Three-dimensional absolute spatial coordinates, This is the external rotation matrix between the pre-calibrated camera coordinate system and the body coordinate system. This is the translation vector between the pre-calibrated camera coordinate system and the body coordinate system. For depth information, This is the intrinsic parameter matrix of the camera. The coordinates of the disease are two-dimensional pixels.
[0023] Optionally, the comprehensive objective function is specifically:
[0024]
[0025] in, For the comprehensive objective function, To measure the smoothness penalty term of the derivative of the control points of a B-spline curve, This is a penalty term for obstacle avoidance repulsion based on the Euclidean symbolic distance field. As a penalty item for the feasibility of aircraft dynamics, For the gravitational penalty term oriented towards the three-dimensional absolute spatial coordinates of the target disease, , , , These are the weighting coefficients for each penalty item.
[0026] Optionally, the data stream isolation mechanism specifically includes:
[0027] The air-to-ground communication link transmits only low-bandwidth UAV flight status and basic control data back to the ground, and asynchronously writes multi-source data and pose logs to the onboard physical storage medium.
[0028] The beneficial effects of this invention are:
[0029] 1. This invention employs a tightly coupled joint error optimization method combining laser, vision, and inertial sensors, which effectively improves the feature loss problem of a single vision sensor in environments with weak texture and sudden changes in lighting, ensuring positioning accuracy and stability in GNSS denied or restricted environments, and exhibiting strong positioning robustness.
[0030] 2. This invention breaks away from the traditional static preset route inspection mode and proposes a control closed loop from real-time reasoning to dynamic trajectory reconstruction, realizing dynamic and refined inspection. The UAV has the ability to automatically change its route for close-range confirmation after discovering suspected defects, effectively reducing the missed detection rate of minor defects and improving the efficiency of a single patrol flight.
[0031] 3. This invention adopts a separate data management architecture that combines basic status data transmission over air with high-dimensional data storage onboard, avoiding the risks of link congestion and system crashes caused by long-distance signal attenuation, significantly improving flight safety in special operating environments, and ensuring high system reliability. Attached Figure Description
[0032] Figure 1 The overall flowchart of the UAV dynamic inspection method based on multi-source state estimation and local gradient in GNSS denied environment provided in the embodiments of the present invention is shown below.
[0033] Figure 2 This is a diagram illustrating the overall data flow architecture of the multi-source state estimation and hardware system in this embodiment of the invention.
[0034] Figure 3 This is a schematic diagram of local trajectory dynamic reconstruction based on potential field constraints and curve control points in an embodiment of the present invention;
[0035] Figure 4 This is a schematic diagram illustrating the principle of reprojecting two-dimensional pixel features into three-dimensional absolute spatial coordinates in an embodiment of the present invention. Detailed Implementation
[0036] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.
[0037] Example 1: A dynamic inspection method for UAVs based on multi-source state estimation and local gradient in GNSS denied environments, comprising the following steps:
[0038] Step 1: Tightly Coupled State Estimation Based on Joint Error Optimization: Acquire multi-source data of the spatial environment aligned with a time reference. Using the relative zero point of the system power-on or a preset physical node as a reference, construct a joint error cost function containing the residuals of the multi-source data. Utilize the error state for nonlinear optimization to continuously calculate the six-degree-of-freedom pose of the UAV and establish a local physical coordinate system. The multi-source data includes 3D laser point clouds, visual images, and inertial measurement data.
[0039] Optionally, the joint error cost function includes the laser geometric observation residual, the visual feature point reprojection photometric residual, and the inertial sensor pre-integration residual, specifically:
[0040]
[0041] in, Let the joint error cost function be... Let be the system state vector. For the index variable of the feature points of the laser point cloud, The set of all valid laser point cloud feature points. For index variables of feature points in visual images, The set of all valid visual image feature points. The information matrix of the IMU pre-integrated residuals. The information matrix of the laser point cloud residual. The information matrix of visual residuals, For dynamic constraints based on inertial measurement data, Geometric constraints for matching laser point clouds with points and surfaces or points and lines. Constraints for visual reprojection photometric residuals.
[0042] Optionally, in this embodiment, the joint error cost function can be iteratively minimized using methods such as error state iterative Kalman filtering and factor graph optimization to update the system state vector. This allows for nonlinear optimization solutions.
[0043] Step 2: Inverse calculation of absolute spatial coordinates of the disease based on airborne real-time inference: The UAV performs automated waypoint navigation in the local physical coordinate system and performs airborne real-time inference on the visual image to obtain the two-dimensional features of the disease. Combined with the real-time depth information provided by the three-dimensional laser point cloud and the camera intrinsic parameter matrix, the two-dimensional features of the disease are reverse-projected. Using the six-degree-of-freedom pose of the UAV, the three-dimensional absolute spatial coordinates of the disease in the local physical coordinate system are obtained.
[0044] Optionally, the automated waypoint navigation specifically includes:
[0045] Using a local physical coordinate system as a reference, multiple six-degree-of-freedom pose nodes containing three-dimensional translation coordinates and three-dimensional spatial attitude angles are pre-recorded to construct a local inspection waypoint map containing sensor observation line-of-sight constraints. The local inspection waypoint map is then optimized offline to obtain the optimal route, and the UAV performs automated flight according to the optimal route.
[0046] Optionally, in this embodiment, global waypoint offline optimization is performed based on graph optimization theory or heuristic routing optimization algorithms;
[0047] Optionally, the step of performing global waypoint offline optimization on the local inspection waypoint map to obtain the optimal route specifically involves:
[0048] The recorded six-degree-of-freedom pose nodes are transformed into a directed weighted graph model, and a comprehensive edge weight cost is constructed for offline solution. Redundant routes are eliminated, and a globally optimal topology sequence that traverses all preset observation nodes is generated. The globally optimal topology sequence is then used as the best route.
[0049] Optionally, in this embodiment, one or more combinations of spatial Euclidean distance, aircraft dynamic energy consumption model, flight time, attitude change cost, and space safety obstacle avoidance penalty are used as the comprehensive edge weight cost, and the solution is obtained offline through graph optimization theory or heuristic routing optimization algorithm.
[0050] Optionally, obtaining the three-dimensional absolute spatial coordinates of the disease in the local physical coordinate system specifically involves:
[0051]
[0052] in, Three-dimensional absolute spatial coordinates, This is the external rotation matrix between the pre-calibrated camera coordinate system and the body coordinate system. This is the translation vector between the pre-calibrated camera coordinate system and the body coordinate system. For depth information, This is the intrinsic parameter matrix of the camera. The coordinates of the disease are two-dimensional pixels.
[0053] Step 3: Dynamic trajectory reconstruction based on local gradient: The three-dimensional absolute spatial coordinates are used as the gravity source to construct a gravity penalty term for the target disease, and the spatial distance field constructed by the three-dimensional laser point cloud is used as the collision repulsion penalty term. The gravity penalty term and the collision repulsion penalty term are introduced into the comprehensive objective function of local trajectory optimization. The local gradient is solved by minimizing the comprehensive objective function, the curve control points are dynamically adjusted, and a collision-free continuous obstacle avoidance trajectory is generated in real time from the current position of the UAV to the observation area of the disease target.
[0054] Optionally, the comprehensive objective function is specifically:
[0055]
[0056] in, For the comprehensive objective function, To measure the smoothness penalty term of the derivative of the control points of a B-spline curve, This is a penalty term for obstacle avoidance repulsion based on the Euclidean symbolic distance field. As a penalty item for the feasibility of aircraft dynamics, For the gravitational penalty term oriented towards the three-dimensional absolute spatial coordinates of the target disease, , , , These are the weighting coefficients for each penalty item.
[0057] It should be noted that traditional trajectory planning aims to avoid obstacles and reach the destination. However, this embodiment creatively converts the identified two-dimensional defects into three-dimensional coordinates, dynamically using them as a "gravity" force and embedding them into the traditional trajectory optimization function. This forces the UAV to automatically deviate from its original route to approach (gravity) the cracks without hitting a wall (repulsive force). This achieves the seamless conversion of visual recognition results into the underlying trajectory control gradient. Therefore, constructing a gravity penalty term for the target defect's three-dimensional absolute spatial coordinates is the core innovation of this embodiment.
[0058] Step 4: Low-level control mapping and data isolation closed loop: The collision-free continuous obstacle avoidance trajectory is discretized into setpoint commands according to the set spatiotemporal sampling parameters and sent to the UAV's built-in low-level flight control system. This drives the UAV to adaptively deviate from the original cruise route and perform close-range observation operations to observe defects on the structural surface. At the same time, a data flow isolation mechanism is constructed to complete the UAV's dynamic inspection.
[0059] Optionally, the data stream isolation mechanism is used to ensure the security of the UAV flight control bandwidth, specifically as follows:
[0060] The air-to-ground communication link transmits only low-bandwidth UAV flight status and basic control data back to the ground, while high-bandwidth multi-source data and high-precision pose logs are asynchronously written to the airborne physical storage medium to avoid congestion of the air-to-ground communication link and flight control bus caused by high-bandwidth sensing data.
[0061] It is understandable that the close-up observation operation proposed in this embodiment specifically involves controlling the UAV to change its original cruise attitude, reconstructing its flight trajectory based on the identified defects, and safely approaching the surface of the structure. This enables the camera to capture high-definition images of tiny cracks, avoiding missed detections. At the same time, the constructed data flow isolation mechanism avoids the problem that the serial bus bandwidth of the air-to-ground communication link is insufficient to support the transmission of massive amounts of image and radar point cloud data, which would lead to congestion of more important control commands and cause the UAV to crash into a wall and explode.
[0062] Understandably, this embodiment establishes a closed-loop process for UAV inspection in GNSS-denied environments through Steps 1-4, encompassing waypoint optimization with UAV pose, multi-dimensional perception, AI decision-making, trajectory reconstruction, and flight control execution. Specifically, Steps 2 and 3 convert the identified two-dimensional defects into three-dimensional coordinates, enabling the UAV to safely approach and observe the defects, achieving dynamic intelligent inspection. Furthermore, since traditional waypoint planning only includes (X, Y, Z) three-dimensional coordinates, it is prone to missed detections during re-inspections in complex GNSS-denied environments such as under bridges and tunnels. This embodiment uses the UAV's six-degree-of-freedom pose as waypoints for planning, employing graph optimization algorithms or heuristic routing optimization algorithms to not only plan "how to fly there" but also strictly constrain "what attitude to adopt for observation."
[0063] Example 2: Based on the technical solution provided in Example 1, this example also provides a UAV dynamic inspection system based on multi-source state estimation and local gradient in a GNSS denied environment, such as... Figure 2 As shown, the hardware architecture of this system mainly consists of a multi-source sensing unit, an airborne edge computing platform, an execution unit, and a storage unit.
[0064] Optionally, the multi-source sensing unit includes a three-dimensional lidar 101, a vision camera 102, and an inertial measurement unit 103. The three-dimensional lidar 101 can be a solid-state radar with non-repeating scanning characteristics to obtain a spatial point cloud with a large field of view. The vision camera 102 is a color RGB camera that acquires real-time image streams. Its depth information is mainly obtained by geometrically projecting the point cloud of the three-dimensional lidar 101 onto the RGB image plane, or it can be obtained by stereo matching of a binocular camera. The inertial measurement unit 103 is used to collect high-frequency three-dimensional acceleration and angular velocity.
[0065] Furthermore, in order to eliminate positioning drift caused by misalignment of data timestamps between multi-source heterogeneous sensors, this embodiment introduces a hardware-level time synchronization mechanism. The airborne computer 100 sends physical-level second pulse signals and frame synchronization signals to the three-dimensional lidar 101 and the vision camera 102 through its underlying pulse width modulation or general-purpose input / output pins, aligning the time reference of multi-source sensing data at the hardware physical level.
[0066] Furthermore, the airborne computer 100 serves as the core computing platform, internally deploying a joint error state estimation module 110, a defect coordinate reverse calculation module 120, and a trajectory dynamic reconstruction module 130. The airborne computer 100 is communicatively connected to the flight control system 140 and the airborne physical storage medium 150, respectively.
[0067] Furthermore, when the system is in an extreme environment such as complete darkness, which causes visual feature extraction to fail or lidar features to degrade, the joint error state estimation module can adaptively adjust the covariance weight of the corresponding sensor residuals and rely on the remaining effective sensors and IMU (Inertial Measurement Unit) pre-integration to maintain the pose output.
[0068] Optionally, the onboard computer used in this embodiment is not limited to a single physical motherboard, but can be a heterogeneous multi-core processor such as an edge computing platform that includes a CPU (Central Processing Unit), GPU (Graphics Processing Unit), and NPU (Neural Processing Unit). The onboard physical storage medium can be an eMMC (Embedded Multi Media Card) built into the motherboard, NVMe flash memory, or an external TF card or solid-state drive.
[0069] like Figure 1 As shown below, a specific example illustrates the application of the system provided in this embodiment in a UAV dynamic inspection method based on multi-source state estimation and local gradient in a GNSS denied environment.
[0070] Step 201: Perform tightly coupled state estimation based on joint error optimization. Addressing the issue of degradation in existing technologies where single lidar systems are prone to degradation in narrow passages such as long straight tunnels or bridge decks lacking bottom features, this embodiment simultaneously fuses radar point cloud residuals, visual photometric residuals, and inertial pre-integration residuals in the airborne computer 100, obtaining the joint error cost function constructed by the joint error state estimation module 110. The system can establish a local physical coordinate system with a specific direction (such as the northeast-sky direction) with the take-off point as the origin and output a six-degree-of-freedom continuous pose at the centimeter level by iteratively solving the joint error cost function through the error state iterative Kalman filter algorithm.
[0071] Step 202: During the cruise operation of the UAV within the local physical coordinate system, the onboard computer 100 uses a pre-set target network model suitable for deployment on an edge computing platform, such as the YOLO series or MobileNet structure, to perform onboard real-time inference on the real-time visual stream.
[0072] Step 203: Determine if target structural defects have been identified. When the target network model identifies defects such as surface cracks or peeling, subsequent steps are triggered.
[0073] Step 204: Perform the reverse calculation of the absolute spatial coordinates of the disease. For example... Figure 4 As shown, the two-dimensional feature location information 401 of the disease in the image, including pixel coordinates or region contour boundaries, is extracted and denoted as... To overcome the limitation of traditional vision models that can only output two-dimensional bounding boxes, the system uses the camera's projection center as a reference and leverages depth information Z and the camera's intrinsic parameter matrix K to inversely calculate two-dimensional pixel coordinates into three-dimensional physical space coordinates. .
[0074] Step 205: Construct the comprehensive objective function for trajectory optimization. For example... Figure 3 As shown, the trajectory dynamic reconstruction module 130 uses the calculated three-dimensional absolute spatial coordinates as the gravity source to construct a gravity penalty term 302 for the target disease, and uses the Euclidean symbol distance field of the environmental point cloud updated in real time as a collision repulsion penalty term 301.
[0075] Step 206: Generate a continuous obstacle avoidance trajectory. To address the issue that traditional polygonal trajectories can easily lead to sudden stops and accelerations, causing attitude loss in multirotor aircraft, this embodiment employs higher-order parametric curves such as B-spline curves. By minimizing a comprehensive objective function that includes smoothness penalties, dynamic feasibility penalties, and gravity and repulsion penalties, the local gradient is solved, and the curve control points 304 are dynamically adjusted to generate a continuous obstacle avoidance trajectory 303 with continuous second derivatives and physical feasibility. If the trajectory planning fails due to extremely confined environments, the system automatically switches to hover protection mode and waits for replanning.
[0076] Step 207: Isolation of underlying control mapping and data flow. The trajectory dynamic reconstruction module 130 discretizes the generated continuous obstacle avoidance trajectory 303 into setpoint commands according to the spatiotemporal sampling parameters and sends them to the flight control system 140.
[0077] Furthermore, the system employs a strict data flow isolation mechanism. The onboard computer 100 asynchronously writes high-bandwidth color images and dense point cloud data directly to onboard physical storage media 150, such as solid-state drives, via PCIe (Peripheral Component Interface Express, a bus and interface standard) or a high-speed external bus. Meanwhile, it transmits status data such as aircraft position, power level, and fault alarms, as well as control data such as computing unit instructions and aircraft behavior commands, back to the ground station via a serial interface and air-to-ground communication link. This data flow isolation mechanism fundamentally eliminates the risk of command delays caused by massive amounts of sensing data occupying flight control bandwidth.
[0078] Optionally, to further improve the efficiency of large-scale inspections, before executing step 201, the system can pre-execute a global waypoint offline optimization step: using a local physical coordinate system as a reference, pre-record preset observation nodes including three-dimensional translation and three-dimensional attitude. Subsequently, the preset observation nodes are transformed into a directed weighted graph model, using the aircraft energy consumption model and spatial distance as edge weights, and redundant paths are eliminated through graph optimization theory or heuristic routing optimization algorithms to generate an optimal topology sequence. The UAV performs macroscopic cruise according to the optimal topology sequence. Once the defect identification in step 203 is triggered, it enters a dynamic local approach process; after the observation ends, the system controls the UAV to return to the globally optimal topology sequence to continue executing the remaining tasks, thereby achieving efficient autonomous closed-loop inspection.
[0079] In summary, addressing the challenges of GNSS signal loss or limitation, unmanned aerial vehicle (UAV) positioning divergence in extreme environments such as low-texture conditions, lack of observation attitude constraints in traditional waypoints, and the inability of preset fixed routes to adaptively approach and detect emerging hazards, this invention provides a UAV dynamic inspection method based on multi-source state estimation and local gradients in GNSS (Global Navigation Satellite System) denied environments. This method overcomes the limitations of existing fixed-route inspections, enabling dynamic reconstruction of flight trajectories based on inference results from onboard computing equipment and autonomous approach to hazard monitoring points. First, the method constructs a joint error function using the UAV's multi-source sensors and performs nonlinear optimization to calculate high-precision pose in the relative space of the inspection. Second, it constructs a six-degree-of-freedom spatial waypoint anchoring benchmark containing three-dimensional position and attitude angles. In the automated inspection phase, the onboard target network model extracts the two-dimensional features of the hazard in real time and reprojects them into three-dimensional spatial coordinates. Finally, the three-dimensional spatial coordinates are introduced as a gravitational constraint into the local trajectory optimization objective function, and the obstacle avoidance continuous trajectory is dynamically reconstructed based on gradient calculation and sent to the flight control system. This invention realizes an adaptive control closed loop from sensing to flight control, which significantly improves the accuracy of disease detection and the robustness of positioning.
[0080] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.
Claims
1. A dynamic inspection method for unmanned aerial vehicles (UAVs) based on multi-source state estimation and local gradient in GNSS denied environments, characterized in that, The method includes the following steps: Step 1: Acquire multi-source data of the spatial environment based on a time reference. Using the relative zero point of the system power-on or a preset physical node as a reference, construct a joint error cost function containing the residuals of the multi-source data. Then, use the error state to perform nonlinear optimization to continuously calculate the six-degree-of-freedom pose of the UAV and establish a local physical coordinate system. The multi-source data includes three-dimensional laser point clouds, visual images, and inertial measurement data. Step 2: The UAV performs automated waypoint navigation within the local physical coordinate system and performs onboard real-time inference on the visual image to obtain the two-dimensional features of the disease. Combining the real-time depth information provided by the three-dimensional laser point cloud with the camera intrinsic parameter matrix, the two-dimensional features of the disease are reverse-projected, and the three-dimensional absolute spatial coordinates of the disease under the local physical coordinate system are obtained using the six degrees of freedom pose of the UAV. Step 3: Using the three-dimensional absolute spatial coordinates as the gravity source, construct a gravity penalty term for the target disease, and use the spatial distance field constructed by the three-dimensional laser point cloud as a collision repulsion penalty term. Introduce the gravity penalty term and the collision repulsion penalty term into the comprehensive objective function of local trajectory optimization. Solve the local gradient by minimizing the comprehensive objective function, dynamically adjust the curve control points, and generate a collision-free continuous obstacle avoidance trajectory in real time from the current position of the UAV to the observation area of the disease target. Step 4: Discretize the collision-free continuous obstacle avoidance trajectory into setpoint commands according to the set spatiotemporal sampling parameters, and send them to the UAV's built-in low-level flight control system. Drive the UAV to adaptively deviate from the original cruise route and perform close-range observation operations to get close to structural surface defects. At the same time, build a data stream isolation mechanism to complete the UAV's dynamic inspection.
2. The UAV dynamic inspection method based on multi-source state estimation and local gradient in a GNSS denied environment as described in claim 1, characterized in that, The joint error cost function is specifically as follows: ； in, Let the joint error cost function be... Let be the system state vector. For the index variable of the feature points of the laser point cloud, The set of all valid laser point cloud feature points. For index variables of feature points in visual images, The set of all valid visual image feature points. The information matrix of the IMU pre-integrated residuals. The information matrix of the laser point cloud residual. The information matrix of visual residuals, For dynamic constraints based on inertial measurement data, Geometric constraints for matching laser point clouds with points and surfaces or points and lines. Constraints for visual reprojection photometric residuals.
3. The UAV dynamic inspection method based on multi-source state estimation and local gradient in a GNSS denied environment as described in claim 1, characterized in that, The automated waypoint navigation specifically refers to: Using a local physical coordinate system as a reference, multiple six-degree-of-freedom pose nodes containing three-dimensional translation coordinates and three-dimensional spatial attitude angles are pre-recorded to construct a local inspection waypoint map containing sensor observation line-of-sight constraints. The local inspection waypoint map is then optimized offline to obtain the optimal route, and the UAV performs automated flight according to the optimal route.
4. The UAV dynamic inspection method based on multi-source state estimation and local gradient in a GNSS denied environment as described in claim 3, characterized in that, The specific steps for performing global waypoint offline optimization on the local inspection waypoint map to obtain the optimal route are as follows: The recorded six-degree-of-freedom pose nodes are transformed into a directed weighted graph model, and a comprehensive edge weight cost is constructed for offline solution. Redundant routes are eliminated, and a globally optimal topology sequence that traverses all preset observation nodes is generated. The globally optimal topology sequence is then used as the best route.
5. The UAV dynamic inspection method based on multi-source state estimation and local gradient in a GNSS denied environment according to claim 1, characterized in that, The obtained three-dimensional absolute spatial coordinates of the disease in the local physical coordinate system are specifically as follows: ； in, Three-dimensional absolute spatial coordinates, This is the external rotation matrix between the pre-calibrated camera coordinate system and the body coordinate system. This is the translation vector between the pre-calibrated camera coordinate system and the body coordinate system. For depth information, This is the intrinsic parameter matrix of the camera. These are the two-dimensional pixel coordinates of the disease.
6. The UAV dynamic inspection method based on multi-source state estimation and local gradient in a GNSS denied environment according to claim 1, characterized in that, The comprehensive objective function is specifically as follows: ； in, For the comprehensive objective function, To measure the smoothness penalty term of the derivative of the control points of a B-spline curve, This is the obstacle avoidance repulsion penalty term based on the Euclidean symbolic distance field. As a penalty item for the feasibility of aircraft dynamics, For the gravitational penalty term oriented towards the three-dimensional absolute spatial coordinates of the target disease, , , , These are the weighting coefficients for each penalty item.
7. A UAV dynamic inspection method based on multi-source state estimation and local gradient in a GNSS denied environment, as described in claim 1, is characterized in that... The data stream isolation mechanism is specifically as follows: The air-to-ground communication link transmits only low-bandwidth UAV flight status and basic control data back to the ground, and asynchronously writes multi-source data and pose logs to the onboard physical storage medium.