An unmanned aerial vehicle autonomous inspection method and system based on visual measurement

Abstract

The present application relates to the technical field of unmanned aerial vehicle power inspection, and particularly relates to an unmanned aerial vehicle autonomous inspection method and system based on visual measurement. The method comprises the following steps: obtaining a tower type and a tower semantic topological descriptor through anisotropic graph convolutional neural network inference; encoding a knowledge graph as a spatially weighted inspection standard semantic constraint vector, and completing the allocation of inspection rules to components through probability graph matching by injecting topological sequence prior, to generate a multi-level inspection semantic road sign map; introducing a tower body plane parameter as a pseudo-measurement constraint to obtain a real-time pose manifold vector; constructing an incremental global pose graph by taking the real-time pose manifold vector as a frame node and a visual landmark feature codebook as a fixed reference node, and optimizing and correcting pose drift to obtain a globally consistent inspection pose vector; combining the multi-level inspection semantic road sign map and the globally consistent inspection pose vector, extracting a visual servo deviation of a target component and integrating it into a control cost function, to generate an inspection control instruction.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) power line inspection technology, specifically to a UAV autonomous inspection method and system based on visual measurement. Background Technology

[0002] Currently, the operation mode of drone inspection of overhead power transmission lines generally adopts a preset route + GPS navigation. This model has significant technical defects in engineering applications: Before operation, precise 3D coordinates of the towers must be manually measured, and fixed waypoints and flight paths must be manually designed. It cannot complete autonomous inspections without prior coordinates or preset paths, resulting in long preparation cycles and poor flexibility. Existing tower recognition technologies are mostly 2D image classification, lacking structured topological representation capabilities. They cannot convert the spatial positions and physical connections of components into structured data usable for path planning; the perception results can only achieve classification, failing to form a closed-loop perception-decision technology. Power grid inspection standards are unstructured text specifications, requiring manual interpretation to convert them into parameters such as waypoint positions and shooting angles. Inspection rules cannot be automatically matched and aligned with front-end visual perception results, creating a digital disconnect between the business rule system and the machine execution system. Existing UAV visual positioning mostly uses a general vision-inertial odometry solution, without introducing structural prior constraints for tower scenarios, easily leading to feature tracking loss and pose drift during close-range inspections. Existing inspection flight control is an open-loop position control, neglecting the actual image quality and standard compliance, easily causing the captured images to deviate from the inspection specifications due to wind disturbances and positioning deviations. Summary of the Invention

[0003] To address the shortcomings of existing methods and the needs of practical applications, this invention provides a visual measurement-based autonomous inspection method for unmanned aerial vehicles (UAVs), comprising the following steps: A semantic topology graph is constructed based on the physical connection rules of power poles. Pole types and semantic topology descriptors are obtained through anisotropic graph convolutional neural network reasoning. A three-level inspection standard knowledge graph is constructed, encoded into spatially weighted inspection standard semantic constraint vectors, and cross-modal semantic alignment is achieved with the pole semantic topology descriptors. Using the real-time pole semantic topology graph as the data graph and the standard pole topology graph as the model graph, probabilistic graph matching with injected topological order priors is used to allocate inspection rules to components. Combining fault priority weights and tower safety envelope constraints, a multi-level inspection semantic roadmap is generated. The tower plane parameters extracted from the tower semantic topology descriptor are used as pseudo-measurement constraints. The constraint weights are adjusted by distance and observation quality to obtain the real-time pose manifold vector. The real-time pose manifold vector is used as a frame node and the visual landmark feature codebook is used as a fixed reference node to construct an incremental global pose graph. The pose drift is corrected by hierarchical incremental optimization to obtain a globally consistent inspection pose vector. Combining the multi-level inspection semantic landmark graph and the globally consistent inspection pose vector, the visual servo deviation of the target component is extracted and incorporated into the model predictive control cost function. Inspection control commands are generated under the hard constraint of the tower safety envelope surface.

[0004] Optionally, the step of constructing a semantic topology graph based on the physical connection rules of power poles, and obtaining the pole type and the pole semantic topology descriptor through anisotropic graph convolutional neural network reasoning, includes the following steps: Based on continuous visible light images captured during the drone's approach to the power pole, a multi-view geometry method is used to recover the camera pose and the sparse 3D point cloud of the power pole. Combined with semantic segmentation of power components, 2D semantic labels are projected onto the 3D point set. Each type of semantic component is clustered into a topological graph node, with node attributes including semantic category, 3D centroid, bounding box size, and main orientation unit vector. Topological edges are constructed using standard physical connection rules for power poles as initial constraints, and edge connections are corrected using spatial adjacency relationships, generating a semantic topological graph with four types of labels: longitudinal connection, suspended connection, parallel connection, and terminal connection. A direction-aware graph convolutional neural network is used to infer the semantic topological graph, obtaining the pole type classification result. A local coordinate system for the pole is constructed by fitting the central axis of the tower. The spatial coordinates and orientations of all nodes in the topological graph are transformed to the local coordinate system, and scale normalization is performed based on the total height of the tower, generating a semantic topological descriptor for the power pole.

[0005] Optionally, the construction of a three-level inspection standard knowledge graph, encoding the knowledge graph into a spatially weighted inspection standard semantic constraint vector, and completing cross-modal semantic alignment with the tower semantic topology descriptor, includes the following steps: Using the list of component semantic categories in the pole semantic topology descriptor as the entity scope constraint, three types of structured entities are extracted from the inspection standard text: inspection target components, inspection quality requirements, and quantitative shooting parameters. A three-level hierarchical knowledge graph is constructed with pole type nodes at the top layer, component category nodes in the middle layer, and rule parameter nodes for quality requirements and shooting parameters at the bottom layer. Nodes are connected by subordinate edges, constraint edges, and association edges, where the association edges correspond one-to-one with the edge types of the semantic topology graph. The complete inspection rules corresponding to each component are encoded into component sub-semantic vectors. After assigning weight coefficients according to the component fault priority set by the power grid historical fault statistics, all component sub-vectors are concatenated along the pole topology and superimposed with the tower type global vector to generate the inspection standard semantic constraint vector. The matching misalignment caused by viewpoint occlusion is corrected through topological consistency. The standard constraint vector is mapped to the visual feature space through linear transformation to complete cross-modal alignment.

[0006] Optionally, the process of using a real-time tower semantic topology map as the data map and a standard tower topology map as the model map, and completing the allocation of inspection rules to components through probabilistic graph matching with injected topology order priors, combined with fault priority weights and tower safety envelope constraints, to generate a multi-level inspection semantic roadmap includes the following steps: Using real-time perceived pole semantic topology maps as data maps and standard pole topology maps of the same type as model maps, two types of topological prior probabilities—vertical order and horizontal arrangement—are injected into probabilistic graph matching. A confidence propagation algorithm is used to solve for the globally optimal node correspondence, mapping inspection rules to actual pole components. Each component to be inspected is abstracted as a task node, and a multi-objective optimization function containing path length cost and priority penalty cost is constructed. Based on height monotonicity and path safety constraints, a global task waypoint sequence with execution order is obtained. A tower safety envelope is constructed based on the pole component bounding box and the legal safety distance. Candidate viewpoints are sampled in the hemispherical space around the component, and the viewpoint quality is evaluated using shooting parameter compliance, visibility occlusion rate, safety margin, and attitude smoothness. The optimal shooting viewpoint pose for each component is obtained. A three-level node system of task layer, waypoint layer, and auxiliary layer is constructed. Using four types of topological edges—main execution edge, task mapping edge, emergency bypass edge, and global return edge—a multi-level inspection semantic roadmap with complete business semantics and safety constraints is generated.

[0007] Optionally, the step of introducing tower plane parameters extracted from the tower semantic topology descriptor as pseudo-measurement constraints, and adjusting the constraint weights through distance and observation quality to obtain the real-time pose manifold vector includes the following steps: The three-dimensional planar parameters of each facade of the tower are fitted from the semantic topological descriptor of the tower as a priori plane set; combined with the semantic mask of the tower in the real-time image and the initial pose of the current frame, the observed plane parameters are calculated, and the plane correspondence is completed by matching the angle between the normal vectors to obtain effective plane constraints; a pseudo-measurement error term of the tower plane composed of the normal vector rotation residual and the distance translation residual is constructed and added to the sliding window optimization objective; the Levenberg-Marquardt algorithm is used to perform joint nonlinear optimization, and the dimension is solved by Schur complement elimination compression to obtain the real-time pose manifold vector.

[0008] Optionally, the weight of the pseudo-measurement error term is obtained by multiplying the distance factor and the observation quality factor; the distance factor is calculated based on the distance from the current camera to the tower plane, and the observation quality factor is calculated based on the proportion of tower mask pixels in the current frame and the edge detection confidence.

[0009] Optionally, extracting the visual landmark feature codebook includes the following steps: Based on semantic 3D point clouds, a set of candidate landmarks is obtained by three-level screening using rigid component point clouds. Each candidate landmark is projected onto a multi-view image to crop local image blocks, and feature descriptors are extracted using a superpoint network with fine-tuned tower scene to form a set of multi-view descriptors for each landmark. The candidate landmarks are pre-divided into tower body group, crossarm group, insulator group, and hardware group according to the tower topology. Feature clustering is performed independently within each group to obtain the visual landmark feature codebook.

[0010] Optionally, the step of constructing an incremental global pose graph using real-time pose manifold vectors as frame nodes and visual landmark feature codebooks as fixed reference nodes, and using hierarchical incremental optimization to correct pose drift and obtain globally consistent inspection pose vectors, includes the following steps: Based on the current frame pose and view frustum range, candidate landmarks are screened using a visual landmark feature codebook. Mismatches are eliminated through a three-level verification process involving spatial constraints within the same group, subordinate constraints across groups, and overall topological constraints. When the number threshold is met, a valid closed loop is determined, and the relative pose observation is solved. A two-class node system is constructed, including frame pose nodes and fixed landmark nodes. Based on constraint edges, a linked storage structure is used to achieve purely incremental updates of nodes and edges. A progressive optimization strategy is used to correct pose drift, and hard constraints on pose adjustment are set at each layer to obtain a globally consistent inspection pose vector.

[0011] Optionally, the step of combining the multi-level inspection semantic roadmap with the globally consistent inspection pose vector, extracting the visual servo deviation of the target component and incorporating it into the model predictive control cost function, and generating inspection control commands under the hard constraint of the tower body safety envelope surface includes the following steps: By combining the multi-level inspection semantic roadmap with the globally consistent inspection pose vector, inspection tasks are dynamically scheduled based on weather and power status; inspection standards are transformed into visual servo biases, which are incorporated into the model predictive control cost function, and weights are adjusted in stages to generate smooth control commands under safety envelope constraints.

[0012] Secondly, to efficiently execute the visual measurement-based autonomous UAV inspection method provided by this invention, this invention also provides a visual measurement-based autonomous UAV inspection system, comprising: an input device, an output device, a processor, and a memory, wherein the input device, output device, processor, and memory are interconnected, and the memory stores program instructions used for the visual measurement-based autonomous UAV inspection method. This visual measurement-based autonomous UAV inspection system of the present invention has a compact structure and stable performance, and can stably execute the visual measurement-based autonomous UAV inspection method provided by this invention, further enhancing the overall applicability and practical application capability of this invention.

[0013] This invention first constructs a semantic topological descriptor for poles that incorporates physical connection rules, providing a structured benchmark for real-time perception in subsequent planning and positioning. This solves the problem of automatic pole type identification and structured representation when a UAV approaches from any angle without a preset flight path. Secondly, by constructing a three-level inspection standard knowledge graph and generating semantic constraint vectors, unstructured inspection standards are transformed into machine-computable quantitative constraints, achieving cross-modal alignment with perception results and resolving the digital disconnect between business rules and machine execution systems. Next, through probabilistic graph matching and fault priority-weighted planning, combined with tower safety envelope constraints, a multi-level inspection roadmap with business semantics is generated, enabling automatic inspection path planning and eliminating reliance on preset flight paths. Finally, by introducing pseudo-measurement constraints on the tower plane and incremental global pose optimization using a landmark codebook, pose stability and global consistency during close-range inspections are significantly improved, resolving the visual odometry drift problem caused by repetitive textures in pole scenes. Finally, by incorporating visual servo bias into model predictive control and using the tower safety envelope as a hard constraint, closed-loop control of inspection shooting quality and flight safety was achieved, solving the problem that traditional open-loop control does not pay attention to the actual shooting composition quality. Attached Figure Description

[0014] Figure 1 A flowchart of an autonomous inspection method for unmanned aerial vehicles based on visual measurement provided in an embodiment of the present invention; Figure 2 This is a framework diagram of a visual measurement-based autonomous inspection system for unmanned aerial vehicles (UAVs) provided in an embodiment of the present invention. Detailed Implementation

[0015] Specific embodiments of the present invention will now be described in detail. It should be noted that the embodiments described herein are for illustrative purposes only and are not intended to limit the invention. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that these specific details are not necessary to practice the invention. In other instances, well-known circuits, software, or methods have not been specifically described to avoid obscuring the invention.

[0016] Throughout this specification, references to an embodiment, example, or illustration mean that a particular feature, structure, or characteristic described in connection with that embodiment or example is included in at least one embodiment of the invention. Therefore, phrases appearing in various places throughout the specification, such as "in one embodiment," "in an embodiment," "an example," or "an illustration," do not necessarily refer to the same embodiment or example. Furthermore, specific features, structures, or characteristics can be combined in any suitable combination and / or sub-combination in one or more embodiments or examples. Moreover, those skilled in the art will understand that the illustrations provided herein are for illustrative purposes and are not necessarily drawn to scale.

[0017] To make the description clearer and more accurate, the key terms used in this application are explained below: Semantic topology graph: refers to a graph data structure constructed with the physical components of power poles as nodes and the engineering connections between components as edges. The attributes of the nodes include semantic category, three-dimensional spatial information, geometric dimensions, etc., and the edge types include vertical connection, suspension connection, etc., which are used to structurally express the ontological composition and spatial topological logic of the pole.

[0018] Semantic topology descriptor: refers to a fixed-dimensional feature vector extracted from the semantic topology graph through a graph neural network. It encodes the semantics of the tower components, the three-dimensional spatial distribution, and the topological connection relationships. It is a unified and computable expression of the tower structure.

[0019] Inspection standard semantic constraint vector: refers to the quantitative feature vector generated by constructing a knowledge graph and embedding vectors from unstructured text inspection specifications, which can directly drive subsequent planning. This vector establishes a digital bridge between business rules and machine instructions.

[0020] Multi-level inspection semantic roadmap: refers to a multi-level directed graph structure that includes a task layer carrying business semantics, a waypoint layer carrying spatial execution information, and an auxiliary layer ensuring safety. It is the spatial carrier of autonomous inspection flight path.

[0021] Visual landmark feature codebook: refers to a benchmark database composed of standardized visual features and their three-dimensional spatial coordinates extracted from stable and highly recognizable structural components on poles and towers, used to provide globally consistent reference anchor points for visual positioning.

[0022] Pose manifold vector: refers to the continuous mathematical expression describing the six degrees of freedom of a UAV in three-dimensional space, including position and attitude information, and forms the basis of real-time trajectory.

[0023] Visual servo bias: refers to the three-dimensional spatial deviation quantified as the difference between the composition of the target component in the real-time image (such as position, size, orientation) and the ideal composition required by the inspection standard. It is used to drive the UAV to finely adjust its flight attitude.

[0024] Please see Figure 1 This invention provides a visual measurement-based autonomous inspection method for unmanned aerial vehicles (UAVs), comprising the following steps: S1. Construct a semantic topology graph based on the physical connection rules of power poles, and obtain the pole type and the pole semantic topology descriptor through anisotropic graph convolutional neural network reasoning.

[0025] In one embodiment, a continuous sequence of images acquired by an airborne visible light camera during the approach to a tower is used as input, and the initial inter-frame pose values ​​obtained by pre-integration from an airborne IMU are simultaneously input. The semantically meaningful sparse 3D point cloud is constructed in three stages: 1. Sparse 3D Reconstruction Stage: FAST corner features are extracted from each frame of the image and corresponding binary descriptors are generated. Adjacent frame features are paired through brute-force matching and epipolar geometry verification. Mismatched point pairs are eliminated using a random sampling consensus algorithm. Based on this, an incremental motion recovery structure framework is adopted to solve the relative extrinsic parameters of the camera frame by frame and triangulate them to generate 3D spatial points. The camera pose and point cloud coordinates are optimized by local bundle adjustment to obtain the sparse 3D point cloud of the camera sequence pose and the tower area.

[0026] 2. Semantic component annotation stage: Input a single frame RGB image into a semantic segmentation network trained on a power pole and tower dataset, and output a pixel-level semantic mask. The semantic categories cover the core components of power grid inspection: tower body main material (angle steel / steel pipe), crossarm, suspension insulator string, tension insulator string, conductor hanging point, tension clamp, and equipotential ring. Each component category corresponds to a unique semantic ID and pixel mask, realizing pixel-level differentiation between the tower body and the background.

[0027] 3. Two-dimensional semantic three-dimensional projection stage: Using the camera intrinsic and extrinsic parameters that have been solved in the current frame, each effective pixel of the two-dimensional semantic mask is back-projected into the three-dimensional space along the light direction. Nearest neighbor matching is performed with the sparse three-dimensional point cloud, and each three-dimensional point is assigned a corresponding semantic label. Three-dimensional points that do not match a semantic label are determined as background points and are directly removed from the point set to obtain a sparse three-dimensional point set with semantic attributes.

[0028] Based on this, using semantically sparse 3D point clouds as the foundation and the engineering connection rules of power poles as the core constraints, a structured semantic topology graph is constructed, including: First, Euclidean density clustering is performed on the semantic 3D point cloud according to semantic categories. Each cluster corresponds to a topological graph node. The node attributes include five-dimensional information: semantic category ID, 3D centroid coordinates, bounding box dimensions (length, width, and height), main orientation unit vector, and number of point clouds within the cluster. For example, the main tower material will be clustered into multiple vertical nodes according to segments, the crossarms will be clustered into two horizontal nodes (left and right), and each string of insulators corresponds to an independent suspension node.

[0029] Then, an initial edge connection is established based on the rule library of typical power grid tower topology. The rule library also needs to define the subordinate connection relationship between components: the tower body node establishes a longitudinal connection edge with all crossarm nodes, each crossarm node establishes a suspension connection edge with all insulator string nodes within its spatial projection range below it, and the end node of the insulator string establishes an end connection edge with the corresponding conductor hanging point and clamp node. The rule library serves as the topology skeleton to ensure that the generated graph structure conforms to the physical connection logic of the real tower.

[0030] Next, the spatial Euclidean distance between the centroids of all nodes is calculated. When the centroid distance between two nodes with different semantic meanings is less than 1.2 times the sum of the sizes of the two types of components, they are determined to be physically adjacent, and corresponding topological edges are added. At the same time, erroneous connections with distances exceeding the reasonable range in the prior initialization are deleted. The added edges include: parallel connection edges of multiple strings of insulators on the same crossarm, longitudinal segmented connection edges between multiple main materials of the tower body, and auxiliary connection edges between the end of the crossarm and the equalizing ring.

[0031] The generated semantic topology graph adopts an undirected graph structure. , where V is the set of nodes with attributes, and E is the set of edges with connection type labels; the connection types of the edges are divided into four categories: vertical connection, dangling connection, parallel connection, and terminal connection, which are used for the differential weight allocation of subsequent graph convolution.

[0032] Furthermore, using the constructed semantic topology graph as input, an improved direction-aware graph convolutional neural network is employed to identify tower types, while simultaneously extracting global structured features to generate tower semantic topology descriptors. Specifically: First, based on the neighborhood feature aggregation of conventional graph convolution, a spatial orientation weight factor is introduced to achieve anisotropic aggregation for tower-type vertical structures, including: For each topological edge, the directional weight is calculated based on the angle between the spatial vector of the centroid of the two end nodes and the vertical principal axis of the tower. Longitudinal connections with an angle close to 0° (such as the upper and lower sections of the tower, and the connection between the crossarm and the tower) are given a high aggregation weight of 1.2, transverse connections with an angle close to 90° (such as the crossarm insulators side by side) are given a basic weight of 0.8, and diagonal connections are given a transition weight by linear interpolation of the angle. At the same time, an additional weight coefficient of 0.1 is added to the suspended connection edge to strengthen the feature transmission of the component subordinate relationship.

[0033] Then, tower type identification and reasoning are performed. The attribute vectors of all nodes in the topology graph are used as input layers, and neighborhood features are aggregated through three graph convolutional layers. Global average pooling is then used to obtain global feature vectors, which are finally connected to a fully connected classification head to output the tower type probability distribution. The tower type set covers the mainstream tower types of the power grid: 110kV straight single-circuit towers, 220kV tension double-circuit towers, 500kV angle towers, 10kV distribution straight poles, etc. The category with the highest confidence is output as the final identification result.

[0034] Next, a semantic topological descriptor for the tower is generated. The global pooling features output from the last layer of graph convolution are extracted and concatenated with the fully connected features from the layer preceding the classifier. This results in the tower's semantic topological descriptor, which simultaneously encodes three types of information: component semantic attributes, relative distribution in three-dimensional space, and topological connections between components. It provides a unified structured representation of the tower structure. In this embodiment, the tower's own structure is used as a reference for dual normalization of attitude and scale to obtain a standardized descriptor, including: Extract all tower-type nodes from the semantic topology graph, perform 3D straight line fitting using the centroid of each node as a sample to obtain the central axis of the tower, and use it as the Z-axis (vertically upward) of the local coordinate system; then extract the direction of the line connecting the centroids of the left and right crossarm nodes as the X-axis (horizontal), and determine the Y-axis (route direction) by the right-hand rule, and finally establish the local coordinate system of the tower with the center of the bottom of the tower as the origin.

[0035] Then, the spatial coordinates and orientation vectors of all nodes in the topology graph are transformed from the world coordinate system to the local coordinate system of the tower, and the relative coordinates and relative orientations of the nodes are recalculated. Based on the updated node attributes, graph convolution forward propagation is performed again to generate attitude-aligned descriptors.

[0036] Next, using the total tower height (the vertical distance from the bottom tower node to the top tower node) as the benchmark, the spatial coordinates and dimensional attributes of all nodes are divided by this benchmark to achieve dimensionless processing. The normalized attribute characteristics are then incorporated into the descriptor to eliminate scale deviations caused by slight differences in tower height and observation distance among towers of the same type.

[0037] S2. Construct a three-level inspection standard knowledge graph, encode the knowledge graph into a spatially weighted inspection standard semantic constraint vector, and complete cross-modal semantic alignment with the tower semantic topology descriptor.

[0038] This invention uses standard inspection texts for different pole types as the original input, extracts entities through a large language model, constructs a three-level knowledge graph of pole type, component, and inspection rules, and then transforms it into semantic constraint vectors through a text embedding model. These vectors are then cross-modal aligned with semantic topological descriptors to achieve a one-to-one correspondence between knowledge rules and perception results.

[0039] First, extract the perception-aligned inspection standard entities, using two types of data as input: one is an unstructured text set of inspection procedures and work instructions for overhead transmission lines corresponding to different voltage levels and tower types; the other is a list of component semantic categories attached to the tower semantic topology descriptor (tower body main materials, crossarms, suspension insulator strings, tension insulator strings, conductor hanging points, tension clamps, equipotential rings). Then, complete the constraint-based entity extraction according to the following process: A large language model fine-tuned by instructions from the power inspection procedure dataset is adopted, with a preset structured output Prompt, a forced output standard JSON format, and field specifications and value boundaries for three types of entities.

[0040] After splitting the standard text into clauses, input each segment into the model, extract three core entities, and populate the corresponding fields: Inspection target component entity: The fields include the component's standard name, mapping semantic ID, applicable tower type, and applicable voltage level; Inspection quality requirements entity: Fields include defect observation type, coverage requirements, judgment accuracy requirements, and key observation areas; Quantitative shooting parameter entity: Fields include shooting distance range, shooting pitch / azimuth angle range, single component pixel ratio requirement, and illumination threshold.

[0041] In the embodiment, all extracted component entities can be matched with the component semantic category list for string matching and semantic similarity dual verification. Any sub-components not in the list are uniformly classified upwards to the parent semantic category; non-standard entities that cannot be classified are directly removed to ensure that the final extracted component set completely corresponds to the semantic node set of the front-end visual perception, and there are no rule entities that the perception cannot recognize, thus achieving pre-alignment between the knowledge system and the perception system.

[0042] Furthermore, using structured entity data as input, with entities as nodes and business logic relationships between entities as edges, a three-level directed knowledge graph is constructed, including: 1. Definition and instantiation of third-level nodes: Top-level nodes: Tower type nodes, uniquely identified by voltage level + tower type code. Node attributes include tower type name, voltage level, corresponding standard number, and overall inspection cycle. The number of nodes is consistent with the total number of tower types covered for inspection.

[0043] Intermediate layer nodes: Component category nodes, uniquely identified by tower type ID + component semantic ID. Node attributes include component semantic ID, component standard name, tower type, spatial priority level, and corresponding topology connection type. Each tower type node has multiple component category nodes under it.

[0044] Bottom-level nodes: Rule parameter nodes, divided into two categories: quality requirement nodes and shooting parameter nodes. Node attributes include rule type, parameter value range, unit of measurement, and applicable environmental conditions. Each component category node contains multiple rule parameter nodes.

[0045] 2. Construction of topological relationship edges: Subordinate edge: A directed edge that points from the top-level tower type node to the middle-level component category node, indicating that the tower type contains the inspection component; Constraint edge: A directed edge that points from the intermediate component category node to the underlying rule parameter node, indicating that the component is constrained by this inspection rule; Associated edges: Undirected edges connect nodes of different component categories under the same tower type. The edge type corresponds one-to-one with the edge type of the semantic topology graph (vertical connection, dangling connection, parallel connection), representing the physical topological adjacency relationship between components.

[0046] Differentiated rules association for different tower types: For the rule differences of the same type of component under different tower types, tower type-specific labels are marked on the bottom rule nodes. Different constraint edges point to the component nodes of the corresponding tower type, realizing differentiated storage and retrieval of rules for different tower types and avoiding confusion between rules of different tower types.

[0047] Based on this, taking the complete rule text corresponding to each component node in the knowledge graph and the component space priority level as input, a computable inspection standard semantic constraint vector is generated through a domain text embedding model, including: Single component rule text serialization: For each component category node, the contents of all its subordinate underlying rule nodes are concatenated into standardized rule text in the order of quality requirements-shooting parameters, such as component semantic ID-inspection defect requirements-shooting distance range-shooting angle range-resolution requirements.

[0048] Component sub-semantic vector encoding: A text embedding model finely tuned in the power field is adopted to encode each standardized rule text into a component sub-semantic vector, and the vector dimension is consistent with the component node feature dimension of the topological descriptor.

[0049] Spatial priority weighted processing: The priority weight coefficient of components is set in advance based on the historical fault statistics of the power grid: the weight of high fault risk components such as crossarms, insulator strings, and clamps is set to 1.5, the weight of medium risk components such as tower main materials and equalizing rings is set to 1.0, and the weight of low risk components such as the lower section of the tower is set to 0.5. The sub-semantic vector of each component is multiplied element by the corresponding weight coefficient to obtain the weighted component sub-vector, so that the rules of high-risk components occupy a higher feature proportion in the vector.

[0050] Generate a global constraint vector: First, encode the overall description text of the tower type into a tower type global vector. Then, according to the component topology order of the semantic topology diagram (tower body from bottom to top, crossarm from inside to outside, insulator from top to bottom), concatenate all weighted component sub-vectors in sequence. Finally, concatenate the tower type global vector at the beginning of the vector to form a complete inspection standard semantic constraint vector.

[0051] Furthermore, feature alignment is performed on the inspection standard semantic constraint vector and the tower semantic topological descriptor, including: Semantic ID hard matching coarse alignment: Based on the semantic ID of the component, a one-to-one mapping is performed. The component sub-vectors with the same semantic ID in the standard constraint vector are paired with the component node feature vectors with the corresponding semantic ID in the topological descriptor to form an initial set of matching pairs.

[0052] Cosine similarity fine-tuning: For each initial matching pair, calculate the cosine similarity between the component sub-vector and the component node feature vector; set a similarity threshold of 0.7, and match pairs below the threshold are marked as suspected mismatches and enter the correction process.

[0053] Topology consistency correction: For suspected mismatched components, rematching is performed based on the spatial relative position and topological connection relationship of the components. For example, multiple strings of insulators on the same crossarm are reordered and matched according to the topological order arranged from the tower body outwards, and the hierarchical relationship of the matched pairs is checked to see if it conforms to the physical structure of the tower. Mismatched pairs with topological inconsistencies are eliminated, and matching misalignments caused by viewpoint obstruction and differences in the number of components are corrected.

[0054] Feature space linear calibration: For all final matching pairs, the linear transformation matrix is ​​solved by least squares to transform all component sub-vectors of the standard constraint vector into the feature space of the topological descriptor, eliminating the feature distribution differences between the text modality and the visual modality, and completing the final alignment.

[0055] S3. Using the real-time tower semantic topology map as the data map and the standard tower topology map as the model map, the inspection rules are assigned to components by probabilistic graph matching with injected topology order priors. Combined with fault priority weights and tower safety envelope constraints, a multi-level inspection semantic roadmap is generated.

[0056] In this embodiment, a real-time pole semantic topology map is used as the data map, and a standard pole topology map of the same type corresponding to the knowledge graph is used as the model map. The nodes of the data map are real-time perceived component semantic nodes, carrying four types of attributes: semantic ID, three-dimensional centroid coordinates, main orientation, and node size, and four types of connection type labels: longitudinal connection, suspended connection, parallel connection, and end connection. The nodes of the model map are standard component nodes in the standard library, carrying standard semantic ID, relative three-dimensional coordinates in the pole local coordinate system, bound inspection rule parameters, and corresponding standard connection type labels.

[0057] Then, a probabilistic graph matching algorithm is used to solve the one-to-one correspondence between nodes in the two types of graphs, assigning inspection rules to actual tower components and obtaining the confidence level of each matching pair, including: Initial node matching probability calculation: For all node pairs in the data graph and model graph, first perform hard matching filtering by semantic ID, and only node pairs with the same semantic category enter the subsequent calculation; then map the two nodes to the local coordinate system of the tower, calculate the relative spatial position deviation and main orientation deviation, and convert the deviation into node attribute matching probability after normalization, which is used as the initial observation probability for matching.

[0058] Edge structure consistency verification: For two sets of candidate matching node pairs, verify whether the connection types of their corresponding edges are consistent, and calculate the relative proportional deviation of the lengths of the two edges. The smaller the deviation, the higher the structure consistency score and the higher the joint probability of the corresponding matching pair. From the perspective of adjacency structure, constrain the matching results to conform to the tower topology.

[0059] Topological order prior probability injection: Two types of structured prior probabilities are introduced as prior weights for matching: First, the vertical order prior, where the main tower material nodes and crossarm nodes must satisfy the spatial arrangement order from top to bottom, and matching pairs that match the order in the model diagram are multiplied by a prior weight coefficient of 1.3; Second, the horizontal arrangement prior, where the insulator string nodes under the same crossarm must satisfy the sequential arrangement order from the tower body to the end of the crossarm, and matching pairs that match the arrangement order in the model diagram are multiplied by a prior weight coefficient of 1.2; Candidate pairs with mismatched order are multiplied by a penalty coefficient of 0.6, suppressing node mismatch caused by unilateral view occlusion from a probabilistic perspective.

[0060] Global matching solution and rule mapping: The confidence propagation algorithm is used to iteratively pass matching probabilities on the graph. After convergence, the globally optimal one-to-one correspondence between nodes is solved, and the matching confidence of each set of matching nodes is output. At the same time, all inspection rules and shooting parameters bound to the nodes of the model graph are fully mapped to the corresponding component nodes of the data graph, so as to complete the accurate allocation of inspection standards to actual tower components.

[0061] Furthermore, based on the multi-objective traveling salesman problem with business priority constraints, and utilizing the set of component inspection tasks that have completed rule mapping and the current global pose of the UAV, a global task waypoint sequence with execution order is solved, including: Task node attribute initialization: Each component to be inspected is abstracted as a task node. The node attributes include: component 3D centroid coordinates, component semantic ID, fault priority weight, estimated shooting duration, and bound inspection rule set. The fault priority weight is set based on the power grid's historical defect statistics and is consistent with the spatial priority weight system: suspension / tension insulator strings and tension clamps are first-level high-risk components with a weight of 1.5; crossarm connection nodes and equipotential rings are second-level medium-risk components with a weight of 1.0; and tower body main material sections are third-level low-risk components with a weight of 0.6.

[0062] Multi-objective cost function construction: The total optimization cost is composed of a weighted average of path length cost and priority penalty cost, logically as follows: Total cost = Path length cost × (1-ω) + Priority penalty cost × ω, where ω is the business weight coefficient, with a default value of 0.4. The path length cost is the sum of the three-dimensional Euclidean distances between adjacent task nodes, representing flight efficiency and energy consumption; the priority penalty cost is the execution sequence number of each task node multiplied by its corresponding priority weight. The later a high-priority component is ranked, the higher the penalty cost, representing the potential hazard investigation value of the inspection business.

[0063] Additional engineering constraints: Two types of tower scenario constraints are added to the solution constraints: First, the height monotonicity constraint, which prioritizes the increasing height order from the bottom of the tower to the top, and then successively inspects the crossarms and insulators, to reduce the energy consumption and attitude adjustment times of the UAV's frequent take-off and landing; Second, the path safety constraint, which stipulates that the connection between adjacent task nodes must not cross the tower's safety envelope. If it does, a safety penalty term is added to the cost.

[0064] Optimal sequence solution: The ant colony algorithm is used to solve the optimal task order under constraints. The current position of the UAV is taken as the starting point of the sequence, and the inspection order with the lowest total cost is output iteratively to generate the initial global waypoint sequence.

[0065] Based on this, according to the three-dimensional position and main orientation of a single component to be inspected, and the shooting parameter requirements in the corresponding inspection standard, the optimal shooting viewpoint pose of the component is obtained through local optimal viewpoint optimization with safety envelope constraints, including: Tower safety envelope construction: Based on the semantic topology graph of the tower, the bounding boxes of all tower body and crossarm component nodes are extracted and merged to generate the minimum bounding polyhedron of the tower body; on this basis, the safe distance is uniformly expanded outward according to the legal safety distance of the corresponding voltage level (1.5 meters for 10kV distribution towers, 3 meters for 110kV-220kV transmission towers, and 5 meters for 500kV and above transmission towers) to generate a closed columnar safety envelope; the area inside the envelope is defined as a hard-constrained no-fly zone, and all candidate viewpoints and flight paths are not allowed to enter this area.

[0066] Uniform sampling of candidate viewpoints: Using the three-dimensional centroid of the component to be inspected as the center of the sphere, within the shooting distance range specified in the inspection standard (such as a typical value of 8-12 meters), candidate viewpoints are uniformly sampled in the hemispherical space facing the UAV with an azimuth step of 15° and a pitch step of 10°. Each candidate viewpoint has its optical axis pointing to the centroid of the component by default, and a set of camera position and attitude parameters are generated accordingly.

[0067] Multi-dimensional evaluation of viewpoint quality: A comprehensive quality score is calculated for each candidate viewpoint, including: ① Shooting parameter compliance: The deviation of shooting distance, shooting pitch angle, and component pixel ratio from the standard requirements is calculated. The smaller the deviation, the higher the score; ② Visibility occlusion rate: Using the ray projection method, light rays are projected from the candidate viewpoint to 20 feature points uniformly sampled on the surface of the component. The intersection with the geometric surfaces of other components of the tower is calculated, and the proportion of occluded feature points is statistically analyzed. The lower the occlusion rate, the higher the score; ③ Safety margin: The minimum distance from the candidate viewpoint to the tower's safety envelope is calculated. The greater the distance, the higher the safety margin and the higher the score; ④ Attitude smoothness: The attitude change between the current viewpoint and the previous waypoint is calculated. The smaller the change, the smoother the flight and the higher the score.

[0068] Optimal viewpoint solution: The scores of all candidate viewpoints are weighted and summed, and the viewpoint with the highest comprehensive score is selected as the optimal shooting viewpoint for the component. The corresponding three-dimensional position coordinates and camera pose quaternions are output. If all sampled viewpoints are obstructed or violate safety regulations, the shooting distance interval is expanded by 1 meter and resampling is performed until a compliant viewpoint is obtained.

[0069] Furthermore, based on the global task sequence, the optimal viewpoint pose of all components, and the tower's safety envelope parameters, a multi-level inspection semantic roadmap is generated using a three-level directed graph structure: task layer, waypoint layer, and auxiliary layer. This includes: Definition and instantiation of a three-level node system: Task layer nodes: Each node corresponds to an independent component inspection task. The node attributes include task ID, corresponding component semantic ID, fault priority, and a complete set of inspection rules. It is the business semantic carrier of the road sign. The number of nodes is consistent with the total number of components to be inspected.

[0070] Waypoint layer nodes: Each node corresponds to an actual flight shooting waypoint. The node attributes include waypoint ID, associated task ID, three-dimensional spatial coordinates, camera attitude quaternion, shooting trigger flag, and estimated dwell time. It is the spatial execution carrier of the waypoint. Each task layer node corresponds to a waypoint layer node, and the two are mapped one-to-one through ID.

[0071] Auxiliary layer nodes: include two types: emergency hovering points and safe return points. Emergency hovering points are set at a fixed height outside the safe distance to the side of the tower for temporary hovering in case of emergencies. Safe return points are set at a safe position 10 meters in front of the tower for mission termination or low battery return. Node attributes include node ID, spatial coordinates, and node function type.

[0072] Then, construct the topological edge system, including: Main execution edge: a directed edge that connects adjacent waypoint layer nodes in the order of inspection. The edge attributes include path length, estimated flight time, maximum allowable flight speed, and path safety margin, forming the main flight path for inspection.

[0073] Task mapping edge: An undirected edge that connects task layer nodes with their corresponding waypoint layer nodes, enabling the mapping of business semantics to spatial location.

[0074] Emergency bypass edge: a directed edge where each waypoint layer node is connected to the nearest emergency hovering point, and the edge attributes are emergency path length and shortest response time.

[0075] Global return edge: a directed edge that connects all waypoint layer nodes to the safe return point of the auxiliary layer, supporting return from any location.

[0076] Finally, two checks are performed on the generated road map: first, semantic consistency check, to confirm that all waypoints are bound to the inspection rules of the corresponding components, with no omissions or mismatches; second, spatial safety check, to confirm that the paths corresponding to all edges do not penetrate the tower's safety envelope, and the safety margin meets the requirements.

[0077] S4. The tower plane parameters extracted from the tower semantic topology descriptor are introduced as pseudo-measurement constraints. The constraint weights are adjusted by distance and observation quality to obtain the real-time pose manifold vector.

[0078] In this embodiment, the semantic topological descriptor of the tower and the semantic node attribute set of the tower body are used as the prior input of the plane, and the real-time image semantic segmentation mask, the initial predicted pose of the current frame, and the camera intrinsic parameters are used as the observation input. The prior plane extraction, observation plane solution, and correspondence matching are completed in three stages. Specifically: First, extract the prior plane parameters of the tower body. Extract all main material nodes of the tower body from the semantic topology map of the tower. Using the 3D centroid of the node and the sampling points on the bounding box surface as the sample set, perform K-means clustering according to the main orientation of the node (the default number of clusters is 4, corresponding to the four facades of the quadrilateral tower). For the 3D point set in each cluster, use the random sampling consensus algorithm to fit the 3D plane equation, and output the plane normal vector (unit 3D vector) and the vertical distance from the plane to the origin of the local coordinate system of the tower for each facade. This constitutes the prior plane parameter set of the tower body. Each plane corresponds to a unique facade ID, corresponding semantic label and spatial orientation. The parameters are defined based on the local coordinate system of the tower and can be directly converted to the world coordinate system through the global pose of the tower, serving as the true reference for the plane constraints.

[0079] Then, based on the tower semantic segmentation mask of the current frame, the initial predicted pose of the current frame (obtained by recursion from the VIO output pose of the previous frame combined with IMU pre-integration), and the camera intrinsic parameter matrix, the real-time observation plane parameters are calculated: The first step is to extract the edge pixels of the tower body from the tower body mask, and use Hough line transform to detect the longitudinal edges of the tower body, outputting the line equations of the two longitudinal edges in the image plane. The second step is to backproject the two image lines into two ray planes in three-dimensional space using the camera intrinsic parameters. The third step is to use the normal range of the prior plane as a constraint, combined with the intersection direction and initial depth of the two ray planes, to solve for the observed normal vector of the tower body facade and the distance from the camera optical center to the observation plane, thus forming the tower body observation plane parameters.

[0080] Next, the spatial angle between the normal vector of the observation plane and the normal vector of each elevation in the prior plane set is calculated. The prior plane with the smallest angle less than 15° is selected as the matching object to complete the one-to-one correspondence between the observation plane and the prior plane. If the angle between all prior planes and the observation plane is greater than 15°, or the proportion of tower body mask pixels is less than 2% of the total area of ​​the image, the tower body observation in the current frame is determined to be unreliable, and no plane constraint is output to avoid erroneous observations interfering with pose optimization.

[0081] Furthermore, using continuous images from the airborne camera and IMU data with synchronized timestamps as raw input, a fixed-size sliding window is constructed using a keyframe mechanism to complete visual feature tracking, IMU pre-integration, and state variable initialization, thus building a basic VIO optimization framework, including: The sliding window is set to hold 15 keyframes. When the inter-frame translation exceeds 0.2 meters or the rotation exceeds 5°, a new keyframe is extracted. This balances onboard computing power and pose estimation accuracy, and is suitable for the motion characteristics of UAVs during low-speed flight and frequent fine-tuning of attitude. When the window is full, an edge-shifting strategy is used to remove the oldest keyframe while retaining the state constraints of the newest frame, thus maintaining a constant window length.

[0082] For each frame of image, superpoint features and 256-dimensional descriptors are extracted. Optical flow tracking combined with descriptor matching is used to complete the feature association between adjacent frames. The fundamental matrix is ​​solved by random sampling consensus algorithm to eliminate mismatched outliers. When the number of shared feature points between adjacent frames is greater than 80 and the average disparity of feature points is greater than 10°, epipolar geometry is used to solve the relative pose of the two frames and triangulation is used to generate initial 3D map points. As subsequent keyframes are added, new map points are continuously triangulated and the shared view relationship is updated.

[0083] Next, the IMU measurements between adjacent keyframes within the sliding window are pre-integrated to obtain the inter-frame relative rotation, velocity increment, displacement increment, and corresponding covariance matrix. The inter-frame pose trajectory calculated by vision is aligned with the IMU pre-integration results. The initial zero bias of the gyroscope is solved through hand-eye calibration, and the initial zero bias of the accelerometer and the initial velocity of each frame are solved through gravity vector direction alignment. This completes the initialization of the state variables of all keyframes within the sliding window.

[0084] Furthermore, an initial nonlinear optimization objective function is constructed, comprising two types of error terms: one is the visual reprojection error term, which is the deviation between the pixel coordinates of map points projected onto each keyframe and the actual observed coordinates; the other is the IMU pre-integration error term, which is the deviation between the IMU pre-integration result and the state variable integration result. In the embodiment, feature weights can also be set: the weight of feature points within the semantic region of the tower body is set to 1.2, and the weight of feature points in the background region is set to 0.8, to strengthen the constraint effect of rigid structural features on pose.

[0085] Furthermore, using the prior plane and observation plane parameters of the tower body as input, a quantifiable pseudo-measurement error term is constructed, and a two-factor adaptive weighting mechanism is designed to dynamically integrate structural constraints into the sliding window optimization objective, avoiding noise introduced by low-quality observations, including: Construction of the pseudo-measurement error term for the tower plane: Using the successfully matched prior plane of the tower as the constraint reference and the observed plane as the measurement value, two types of residuals are constructed: First, the normal vector rotation residual, which is the spatial angle between the normal vector of the observed plane and the normal vector of the prior plane, transformed into the rotation constraint residual of the corresponding frame pose; second, the distance translation residual, which is the difference between the distance from the camera optical center to the prior plane calculated using the current frame pose and the distance solved using the observed plane, transformed into the translation constraint residual of the corresponding frame pose. The two types of residuals are weighted and summed in a 6:4 ratio to form the pseudo-measurement error term for the tower plane in a single frame, which is then added as the third type of error term to the sliding window optimization objective.

[0086] Two-factor adaptive weight calculation: The total constraint weight is obtained by multiplying the distance factor and the observation quality factor. The weight range is 0~1.0 and is dynamically updated frame by frame.

[0087] The distance factor is calculated based on the straight-line distance from the optical center of the camera in the current frame to the plane of the tower. When the distance is ≤10 meters, the weight is 1.0. When the distance is between 10 and 20 meters, the weight decreases linearly to 0.2. When the distance is >20 meters, the weight is 0. The plane constraint is completely turned off to avoid interference caused by insufficient accuracy of long-distance observation.

[0088] The observation quality factor is calculated based on the pixel ratio of the tower body mask in the current frame and the edge detection confidence. It is 1.0 when the mask ratio is ≥10% and the edge confidence is ≥0.8, and 0 when the ratio is <2% or the confidence is <0.5. The intermediate interval is linearly interpolated.

[0089] Constraint access and inter-frame association: Tower plane pseudo-measurement constraints are accessed in the form of frame-by-frame single-point constraints. As long as each frame passes the validity check within the sliding window, a plane error term with the corresponding weight is added. Constraints only apply to the pose state variables of the corresponding key frame and do not affect other state variables such as IMU zero bias and map points, thus avoiding state coupling anomalies caused by constraint diffusion.

[0090] Based on this, joint nonlinear optimization is performed on the multi-source error terms within the sliding window framework to solve for the optimal state variables of all keyframes within the window, obtaining standardized real-time pose and confidence, including: The optimization state variables and objective function are defined as follows: The state variables to be optimized for each keyframe within the sliding window include: 3D position in the world coordinate system, rotational attitude represented by a unit quaternion, 3D linear velocity, 3D zero bias of the gyroscope, and 3D zero bias of the accelerometer; it also includes the spatial coordinates of all common-view 3D map points within the window. The overall optimization objective function is the weighted sum of squares of three types of error terms: visual reprojection error term, IMU pre-integration error term, and tower plane pseudo-measurement error term. The weights of each term are dynamically configured according to the aforementioned rules.

[0091] Nonlinear iterative solution configuration: The Levenberg-Marquardt algorithm is used to perform iterative optimization. The map point variables are marginalized by Schur complement elimination, and only the key frame state variables are retained as the core solution variables. The solution dimension is compressed to ensure that the optimization time per frame is controlled within 10 milliseconds, which meets the pose output frequency of more than 20Hz and meets the requirements of airborne real-time control.

[0092] After optimization and convergence, the six-degree-of-freedom pose of the latest keyframe within the sliding window is extracted to form a real-time pose manifold vector. The vector is fixed to a 7-dimensional structure: the first 3 dimensions are the three-dimensional position coordinates in the world coordinate system, and the last 4 dimensions are the rotational attitude represented by unit quaternions. The pose estimation confidence score is also output. The confidence score is calculated by normalizing and weighting three indicators: the number of effective feature points, the weight of the planar constraint, and the size of the total residual. The value ranges from 0 to 1, representing the reliability of the current pose.

[0093] In the embodiment, a pose degradation strategy can also be triggered when the pose confidence is below 0.3: stop the tower plane constraint access, use the IMU pre-integration recursion result as the main body of pose output, and only make low-speed smooth corrections for visual constraints to avoid pose jumps caused by optimization divergence and ensure the stability of flight control; when feature tracking is restored and the confidence rises back to above 0.5, full constraint optimization is restored.

[0094] S5. Construct an incremental global pose graph by using the real-time pose manifold vector as frame nodes and the visual landmark feature codebook as fixed reference nodes, and use hierarchical incremental optimization to correct pose drift and obtain a globally consistent inspection pose vector.

[0095] In this embodiment, firstly, using semantically labeled sparse 3D point clouds, tower semantic topology graph node attribute sets, and component semantic ID mapping tables as inputs, stable landmark candidates are screened through a two-stage constraint process of semantic preliminary screening and structural fine screening, including: Semantic category initial screening: Based on semantic IDs, the candidate range of landmarks is defined, and only point clouds corresponding to rigid structural components of the tower are retained in the candidate pool. This covers five categories: main tower body, crossarm body, upper insulator hanging point, tension clamp, and equipotential ring fixed end. Flexible and easily swaying components such as conductors and lower insulator fittings, as well as background point clouds such as vegetation, ground, and sky, are excluded. Each type of component corresponds to a preset landmark generation rule. For example, the main tower body corresponds to the angle steel intersection node, and the crossarm corresponds to the end point and the connection node between the end point and the tower body, ensuring that the candidate points are all attached to a long-term fixed rigid structure.

[0096] Structural stability screening: Euclidean density clustering is performed on the point clouds in the candidate pool according to semantic categories. Each cluster corresponds to a candidate landmark ontology, which is then subjected to three checks: ① Spatial rigidity verification: Calculate the standard deviation of the three-dimensional coordinates of the point cloud within the cluster. If it is less than 0.05 meters, the structure is considered stable, and edge noise point clusters with large dispersion are removed. ② Geometric recognition verification: Perform local geometric fitting on the point cloud within the cluster. The cross nodes of the tower body must meet the requirement that the residual of the double orthogonal straight line fitting is less than 0.03 meters, and the crossarm endpoints must meet the requirement that the edge features are significant in one direction, to ensure that the geometric shape is unique and not easily confused with other points. ③ Visibility verification: Project sampling light rays from the candidate cluster to the surrounding hemispherical space, and count the angle range blocked by other tower components. If the obstruction ratio is less than 30% of the total viewing angle, the visibility is deemed qualified, and invalid points that are blocked all year round are removed.

[0097] All candidate landmarks that pass all verifications are encapsulated into structured entries, including: candidate landmark temporary ID, corresponding semantic ID, 3D centroid coordinates in world coordinate system, local point cloud bounding box size, and effective viewing angle range.

[0098] Furthermore, using the candidate landmark set, the multi-view continuous image sequence collected during the drone's approach to the tower, and the corresponding frame camera intrinsic and extrinsic parameters as input, standardized multi-view super-point features are extracted and illumination compensation is performed, including: Multi-view projection and cropping of landmark areas: For each candidate landmark, its three-dimensional centroid coordinates and the camera pose of each frame are projected onto the two-dimensional image plane through a pinhole camera model to obtain pixel coordinates; a local image block of 64×64 pixels is cropped with the pixel as the center, and two filtering operations are performed at the same time: edge frames with projection points less than 16 pixels from the image edge are removed, and frames in which the landmark area is marked as occluded by semantic mask are removed, thus obtaining a set of clear image blocks with multiple views corresponding to the landmark.

[0099] Superpoint model feature extraction: A superpoint network fine-tuned with a tower metal component dataset is used to extract keypoint locations and 256-dimensional descriptors for each image block. The top 10 keypoints with the highest response values ​​are retained for each image block. The keypoint closest to the center of the landmark projection is selected as the representative feature point of the landmark, and the corresponding descriptor is output.

[0100] 3D feature alignment: The 2D feature points under each viewpoint are back-projected to the 3D world coordinate system through the camera pose of the corresponding frame, and registered and calibrated with the 3D centroid of the landmark to eliminate projection deviation; All viewpoint descriptors of the same landmark are uniformly aligned to the feature space under the world coordinate system to form a set of multi-viewpoint descriptors corresponding to the landmark.

[0101] Local illumination normalization: Two-level illumination compensation is performed on each descriptor to reduce feature drift caused by changes in outdoor illumination. ① Neighborhood brightness compensation: Calculate the average gray value of the 16×16 pixel area around the feature point, and use the average gray value as a reference to perform linear offset correction on the descriptor vector to offset the overall drift of the descriptor caused by the overall brightness change. ② Contrast normalization: Local contrast normalization is performed on the corrected descriptor vector, followed by L2 norm normalization to eliminate the difference in feature amplitude caused by varying illumination intensity.

[0102] Based on this, using the standardized multi-view description subsets of all candidate landmarks and the topological grouping rules of the tower semantic topology map as input, we group and cluster them according to topological structure to generate a structured visual landmark feature codebook, including: Based on the component hierarchy of the tower semantic topology, all candidate landmarks are divided into four independent groups: tower body group (cross-joint nodes of main tower body materials), crossarm group (connection nodes between crossarm endpoints and tower body), insulator group (upper and lower hanging points of insulators), and hardware group (fixing points of clamps and equipotential rings). Each group corresponds to a semantic ID, and subsequent clustering is performed independently within each group. Features of different groups do not participate in cross-calculation, thus compressing the matching search range from a spatial dimension.

[0103] For all descriptors within each group, the K-Means++ algorithm is used to perform clustering. The number of clusters is the same as the number of candidate landmarks in that group. Each cluster center corresponds to a final landmark, and the cluster center vector is taken as the baseline feature descriptor corresponding to that landmark.

[0104] Each clustering result corresponds to a codebook entry, containing: ① Landmark Unique ID: A globally unique identifier generated in the format of group code + serial number; ② Semantic Tags: Corresponding to the unified component semantic ID and the component standard name; ③ Three-dimensional coordinates: The three-dimensional centroid coordinates of the landmark in the world coordinate system, which serve as the fixed reference point for subsequent pose optimization; ④ Baseline descriptor: 256-dimensional cluster center feature vector, used for real-time feature matching; ⑤ Effective observation range: azimuth angle range and elevation angle range, which represent the range of viewing angles from which the landmark can be effectively observed; ⑥ Grouping Index: The topological group number to which it belongs, used for quick retrieval and location.

[0105] Finally, using four topological groups as primary indexes, a secondary grid index is established within each group based on three-dimensional spatial coordinates. During subsequent real-time matching, candidate matching groups and local grids can be quickly locked based on the current position and viewpoint of the UAV, meeting the requirements of airborne real-time computing.

[0106] In this embodiment, the initial visual landmark feature codebook, multi-scene tower test image set, real-time inspection frame images and pose data can also be used as input. Initial reliability is ensured through offline verification, and continuous optimization is achieved through online incremental updates, including: Offline robustness verification: ① Construct a test sample set: Cover tower images with different lighting conditions (sunny, cloudy, dawn and dusk), different observation distances (6-15 meters), and different observation angles. Each frame of the image is labeled with the real landmark location as the ground truth. ② Matching performance test: Perform feature matching on the test set using the initial codebook, and calculate the precision (number of correct matches / total number of matches) and recall (number of correct matches / total number of true values) for each codebook entry. ③ Item filtering and merging: Set a precision threshold of 0.8 and a recall threshold of 0.6. Items that fail to meet the two criteria are removed. Duplicate items with a 3D distance of less than 0.2 meters and a descriptor cosine similarity greater than 0.9 are merged into a single item, and the mean of the coordinates and the mean of the descriptor are taken as the attributes of the new item.

[0107] Online incremental update mechanism: ① New Landmark Candidate Trigger: During real-time inspection, if a stable feature cluster is detected at the same location in 5 consecutive frames of images, and the feature cluster corresponds to a clear rigid component semantics and cannot be matched with an existing codebook entry, it is marked as a candidate new landmark. ② Multi-view verification: Collect feature descriptors from at least 6 different views of the candidate landmark, calculate the average cosine similarity of the descriptors, and if it is greater than 0.75, it is determined to be a valid new landmark; ③ Data entry update: Valid new landmarks are assigned to the corresponding topological groups according to semantics, new codebook entries are generated and the index structure is updated; the update operation is performed in a low-priority thread in the background and does not occupy the computing resources of front-end pose estimation and flight control.

[0108] In another embodiment, using the current keyframe image and initial pose, a visual landmark feature codebook with topological grouping index, and tower semantic topological structure constraint rules as input, a highly reliable absolute pose constraint is obtained, including: Based on the initial pose of the current frame, the spatial range of the camera's view frustum is calculated. Combining the topological grouping index and the secondary spatial grid index of the codebook, candidate landmark groups and local grids within the view frustum are selected. For example, when the UAV is operating directly on the left side of the tower, the landmarks of the left sub-region of the tower group plus the left crossarm group are locked first, compressing the matching retrieval range from the entire codebook to a candidate subset. The computational load is reduced by more than 60%, and the number of candidate landmarks per frame is controlled between 30 and 50, balancing matching accuracy and airborne real-time performance.

[0109] Then, a 256-dimensional superpoint feature descriptor that is completely consistent with the codebook is extracted from the current keyframe, and brute-force matching is performed with the baseline descriptor of the candidate landmark set. The nearest neighbor / second nearest neighbor ratio is used for filtering (the ratio threshold is set to 0.7) to remove fuzzy matching pairs. Then, the cosine similarity of the descriptors is calculated for the remaining matching pairs, and matching pairs with a similarity greater than 0.75 are retained to obtain the initial matching landmark set.

[0110] Next, based on the pole semantic topology rules, a multi-dimensional structural constraint verification is performed on the initial matching set, including: ①Spatial constraint verification within the same group: For landmarks belonging to the same crossarm group, the vertical height difference of the three-dimensional coordinates shall not exceed 0.3 meters (to match the horizontal structural characteristics of the crossarm); for landmarks belonging to the same tower group, the horizontal offset shall not exceed 1.2 times the width of the tower, and the longitudinal coordinates must conform to the increasing order of the tower from bottom to top.

[0111] ② Cross-group subordinate constraint verification: The vertical height difference between the insulator group landmark and the corresponding crossarm group landmark must fall within the range of 0.5~1.5 meters (the standard suspension length range of the matching insulator). Matching pairs that do not conform to the subordinate spatial relationship are directly rejected.

[0112] ③ Overall topology verification: Calculate the relative position matrix between all matching landmarks and compare it with the relative position matrix of the standard topology map. Matching combinations with an overall deviation greater than 0.2 meters are determined to be invalid, and the optimal matching subset is re-selected through a random sampling consensus algorithm.

[0113] When the number of valid matching landmarks that pass all verifications is ≥3, it is determined that a valid loop closure has been triggered. The perspective n-point algorithm is adopted, with the fixed three-dimensional coordinates of the matching landmarks as the three-dimensional reference and the corresponding image pixel coordinates as the observation values, to solve the absolute pose observation value of the current frame relative to the landmarks. The reprojection error of the pose solution is calculated simultaneously. If the error is less than 2 pixels, it is determined to be a valid loop closure constraint, and the constraint edge information with covariance matrix is ​​obtained.

[0114] Furthermore, using the frame-by-frame real-time pose manifold vector, IMU pre-integration results, closed-loop constraint edges, and fixed 3D coordinates of the landmark codebook as input, a standardized node and edge system is set up to construct an incrementally updatable global pose graph, which serves as the optimization carrier for drift correction, including: Two types of node system definition and incremental addition: The pose graph is constructed entirely based on a unified world coordinate system and includes two types of nodes: ① Frame Pose Nodes: Each keyframe corresponds to one node. The node ID corresponds one-to-one with the keyframe ID of the sliding window. The state variable to be optimized for each node is a 7-dimensional pose (3-dimensional spatial position + 4-dimensional unit quaternion rotation). The initial value of the node is directly adopted from the real-time pose manifold vector. For each new keyframe, a corresponding node is synchronously appended to the end of the pose graph without modifying the existing node storage, thus realizing a purely incremental node update.

[0115] ② Fixed landmark nodes: Each codebook entry corresponds to a fixed reference node. The node ID corresponds one-to-one with the unique landmark ID. The node state quantity is 3D world coordinates. This type of node is set to a fixed value throughout the process and does not participate in optimization iteration. It serves as the absolute coordinate reference for global positioning and anchors the global scale and position of the pose map.

[0116] Definition and incremental addition of three types of constraint edges: The pose graph contains three types of constraint edges with information matrices, and all edges participate in subsequent optimization in the form of residuals: ① Inter-frame odometry edge: a directed edge that connects two adjacent frame pose nodes. The constraint value is the relative pose increment between adjacent frames. The information matrix is ​​obtained by inverting the pose covariance matrix output by VIO, which characterizes the reliability of the inter-frame constraint.

[0117] ②IMU pre-integration edge: Connects adjacent frame nodes in parallel with the inter-frame odometer edge. The constraint values ​​are the relative rotation, velocity and displacement increments obtained by IMU pre-integration. The information matrix is ​​derived from the IMU device noise model to supplement high-frequency motion constraints and suppress short-term drift of pure vision.

[0118] ③ Closed-loop constraint edge: Undirected edge, added when a valid closed loop is triggered, connecting the current frame pose node with multiple corresponding fixed landmark nodes. The constraint value is relative pose observation. The information matrix is ​​obtained by normalizing the closed-loop reprojection error and is the core constraint source for global drift correction.

[0119] Based on this, using the incrementally updated global pose graph and flight control stability constraint threshold as inputs, a three-layer progressive optimization strategy of local compact optimization, mid-range smoothing, and global fine-tuning is adopted to correct pose drift. This strategy ensures global consistency while avoiding pose jumps, thus meeting the stability requirements of airborne flight control, including: The first layer selects a local window of 10 frames before and after the closed-loop frame as the first optimization domain. Using the closed-loop constraint as the core anchor point and inter-frame odometry and IMU constraints as the foundation, iterative optimization is performed using the Gauss-Newton method, with an upper limit of 5 iterations. The optimization objective is to minimize the sum of squared residuals of all constraint edges within the window. Hard constraints are added during the optimization process: single-frame translation adjustment should not exceed 0.1 meters, and rotation adjustment should not exceed 1° to avoid sudden changes in local pose. This layer is used to quickly absorb closed-loop deviations, compressing accumulated drift into a local interval and rapidly improving near-device pose accuracy.

[0120] The second layer extends the optimization domain to the middle range of 50 frames before and after the closed-loop frame, performing three iterations with a smaller iteration step size. An additional constraint on the smoothing of pose changes between adjacent frames is added to the optimization objective, forcing a linear transition in pose adjustments between adjacent frames and avoiding obvious pose steps between the optimized and unoptimized regions. This layer smoothly diffuses local pose corrections to the middle trajectory, eliminating abrupt changes in the trajectory at the optimization boundary and ensuring pose continuity during mid-flight.

[0121] The third layer extends the optimization domain to all historical frames and employs an incremental pose graph optimization method. It only updates the states of nodes affected by loop constraints, without performing a complete re-solution of the entire graph. The iteration limit is set to 2, performing only minor smoothing corrections to the global trajectory without altering the relative relationships of local poses. This layer ensures global consistency across the entire inspection trajectory and has low computational load, running in a background idle thread and not consuming computational resources from front-end pose estimation and flight control.

[0122] After each layer of optimization is completed, the latest frame attitude adjustment amount output to the flight controller is verified. If the translation adjustment amount is greater than 0.2 meters or the rotation adjustment amount is greater than 2°, all correction amounts are scaled proportionally to force the stability threshold to be met, ensuring that the attitude received by the flight controller does not change, thus avoiding flight oscillations caused by sudden attitude changes.

[0123] Finally, using the optimized latest frame global pose, multi-level inspection semantic landmark map, and tower coordinate system transformation parameters as input, a globally consistent inspection pose vector is obtained and spatially aligned with the inspection path is completed. Specifically: The optimized latest frame six-DOF pose is used to form a globally consistent inspection pose vector. The first three dimensions are the three-dimensional position coordinates in the world coordinate system, and the last four dimensions are the rotational attitude represented by unit quaternions. The global confidence of the pose is calculated by normalizing and weighting three indicators: the number of effective closed-loop constraints, the closed-loop reprojection error, and the total optimization residual. The value range is 0 to 1.

[0124] In this embodiment, the local coordinate system of the tower is used as a transformation medium to uniformly map the coordinates of all waypoint layer nodes in the semantic roadmap (initially defined based on the local coordinate system of the tower) to the world coordinate system through the tower global pose transformation matrix.

[0125] Next, the three-dimensional Euclidean distance from the current global pose of the UAV to each waypoint layer node in the roadmap is calculated. Combined with the flight path direction, the waypoint that is closest to the path and located ahead is selected as the current target waypoint. The task mapping edge is used to associate with the corresponding task layer node to obtain the inspection rules to be executed and the shooting parameter requirements, and update the inspection task progress.

[0126] When the spatial distance between the UAV and the target waypoint is less than 0.5 meters and the attitude deviation is less than 5°, it is determined that the UAV has reached the target waypoint and the next task node is automatically scheduled.

[0127] When the systematic deviation between the global pose and the waypoints on the road map exceeds 0.3 meters, a small calibration is performed on the overall spatial position of the road map based on the absolute coordinates of the closed-loop landmarks to eliminate the scale deviation of the initial perception reconstruction and ensure that the spatial accuracy of subsequent waypoints matches the global pose.

[0128] S6. Combining the multi-level inspection semantic roadmap with the globally consistent inspection pose vector, extract the visual servo deviation of the target component and integrate it into the model prediction control cost function to generate inspection control commands under the hard constraint of the tower body safety envelope surface.

[0129] In this embodiment, a multi-level inspection semantic roadmap (including task-level node priority, waypoint-level spatial coordinates, and main execution edge topology) is used as input. A globally consistent inspection pose vector, real-time wind speed data from airborne meteorological sensors, and remaining flight control battery power and estimated range are also used to generate the current target task ID and corresponding waypoint parameters, a dynamically adjusted task execution sequence, and an emergency state trigger flag. This enables the orderly advancement of inspection tasks and adaptive environmental adjustments. Specifically: The current global pose of the UAV is projected onto the main execution edge path of the roadmap, and the percentage of executed mileage along the path is calculated. Combined with the topological order of adjacent waypoint nodes, the current task segment and the next target waypoint are determined. The localization process does not use simple nearest neighbor matching; instead, it uses topological order as a hard constraint, performing distance matching only within the range of the two waypoints before and after the current task segment. This avoids task jumps and scheduling chaos caused by pose fine-tuning and local detours.

[0130] Under normal circumstances, tasks are scheduled according to the task priority and execution order of the roadmap, advancing waypoints sequentially. A two-dimensional arrival determination rule is set: when the three-dimensional Euclidean distance between the UAV and the target waypoint layer node is ≤0.5m, and the yaw and pitch angle deviations between the camera attitude and the target waypoint attitude are both ≤5°, it is determined that the current waypoint has been reached, the corresponding task layer node is marked as completed, and the next task node is automatically scheduled.

[0131] Then, the task sequence is dynamically trimmed based on multi-level wind speed. A three-level scheduling strategy is set based on real-time wind speed data, which is fully aligned with the fault priority system: wind speed ≤5m / s is normal level, and all inspection tasks are executed in full; 5-8m / s is medium risk level, and the third-level low-priority tower main material inspection tasks are removed, while the first-level (insulator strings, tension clamps) and second-level (crossarm connection points, equalizing rings) high-value tasks are retained; ≥8m / s is high risk level, and only the first-level core component inspection tasks are retained, while all other tasks are marked as pending reshoots and postponed until the weather conditions are suitable. After task trimming, the remaining tasks are rearranged according to the original topology order to ensure that the core inspection tasks are implemented first.

[0132] In this embodiment, the remaining range is estimated by combining the remaining battery power and energy consumption per unit distance, and two levels of battery power constraints are set: when the remaining battery power is ≤20% and insufficient to cover the remaining mission range, the missions are cut off one by one in order of priority from low to high, and priority is given to ensuring the completion of inspection of high-priority components; when the remaining battery power is ≤15%, a forced return flag is triggered, the current mission is interrupted directly, and the drone is scheduled to fly to the safe return point of the auxiliary layer along the shortest path. All cut missions retain the mission ID and status mark, which supports automatic access to the sequence when reshooting later.

[0133] The task scheduling of this invention does not use the shortest path as the sole criterion. Instead, it takes the fault priority of power grid equipment, on-site weather conditions, and airborne energy status as scheduling constraints, and directly embeds the business objective of prioritizing the investigation of core hidden dangers into the task scheduling logic, rather than executing the fixed path sequence of general UAVs.

[0134] Furthermore, using real-time RGB images from the airborne camera, inspection standard parameters bound to the current target task (including standard shooting distance, component pixel ratio range, and shooting pitch / azimuth angle range), a semantic segmentation model of the power components, and the camera intrinsic parameter matrix as input, the three-dimensional visual deviation vector and deviation confidence in the camera coordinate system are obtained, including: 1. Semantic mask extraction and geometric feature calculation for target components: Based on a semantic segmentation network and semantic ID system, the current frame is segmented at the pixel level to extract the semantic mask corresponding to the target component. Three types of geometric features are calculated for the mask region: first, the centroid coordinates of the pixels in the mask region, representing the position of the component in the image; second, the width and height of the minimum bounding rectangle of the mask, representing the scale of the component in the image; and third, the main orientation angle of the minimum bounding rectangle of the mask, representing the deflection angle of the component in the image.

[0135] 2. Ideal composition parameter calibration for inspection standard mapping: This process maps the inspection standard requirements to ideal composition parameters on the image plane. The ideal centroid is located at the image center, corresponding to the standard requirement of centering the component. The ideal circumscribed rectangle size is calculated by taking the median value of the standard pixel percentage range (e.g., 20% for insulator strings, given a standard percentage of 15%-25%). The ideal main orientation angle is calculated based on the standard shooting angle requirements. All ideal parameters correspond one-to-one with the inspection standard, with no custom deviations.

[0136] 3. Quantitative calculation of three types of visual deviations: Based on actual geometric features and ideal composition parameters, three types of deviations that can be directly controlled are calculated: ① Position deviation: Centroid pixel deviation is converted into horizontal and vertical translation deviations in the camera coordinate system through camera intrinsic parameters, in meters; ② Scale deviation: The ratio of actual proportion to ideal proportion, combined with the current initial depth value, is converted into forward and backward translation deviations in the camera optical axis direction, in meters; ③ Angular deviation: The angle between the mask's main orientation and the ideal orientation is decomposed into rotational deviations in the yaw and pitch directions, in degrees. The three types of deviations together constitute a three-dimensional visual deviation vector.

[0137] 4. Occlusion perception bias confidence calibration: Calculate the occlusion ratio of the target component mask, i.e. the percentage of pixels covered by other component masks and background areas; when the occlusion rate is <10%, the bias confidence is 1.0, and it decreases linearly in the occlusion rate range of 10%-30%. When the occlusion rate is >30%, the confidence is set to 0. The bias of this frame does not participate in subsequent control feedback to avoid erroneous observations caused by occlusion interfering with flight attitude.

[0138] In this embodiment, visual deviations are generated directly by the inspection standards. Each deviation corresponds to a specific inspection operation specification, rather than arbitrary target tracking by general visual servoing. This ensures the consistency between the control target and the inspection business requirements from the feedback source.

[0139] Furthermore, using the 3D visual deviation vector, current global pose and velocity, UAV quadrotor dynamics model, tower safety envelope geometric parameters, and UAV flight performance limit parameters as inputs, three-axis velocity commands and yaw rate commands in the control time domain, as well as predicted trajectory sequences, are generated to achieve an adaptive balance between flight efficiency and image quality while ensuring safety. This includes: The optimized state variables are set to 10 dimensions: 3D position, 3D linear velocity, 3D attitude represented by Euler angles, and yaw rate in the world coordinate system; the control variables are set to 4 dimensions: three-axis velocity commands and yaw rate commands. The prediction time domain is set to 10 steps, the control time domain is set to 5 steps, the sampling period is 0.1s, and the control frequency is matched to the 10Hz control frequency of the airborne flight controller, balancing prediction accuracy and real-time computation.

[0140] Then, a multi-objective cost function is constructed. The total cost function consists of a weighted sum of three terms: ① Trajectory tracking term: the sum of squares of the deviations between the predicted pose and the target waypoint pose, representing the waypoint tracking accuracy; ② Image quality term: the sum of squares of the magnitudes of the visual deviation vectors, representing the compliance of the image composition; ③ Control smoothing term: the sum of squares of the changes in control variables in adjacent steps, representing flight stability.

[0141] When the distance between the drone and the target waypoint is greater than 3m, it is in cruise mode, with the weight of the trajectory tracking item set to 1.0, the weight of the shooting quality item set to 0.2, and the maximum allowable flight speed set to 3m / s, prioritizing inspection efficiency; when the distance is ≤3m, it is in shooting fine-tuning mode, with the weight of the shooting quality item increased to 1.5, the weight of the trajectory tracking item decreased to 0.5, and the maximum flight speed limited to 0.8m / s, prioritizing shooting composition accuracy, and achieving scene adaptive control with fast cruise and stable shooting.

[0142] Furthermore, two types of constraints are introduced into the optimization problem as non-relaxable hard constraints: ① Tower safety envelope constraint: all spatial points on the predicted trajectory must be located outside the tower safety envelope, and the minimum distance to the envelope must be ≥0.5m. Violation of this constraint will directly determine the solution as infeasible; ② Flight performance constraint: three-axis velocity ≤3m / s, three-axis acceleration ≤2m / s². 2 The yaw rate is ≤15° / s, which meets the physical limits of quadcopter drones. If no feasible solution is found in the optimization, the image quality weights are relaxed first, ensuring that safety and performance constraints are not exceeded.

[0143] Finally, the nonlinear optimization problem is transformed into a standard quadratic programming form for solution. The single-step solution time is controlled within 5ms, which meets the requirements of airborne real-time computing. After the solution is completed, only the control quantity of the first step in the control time domain is output for execution. The next cycle is based on the updated state to solve again, realizing rolling optimization and ensuring the continuity and disturbance resistance of control.

[0144] In this embodiment, the control sequence output by the MPC, the double threshold for image capture, the image with synchronization timestamp, and the raw IMU data can also be used as inputs to output flight control execution commands, inspection datasets with semantic labels, and feedback data streams from the front-end perception and positioning modules, forming a closed-loop control: The obtained control sequence is hardware-time-stamped with the flight control system clock, and the triaxial speed and yaw rate commands are sent to the flight control actuators via the onboard CAN bus. A rolling mechanism is used during execution, updating the control commands every 0.1 seconds to always use the optimal control values ​​in the latest state, avoiding control deviations caused by command lag.

[0145] The system employs a dual-condition triggering system for shooting: posture and vision. When both conditions are met simultaneously and for more than two consecutive frames, the system automatically triggers high-definition camera shooting. The conditions are: ① Posture condition: The three-dimensional distance between the UAV and the target waypoint is ≤0.3m and the attitude deviation is ≤3°; ② Vision condition: All three types of visual deviations fall within the allowable range of the inspection standards.

[0146] After the images are captured, they are encapsulated with structured tags. The tag fields include: unique tower ID, component semantic ID, global pose at the time of capture, capture timestamp, and matching confidence, so as to achieve traceability of one image corresponding to one component and one image with one pose, which fully matches the data archiving specifications of power grid inspection.

[0147] The real-time acquired images and IMU data are synchronously transmitted back at a fixed frequency to form a closed loop. When the visual deviation exceeds the standard threshold for 3 consecutive frames or the number of effective feature tracking points is lower than the threshold, the abnormal status is immediately reported to the task scheduling module, triggering the UAV to hover in place and activating the pose degradation protection mechanism. If the status cannot be recovered for 10 seconds, the UAV is automatically scheduled to fly to the nearest emergency hovering point. The task will continue to be executed after the status is recovered, thus ensuring flight safety from the control level.

[0148] This invention is applied to the autonomous inspection of overhead power transmission lines by unmanned aerial vehicles (UAVs), and is particularly suitable for operational conditions where there are no pre-mapped coordinates or preset flight routes. The UAV is equipped with a visible light camera and an inertial measurement unit. When flying over power poles of unknown types, it can autonomously complete pole type identification, inspection standard loading, inspection path planning, high-precision positioning, and closed-loop flight control, outputting inspection image data with precise pose labels.

[0149] It should be noted that the specific implementation methods described above, such as image processing, numerical simulation, and the construction and training of machine learning models, can all be accomplished by the processor by calling the corresponding computer program instructions stored in memory. Those skilled in the art can implement the above functions using algorithms and tools known in the prior art, according to actual needs.

[0150] Please see Figure 2 In an embodiment, to efficiently execute the visual measurement-based autonomous UAV inspection method provided by the present invention, the present invention also provides a visual measurement-based autonomous UAV inspection system, comprising: an input device 1, an output device 2, a processor 3, and a memory 4, wherein the input device 1, output device 2, processor 3, and memory 4 are interconnected, and the memory 4 stores program instructions for executing the steps of the visual measurement-based autonomous UAV inspection method. The visual measurement-based autonomous UAV inspection system of the present invention has a compact structure and stable performance, and can stably execute the visual measurement-based autonomous UAV inspection method of the present invention, further improving the overall applicability and practical application capability of the present invention.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the present invention.

Claims

1. A method for autonomous inspection of unmanned aerial vehicles (UAVs) based on visual measurement, characterized in that, Includes the following steps: A semantic topology graph is constructed based on the physical connection rules of power poles, and the pole type and the pole semantic topology descriptor are obtained through anisotropic graph convolutional neural network reasoning. A three-level inspection standard knowledge graph is constructed, and the knowledge graph is encoded into a spatially weighted inspection standard semantic constraint vector, and cross-modal semantic alignment is completed with the tower semantic topology descriptor. Using real-time tower semantic topology map as data map and standard tower topology map as model map, the distribution of inspection rules to components is completed by injecting topology order prior probabilistic graph matching. Combined with fault priority weight and tower body safety envelope constraint, a multi-level inspection semantic roadmap is generated. Tower body plane parameters extracted from the tower semantic topology descriptor are introduced as pseudo-measurement constraints. The constraint weights are adjusted by distance and observation quality to obtain the real-time pose manifold vector. An incremental global pose graph is constructed by using real-time pose manifold vectors as frame nodes and visual landmark feature codebooks as fixed reference nodes. Then, a hierarchical incremental optimization is used to correct pose drift and obtain a globally consistent inspection pose vector. By combining the multi-level inspection semantic roadmap with the globally consistent inspection pose vector, the visual servo deviation of the target component is extracted and incorporated into the model predictive control cost function, and inspection control commands are generated under the hard constraint of the tower body safety envelope surface.

2. The UAV autonomous inspection method based on visual measurement according to claim 1, characterized in that, The process of constructing a semantic topology graph based on the physical connection rules of power poles, and obtaining the pole type and semantic topology descriptor through anisotropic graph convolutional neural network inference, includes the following steps: Based on continuous visible light images during the drone's approach to the tower, the camera pose and sparse 3D point cloud of the tower are recovered by multi-view geometry method, and the 2D semantic labels are projected onto the 3D point set by combining semantic segmentation of power components. Each semantic component is clustered into a topological graph node. The node attributes include semantic category, three-dimensional centroid, bounding box size and main orientation unit vector. Topological edges are constructed using the standard physical connection rules of power poles as initial constraints. The edge connections are corrected by combining spatial adjacency relationships to generate a semantic topological graph with four types of labels: vertical connection, suspended connection, parallel connection and terminal connection. The semantic topology graph is inferred by a direction-aware convolutional neural network to obtain the tower type classification result. The tower's central axis is fitted to construct a local coordinate system. The spatial coordinates and orientations of all nodes in the topology graph are transformed to the local coordinate system. The scale is normalized based on the total height of the tower to generate a semantic topology descriptor for the tower.

3. The UAV autonomous inspection method based on visual measurement according to claim 1, characterized in that, The construction of a three-level inspection standard knowledge graph, encoding the knowledge graph into a spatially weighted inspection standard semantic constraint vector, and aligning it with the tower semantic topology descriptor across modalities includes the following steps: Using the component semantic category list of the pole semantic topology descriptor as the entity scope constraint, three types of structured entities are extracted from the inspection standard text: inspection target components, inspection quality requirements, and quantitative shooting parameters. A three-level hierarchical knowledge graph is constructed with the top layer as pole type nodes, the middle layer as component category nodes, and the bottom layer as rule parameter nodes for quality requirements and shooting parameters. The nodes are connected by subordinate edges, constraint edges, and association edges, where the association edges correspond one-to-one with the edge types of the semantic topology graph. Encode the complete inspection rules corresponding to each component into a component sub-semantic vector. After assigning weight coefficients according to the component fault priority set by the power grid historical fault statistics, splice all component sub-vectors along the tower topology order and superimpose the tower type global vector to generate the inspection standard semantic constraint vector. By correcting the matching misalignment caused by viewpoint occlusion through topological consistency, the standard constraint vector is mapped to the visual feature space through linear transformation, thus completing cross-modal alignment.

4. The UAV autonomous inspection method based on visual measurement according to claim 1, characterized in that, The process involves using a real-time tower semantic topology map as the data map and a standard tower topology map as the model map. By injecting topological order priors into probabilistic graph matching, inspection rules are assigned to components. Combined with fault priority weights and tower safety envelope constraints, a multi-level inspection semantic roadmap is generated. This includes the following steps: The real-time perceived semantic topology map of the poles and towers is used as the data map, and the topology map of the same type of standard poles and towers is used as the model map. Two types of topological prior probabilities, namely vertical order and horizontal arrangement, are injected into the probabilistic graph matching. The confidence propagation algorithm is used to solve the global optimal node correspondence and map the inspection rules to the actual pole and tower components. Each component to be inspected is abstracted as a task node, and a multi-objective optimization function containing path length cost and priority penalty cost is constructed. Based on the high monotonicity and path safety constraints, a global task waypoint sequence with execution order is obtained by solving the function. Based on the tower component enclosure box and the legal safety distance, a tower safety envelope is constructed. Candidate viewpoints are sampled in the hemispherical space around the component. The viewpoint quality is evaluated by shooting parameter compliance, visibility occlusion rate, safety margin, and attitude smoothness. The optimal shooting viewpoint pose for each component is then obtained. A three-level node system consisting of a task layer, a waypoint layer, and an auxiliary layer is constructed. Four types of topological edges—main execution edge, task mapping edge, emergency bypass edge, and global return edge—are used to generate a multi-level inspection semantic roadmap with complete business semantics and security constraints.

5. The UAV autonomous inspection method based on visual measurement according to claim 1, characterized in that, The process of introducing tower plane parameters extracted from the tower semantic topology descriptor as pseudo-measurement constraints, and adjusting the constraint weights through distance and observation quality to obtain the real-time pose manifold vector includes the following steps: The three-dimensional plane parameters of each facade of the tower are fitted from the semantic topological descriptor of the tower as a prior plane set; combined with the semantic mask of the tower in the real-time image and the initial pose of the current frame, the observed plane parameters are calculated, and the plane correspondence is completed by matching the angle between the normal vectors to obtain the effective plane constraint; A pseudo-measurement error term for the tower plane, consisting of the normal vector rotation residual and the distance translation residual, is constructed and optimized using a sliding window. The Levenberg-Marquardt algorithm is used to perform joint nonlinear optimization, and the dimension is reduced by Schur complement elimination to obtain the real-time pose manifold vector.

6. The UAV autonomous inspection method based on visual measurement according to claim 5, characterized in that, The weight of the pseudo-measurement error term is obtained by multiplying the distance factor and the observation quality factor; The distance factor is calculated based on the distance from the current camera to the plane of the tower. The observation quality factor is calculated based on the proportion of tower body mask pixels in the current frame and the confidence level of edge detection.

7. The UAV autonomous inspection method based on visual measurement according to claim 1, characterized in that, Extracting the visual landmark feature codebook includes the following steps: Based on semantic 3D point clouds, a candidate landmark set is obtained by using rigid component point clouds for three-level filtering. Each candidate landmark is projected onto a multi-view image to crop a local image patch, and a superpoint network with fine-tuned tower scene is used to extract feature descriptors, forming a set of multi-view descriptors for each landmark; Candidate landmarks are pre-divided into tower body group, crossarm group, insulator group, and hardware group according to the tower topology. Feature clustering is performed independently within each group to obtain the visual landmark feature codebook.

8. The UAV autonomous inspection method based on visual measurement according to claim 1, characterized in that, The process of constructing an incremental global pose graph using real-time pose manifold vectors as frame nodes and visual landmark feature codebooks as fixed reference nodes, and then using hierarchical incremental optimization to correct pose drift and obtain globally consistent inspection pose vectors includes the following steps: Based on the current frame pose and the range of the view frustum, candidate landmarks are screened through the visual landmark feature codebook. Mismatches are eliminated through three levels of verification: same-group spatial constraints, cross-group subordinate constraints, and overall topological constraints. When the number threshold is met, a valid closed loop is determined and the relative pose observation is solved. A two-type node system, including frame pose nodes and fixed landmark nodes, is constructed. Based on the constraint edges, a linked storage structure is used to realize the pure incremental update of nodes and edges. The pose drift is corrected by a progressive optimization strategy, and a hard constraint on the pose adjustment amount is set for each layer to obtain a globally consistent inspection pose vector.

9. The UAV autonomous inspection method based on visual measurement according to claim 1, characterized in that, The process of combining the multi-level inspection semantic roadmap with the globally consistent inspection pose vector, extracting the visual servo deviation of the target component and incorporating it into the model predictive control cost function, and generating inspection control commands under the hard constraint of the tower body safety envelope surface includes the following steps: By combining the multi-level inspection semantic roadmap with the globally consistent inspection pose vector, inspection tasks are dynamically scheduled based on weather and power status. The inspection standards are transformed into visual servo biases, which are then incorporated into the model predictive control cost function. The weights are adjusted in stages to generate smooth control commands under the safety envelope constraint.

10. A visual measurement-based autonomous inspection system for unmanned aerial vehicles (UAVs), characterized in that, The vision-based autonomous inspection system for unmanned aerial vehicles (UAVs) includes an input device, an output device, a processor, and a memory. The input device, output device, processor, and memory are interconnected. The memory stores program instructions, which are used to execute the vision-based autonomous inspection method for UAVs according to any one of claims 1-9.

Images