Power transmission corridor parameterization reconstruction method fusing nonlinear geometric prior and semantic topology

Abstract

The invention relates to a power transmission corridor parameterization reconstruction method fusing nonlinear geometric prior and semantic topology, and the method comprises the steps: obtaining point cloud data, image data and IMU inertial data of a power transmission corridor, carrying out the adaptive enhancement, and obtaining a high-dynamic-range image and time-synchronized point cloud data; based on the high dynamic range image, performing semantic segmentation by adopting a Wire-Aware BiSeNetV2 network to obtain a pixel-level semantic segmentation result, and dividing the point cloud data subjected to time synchronization into multiple types of semantic tags; based on the point cloud data with the semantic tag, a semantic-guided decoupling state estimation method is adopted for processing, and the six-degree-of-freedom pose of the unmanned aerial vehicle is obtained; and based on the six-degree-of-freedom pose of the unmanned aerial vehicle, fitting is carried out by adopting a catenary constrained parameterized mapping method, and drift correction is carried out to obtain parameters of the power transmission corridor wire. Compared with the prior art, the method has the advantages of high precision, high robustness and the like.

Description

Technical Field

[0001] This invention relates to the field of parameter reconstruction technology, and in particular to a parameterized reconstruction method for power transmission corridors that integrates nonlinear geometric priors and semantic topology. Background Technology

[0002] With the development of drone technology, autonomous navigation and environmental reconstruction technologies based on multi-sensor fusion (LiDAR, vision, IMU) have been widely applied in power transmission corridor inspections. However, power transmission corridors are typical unstructured, highly dynamic, and geometrically degraded environments, and existing SLAM (Simultaneous Localization and Mapping) technologies still face severe challenges in practical applications.

[0003] Firstly, in terms of visual perception, power line inspections typically employ an upward-looking perspective, resulting in extreme dynamic range contradictions within the field of view (strong background sky versus weak foreground backlit towers). Traditional automatic exposure algorithms, based on full-image statistics, are easily misled by bright skies, leading to reduced exposure and causing critical dark areas of the towers to fall into sensor dead zones, resulting in loss of texture features and consequently, loss of visual odometry lock.

[0004] Secondly, in terms of geometric modeling, existing SLAM systems are generally based on the "rigid body assumption of the environment". However, power transmission lines are semi-rigid objects, which will gallop due to wind force, thus violating the rigid body constraints. At the same time, the slender wires have translational invariance along the axial direction, resulting in the state along the line being unobservable (geometric degradation), which easily leads to unbounded cumulative drift (corridor effect).

[0005] Finally, regarding loop closure detection, insulator strings and towers with highly repetitive shapes are widely present in transmission corridors. This repetitive texture leads to severe "perceptual aliasing," making loop closure detection algorithms based on the traditional bag-of-words model prone to mismatches, resulting in backend optimization failures and making it difficult to build a globally consistent, high-precision map.

[0006] Therefore, there is an urgent need for a parametric reconstruction method for power transmission corridors that can adapt to strong backlight environments, solve the problems of non-rigid bodies and geometric degradation, and effectively resist aliasing. Summary of the Invention

[0007] The purpose of this invention is to provide a parametric reconstruction method for power transmission corridors that integrates nonlinear geometric priors and semantic topology to improve the accuracy of parameters.

[0008] The objective of this invention can be achieved through the following technical solutions: A parametric reconstruction method for power transmission corridors that integrates nonlinear geometric priors and semantic topology includes: The point cloud data, image data, and IMU inertial data of the power transmission corridor are acquired and adaptively enhanced to obtain high dynamic range images and time-synchronized point cloud data. Based on the high dynamic range image, semantic segmentation is performed using the Wire-Aware BiSeNetV2 network to obtain pixel-level semantic segmentation results, and the time-synchronized point cloud data is divided into multiple semantic labels; Based on point cloud data with semantic labels, a semantically guided decoupled state estimation method is used to process the data to obtain the six-degree-of-freedom pose of the UAV. Based on the six-degree-of-freedom pose of the UAV, a parametric mapping method with catenary constraints is used for fitting and drift correction to obtain the parameters of the power transmission corridor conductor.

[0009] Furthermore, the step of performing adaptive enhancement includes: Combining the IMU inertial data, the point cloud from the previous frame is... Transform from radar coordinate system to current camera coordinate system The three-dimensional coordinates under the current camera coordinates are obtained, and are represented as follows: , In the formula, For the present 3D coordinates of the point cloud at any given time in the current camera coordinate system , It is an extrinsic parameter matrix. The pose at the current moment is predicted from IMU inertial data. The pose of the previous frame. This is the extrinsic parameter matrix; Based on the aforementioned three-dimensional coordinates, the pinhole model is used to extract the point cloud from the previous frame. Projecting the image onto a two-dimensional image plane yields the projected pixel coordinates, forming a sparse mask image with the same resolution as the image data. The expression for calculating the projected pixel coordinates is as follows: , In the formula, These are the projected pixel coordinates. For projection function, This is the camera intrinsic parameter matrix; The sparse mask image Based on the reflectivity intensity of the projection point Determine the effective statistical domain ROI: And calculate the weighted average reflectance of the effective statistical domain ROI. The effective statistical domain represents the tower area; A reflectivity-exposure feedforward control loop is used to calculate the optimal exposure time for the next frame. The calculation expression is: , In the formula, To determine the optimal exposure time for the next frame, The exposure time currently being used in the current frame. The preset target brightness value, The sparse mask image from the previous frame. Average gray level within, / The LiDAR-weighted average reflectance of the previous and current frames. This is the sensitivity coefficient; Based on the optimal exposure time of the next frame Data acquisition is performed to obtain high dynamic range images and time-synchronized point cloud data.

[0010] Furthermore, the semantic tags are categorized as follows: rigid tower point, semi-rigid power line point, and invalid background point.

[0011] Furthermore, the step of obtaining the six-DOF pose of the UAV includes: 1) Based on the point cloud data with semantic labels, construct corresponding observation models according to different semantic labels; 2) For the observation model, iterative loops are performed within the decoupled error state iterative Kalman filter framework, and the Kalman gain is calculated. The Kalman gain mentioned above The calculation expression is: , in: , In the formula, Let the prior error state covariance matrix be... To measure the Jacobian matrix, It is a diagonal matrix. It is the variance function. For the first semantic tags; 3) Based on the Kalman gain The observation residuals calculated by the observation model are integrated to calculate the error state update amount and correct the UAV state. 4) Determine whether the iteration termination condition has been met. If yes, the corrected UAV state is the UAV's six-degree-of-freedom pose. If no, return to step 1) to update the observation model and perform iterative calculations until the termination condition is met.

[0012] Furthermore, the observation model comprises two parts: a residual model and a variance function, wherein, For rigid tower points, the residual model is a point-to-surface residual model, expressed as: , In the formula, This is a point-to-plane residual model, representing the distance residual from a point to a plane. It is a plane normal vector. For the current observation point, The corresponding point on the map; Its variance function is expressed as: , In the formula, To observe the variance, The fundamental physical ranging noise of LiDAR sensors, For the first semantic tags, For rigid tower points; For a semi-rigid electric line point, its residual model is a point-to-line residual model, expressed as: , In the formula, For the point-to-line residual model, it represents the perpendicular distance residual from a point to a line. It is the identity matrix. Let be the unit direction vector of the line. The coordinates of the current measured power line point cloud. For any point on the corresponding line, The projection matrix; Its variance function is expressed as: , In the formula, To observe the variance, The fundamental physical ranging noise of LiDAR sensors, The coefficient of thermal expansion is 1 / 3. For wind speed, A function based on wind speed. For semi-rigid electric field line points; For invalid background points, their variance function is set to To completely remove invalid background points, among which Invalid background point.

[0013] Furthermore, for semi-rigid electric line points, the method further includes: introducing a robust kernel function to reweight the variance function, wherein the reweighted residual function is expressed as: , In the formula, The residual function after reweighting. The original residual value, This is the threshold parameter.

[0014] Furthermore, the step of obtaining the parameters of the transmission corridor conductor includes: Set up a sliding window ,in The length of the sliding window. For the first t Power line point cloud data frames at any given time; Using the prior pose at the current moment Transform all electric field point clouds within the sliding window to the world coordinate system to obtain: , In the formula, For the first transformation to the world coordinate system within the sliding window i A point cloud of power lines, Before the conversion within the sliding window i A point cloud of power lines; Based on the point cloud of all electric power lines to the world coordinate system, a nonlinear least squares-based objective function is constructed to minimize the sum of orthogonal distances from all observation points to the catenary model. The expression for the objective function is: , in: , , , In the formula, This is the optimal catenary parameter vector. The state variables to be optimized are: The coordinates of the lowest point of the catenary in the world coordinate system are: Let be the azimuth angle of the vertical plane containing the catenary relative to the X-axis of the world coordinate system. These are the shape parameters of the catenary. For horizontal tension, Weight per unit length This represents the total number of power line point clouds contained within the sliding window. For the residual function, Let be the perpendicular distance from the point to the two-dimensional vertical plane. This refers to the height residual between the observed altitude of the UAV in its current pose and the theoretical catenary model during the ESIKF update. The horizontal projection distance of the point in the vertical plane containing the traverse; The expression for the catenary model is as follows: , In the formula, For the horizontal coordinate The theoretical calculated height of the conductor. It is a hyperbolic cosine function; In the error state iterative Kalman filter framework update, the objective function based on nonlinear least squares is solved iteratively to obtain the optimal catenary parameter vector. ; This also includes the following: Based on the height residual Calculate its Jacobian matrix relative to the UAV's position along the line, expressed as: , The Jacobian matrix is ​​introduced as an observation constraint into the ESIKF update process: , In the formula, For Jacobian matrices, The height residual in the plane, The location of the drone along the route. This is the positional correction amount along the direction of the transmission line. To correct the gain, This represents the actual altitude of the drone or observation point in the world coordinate system at the current moment. To determine the optimal catenary parameters based on the fitted data .

[0015] Furthermore, it also includes loop closure detection and global optimization steps based on topological fingerprinting, specifically including: Using the RANSAC algorithm combined with the surface normal direction constraint of the insulator point cloud, the central axis of the insulator is extracted from the insulator point cloud and used as the insulator skeleton. Project the insulator point cloud onto the coordinate system of the insulator's central axis, and calculate the radius distribution function along the insulator's central axis. , is represented as: , In the formula, The projection coordinates of the point cloud points in a plane perpendicular to the central axis; For the radius distribution function Perform a Fast Fourier Transform to obtain the frequency domain fingerprint, including the dominant frequency. Phase Amplitude spectrum, of which the dominant frequency Corresponding insulator disc spacing phase The amplitude spectrum corresponding to the axial offset of the insulator string relative to the starting point is determined by the specific shape of the disc, serving as the unique fingerprint of the insulator string; Construct an insulator topology graph, where the insulator skeleton serves as a node, the node also includes the frequency domain fingerprint, and the connection relationship serves as an edge; When the drone flies over an area, it records the current position and a sequence of insulator skeletons of a series of insulators. When it flies to a new position, it uses the current skeleton sequence as the query sequence. Using the Needleman-Wunsch sequence alignment algorithm, the query sequence is compared with all historical insulator skeleton sequences in the insulator topology map. If the similarity score between a historical insulator skeleton sequence and the query sequence exceeds a threshold, a loop closure is determined to have occurred. Global optimization is then performed through geometric correction to complete the loop closure detection process. Geometric correction is performed using the following formula: , In the formula, This represents the precise axial displacement after phase-locking correction. This is a rough displacement estimated by the front-end odometer. This is the rounding function. The physical spacing between insulator discs. The current observed phase angle is extracted using FFT. To convert the phase angle into a scaling factor for physical distance, It represents the displacement within the period.

[0016] Furthermore, the Wire-Aware BiSeNetV2 network embeds a lightweight coordinate attention mechanism at the end of the standard detail branch, which performs global pooling on the feature map obtained at the end of the standard detail branch along the X / Y axes respectively.

[0017] Furthermore, the Wire-Aware BiSeNetV2 network is trained using a boundary-class weighted hybrid loss function, the expression of which is: , In the formula, For boundary-class weighted mixed loss function, To uncover losses in online difficult cases, For balance coefficient, For boundary loss; in: , , In the formula, The total number of pixels in the difficult example. For the selected set of difficult-to-find pixels, For pixels Category The true label, Predict pixels for the network Category The probability, This is the edge probability map predicted by the network. This is a true edge label map.

[0018] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention first extracts multiple semantic labels through semantic segmentation, then uses decoupled state estimation to calculate the pose of high-frequency and high-precision UAVs, and combines the parametric mapping method of catenary constraint to fit the conductor parameters. Furthermore, through drift correction, more accurate transmission corridor parameters are obtained.

[0019] (2) This invention maps the three-dimensional geometric information sensed by the lidar onto the two-dimensional image of the camera through cross-modal spatiotemporal projection, thereby achieving the alignment of "light" and "geometry". It also determines the effective statistical domain through a geometric saliency mask, forcibly filters out the interference of the sky background, and calculates the optimal exposure time through a reflectivity-exposure feedforward control loop. This invention achieves the breakthrough of the "semantic blind zone" of the visual sensor by utilizing the active perception capability of the radar point cloud, thereby achieving light-geometry consistency.

[0020] (3) By improving the Wire-Aware BiSeNetV2 network architecture, this invention embeds a lightweight coordinate attention mechanism at the end of its detail branches, which can forcibly "activate" the spatial structural features of power lines with long-distance dependence, preventing them from being submerged by the background. At the same time, it uses a boundary-class weighted hybrid loss function to solve the problems of small conductors being easily ignored and tower edges being blurred.

[0021] (4) The present invention performs cyclic iteration in the error state iterative Kalman filter framework to achieve deep reconstruction of all observation models, solves problems such as non-rigid body interference, and improves the accuracy of UAV pose estimation.

[0022] (5) This invention solves the problem of unstable parameter estimation caused by the sparse scanning of single-frame LiDAR point cloud by introducing a catenary model as a physical constraint and combining it with a sliding window-based nonlinear optimization and drift correction mechanism.

[0023] (6) The present invention also designed a closed-loop detection and global optimization based on topological fingerprint. By extracting the structure-frequency descriptor, the unique fingerprint of the insulator string is obtained. By using loop closure detection and phase locking anti-drift mechanism, the problem of "perception aliasing" between vision and lidar is avoided. Sequence comparison is used to solve similarity interference and phase locking is used to solve axial drift, thereby achieving robust and high-precision closed-loop detection. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the Wire-AwareBiSeNetV2 network structure of the present invention. Detailed Implementation

[0025] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0026] This embodiment provides a parametric reconstruction method for power transmission corridors that integrates nonlinear geometric priors and semantic topology, such as... Figure 1 As shown, the method includes the following steps: 1. LiDAR-linked adaptive exposure system based on "optical-geometric consistency" 1.1 Theoretical Basis: Active Sensing-Driven Radiation Control Paradigm The primary challenge faced by visual sensors in power line inspection stems from the extreme dynamic range constraint. Because inspection drones often employ a low-angle view, a strong background (sky) is inevitably created in the field of view. Lux) and weak foreground (backlit tower) The binary opposition structure of "Lux".

[0027] This structure directly leads to the systematic failure of traditional automatic exposure (AE) algorithms. Traditional AE is essentially a "reactive" control based on full-image statistics. Because it cannot distinguish between "invalid background" and "valid foreground," the algorithm is misled by large areas of bright sky, thus erroneously and drastically reducing the exposure time. The ultimate consequence of this causal chain is that the critical dark areas of the tower enter the sensor's "dark dead zone," the texture signal-to-noise ratio drops to zero, and consequently, the VIO front end loses its lock.

[0028] To address this pain point, the core logic of this solution lies in leveraging LiDAR's active sensing capabilities to overcome the "semantic blind spots" of visual sensors and construct a "light-geometric consistency" model. This model relies on the following two physical assumptions to transform exposure control from "blind statistics" to "precise mapping": 1) Geometric Existence Assumption: Any region where LiDAR provides an echo is a region with a physical structure (tower, wire), and is also a region where SLAM needs to extract texture features; any region where LiDAR provides no echo (sky) is a geometrically invalid region, and its luminosity information is meaningless to SLAM.

[0029] 2) Spectral Correlation Hypothesis: Although visible light differs from the 905nm near-infrared band of LiDAR, industrial materials such as corroded steel, insulators, and aluminum stranded wire exhibit a strong monotonically positive correlation in reflectivity / Albedo across the two bands. Therefore, the reflectivity map of LiDAR can serve as a "predictive photometric map".

[0030] 1.2 Algorithm Implementation Process 1.2.1 Spatiotemporal Alignment In this embodiment, the UAV's lidar, camera, and IMU are used to simultaneously collect data in three modes: point cloud data, image data, and IMU inertial data of the power transmission corridor.

[0031] In order to use point clouds to guide visual exposure, the 3D point cloud must be accurately mapped onto the 2D image plane.

[0032] Predicting the current camera time using Error State Iterative Kalman Filtering (ESIKF) in a Laser Inertial Odometry Framework position The previous radar point cloud frame will be displayed. Transform from radar coordinate system to current camera coordinate system And using pinhole model projection: , , In the formula, :current At any given moment, the three-dimensional coordinates of the point cloud in the current camera coordinate system. . : Extrinsic parameter matrix (Body to Camera). Represents the rigid body transformation from the IMU coordinate system to the camera coordinate system, usually obtained from calibration. : Current pose. The pose of the machine in the world coordinate system at any given time is predicted by ESKF. : The inverse matrix of the current pose, used to convert world coordinates back to the current body coordinates. : Pose at the previous frame. Previous radar point cloud acquisition time. The body position. : Extrinsic parameter matrix (LiDAR to Body). Represents the transformation from the radar coordinate system to the body coordinate system. : Original point cloud. The original point cloud data (in the radar coordinate system) acquired by the radar in the previous frame. Pixel coordinates. Column coordinates of the point cloud projection onto the 2D image plane. ) and row coordinates ( ). Projection function. Usually refers to perspective division, i.e. . Camera intrinsic parameter matrix. Includes focal length. He Guangxin .

[0033] In this process, a sparse mask map with the same resolution as the image is generated. The reflectivity intensity of the projection point was recorded. .

[0034] 1.2.2 Generation of Geometric Saliency Mask For the sky region, since the LiDAR beam does not return, Empty. The system only counts... Calculate the weighted average reflectance of the geometrically effective region within the area. This step physically achieves perfect semantic segmentation, with the system "forcibly ignoring" the bright pixels in the sky that occupy the frame and concentrating 100% of the metering weight on the tower structure.

[0035] Based on the cross-modal projection described in Section 1.2.1, a sparse mask image is generated. In essence, it's the geometric index of a 3D point cloud onto a 2D image plane. Regarding the relationship between the mask and reflectivity, this mask defines the calculation of the weighted average reflectivity. Effective Statistical Region of Interest (ROI): For the sky region, since the LiDAR beam has no return, the corresponding mask location... Empty; the system only counts Calculate the geometric effective region This mechanism allows the mask to act as a "semantic segmenter" at the physical level, ensuring that reflectivity calculations are determined solely by the material of the tower or conductor, thus allowing the system to "forcefully ignore" the bright pixels in the sky that occupy the image, and concentrate 100% of the metering weight on the tower structure.

[0036] 1.2.3 Reflectivity-Exposure Feedforward Control Loop This embodiment designs a control law that includes feedforward and feedback to calculate the optimal exposure time for the next frame. : , In the formula, The optimal exposure time for the next frame, calculated by the algorithm. : The exposure time currently being used in the current frame. : Preset target brightness value (the average grayscale value that the image is expected to achieve). The average gray level within the LiDAR mask area of ​​the previous frame. This parameter ensures that exposure is based solely on valid targets such as towers / wires, ignoring bright skies. / The average LiDAR reflectance of the previous and current frames. Used to predict and adjust exposure in advance based on changes in the reflectance of object materials (such as high-reflectivity insulators or low-reflectivity angle steel). Sensitivity coefficient. An exponential parameter used to adjust the intensity of the effect of the geometric feedforward term on exposure time. Photometric feedback term: This is the average gray level within the LiDAR mask area of ​​the previous frame. This ensures that exposure adjustments are based solely on the tower's brightness. Photometric Feedback Item: It is the geometric saliency mask described only in section 1.2.2 of the previous frame image. The average gray level within the coverage area. This ensures that exposure adjustments are based solely on geometrically valid physical targets (such as towers), avoiding interference from bright skies on image brightness statistics. Geometric feedforward term: Predicts based on the changing trends of LiDAR reflectivity.

[0037] 1) Scenario 1 (scanning insulators): When the insulator string has high reflectivity (reflectivity) When entering the field of view, A dramatic increase, ratio The algorithm instructs the camera to reduce exposure in advance to prevent the insulator from being overexposed and losing texture.

[0038] 2) Scenario 2 (Scanning over rusted angle steel): When low-reflectivity angle steel (reflectivity) When it occupies the field of view, the ratio The algorithm instructs the camera to significantly increase exposure. At this point, the background sky may be severely overexposed (completely white), but this is exactly what the system expects—to sacrifice the invalid background and preserve the geometric features of the core target.

[0039] 1.3 Hardware-level closed-loop triggering Calculated It must take effect immediately. After the parameters are calculated by the onboard computing platform (time required) The data is sent to the microcontroller via USB serial port. The microcontroller adjusts the PWM pulse width in the next PPS cycle and controls the integration time of the industrial camera via a hard-triggered pin. This mechanism ensures that the exposure parameters are "predicted values" for the current frame's geometry, rather than "corrected values" based on the hysteresis image.

[0040] This embodiment is based on the optimal exposure time of the next frame. Data acquisition is performed to obtain high dynamic range images and time-synchronized point cloud data.

[0041] 2. Semantically Guided Decoupling State Estimation of "Tower-Line-Background" 2.1 Semantic Aware Front-End: Wire-AwareBiSeNetV2 for Power Scenarios 2.1.1 Analysis of Pain Points in Scenarios and Defects of Native Models Existing SLAM systems are generally based on the "rigid body assumption of the environment," but directly applying native BiSeNetV2 to the multimodal dynamic environment of the power corridor has three major drawbacks: 1) Small Object Vanishing: Transmission lines occupy only 1-3 pixels in width from a long distance. The detail branches of the native network are prone to losing weak cable features during downsampling, resulting in semantic mask breaks.

[0042] 2) Boundary erosion under strong backlight: In the backlit scenes commonly seen during inspections, halos appear at the edges of the tower. The native loss function tends to fit large areas, causing the critical rigid tower edges to be misclassified as the sky background.

[0043] 3) Quantization Collapse Due to Dynamic Range Mismatch: Deploying the model to an edge NPU requires INT8 quantization. Power scene images exhibit extreme dynamic range: the sky background is extremely bright (pixel value > 200), while the power lines under backlight are extremely dark (pixel value < 20). In standard post-training quantization (PTQ), the quantization parameters (Scale / Zero-point) are determined by the statistical distribution of the entire image. Since the sky occupies more than 90% of the image, the quantization interval mainly serves the bright background, causing the "power line features" located in the low-value region of the histogram's long tail to fall into the quantization dead zone. The parameters are truncated to zero, directly leading to a sharp drop in the detection rate of fine structures.

[0044] 2.1.2 Algorithm Improvement: Wire-Aware BiSeNetV2 Network Architecture 1) Introduce a lightweight coordinate attention mechanism (LCA): such as Figure 2As shown, this embodiment embeds an LCA (Lightweight Coordinate Attention) module at the end of the standard detail branch of the Wire-AwareBiSeNetV2 network, which performs global pooling along the X and Y axes on the feature maps obtained at the end of the standard detail branch. This forcibly "activates" the spatial structural features of power lines, which have long-distance dependencies, preventing them from being submerged by the background.

[0045] 2) Boundary-Class Weighted Hybrid Loss Function: To address the issues of overlooking fine conductors and blurred tower edges, the network training employs a hybrid loss function, composed of a weighted average of the Online Hard Example Discovery Loss (OHEM Loss) and the Boundary Loss. , Itemized Explanation: a. Online Hard Case Mining Loss (OHEM Loss) – Focusing on Fine Conductors To address the issue of very few wire pixels and the high risk of misjudgment, a cross-entropy-based filtering mechanism is employed, calculating only the top wires with the highest loss values. Pixels: , In the formula, The selected set of difficult pixel examples (i.e., the pixel region with the largest difference between the predicted probability and the true label, usually containing thin wires). The total number of pixels in the difficult example. : pixel Category The true label (0 or 1). Network predicts pixels Category The probability of.

[0046] b. Boundary Loss – Sharpening tower edges To penalize the fuzzy prediction of structural edges, boundary constraints based on the Dice coefficient are introduced: , In the formula, : The edge probability map predicted by the network (obtained by processing the segmentation results with various operators or by independent branches). Ground Truth Boundary: A map showing the true edges of an object. Balance coefficient: Used to adjust the weight of boundary constraints in the total loss.

[0047] 3) NPU-Specific Quantization-Aware Training (QAT): To address the aforementioned dynamic range mismatch issue, this embodiment abandons the traditional PTQ scheme. During the PyTorch training phase, fake quantization nodes simulating the hardware characteristics of the AMD NPU are introduced. This forces the network to "sense" the truncation error during backpropagation, thereby automatically adjusting the weight distribution and learning robust feature representations insensitive to INT8 quantization noise. Experimental results show that this strategy reduces the wire segmentation mIoU on the NPU by less than 0.5% compared to the FP32 floating-point model.

[0048] This embodiment utilizes the improved Wire-AwareBiSeNetV2 network described above to perform semantic segmentation on the acquired high dynamic range images, obtaining pixel-level semantic segmentation results. These pixel-level semantic segmentation results are then used to divide the time-synchronized point cloud data into multiple semantic labels (rigid tower points, semi-rigid power line points, and invalid background points). Subsequently, in state estimation, corresponding observation models are constructed based on the different semantic labels.

[0049] 2.2 Hardware Acceleration: Advantages of Deep Heterogeneous Deployment of Processors with NPU Acceleration Units This solution leverages the hardware features of processors with NPU acceleration units for deep optimization, fully unleashing their unique advantages in edge computing: This solution leverages the hardware features of processors with NPU acceleration units for deep optimization, fully unleashing their unique advantages in edge computing: 1) Zero-penalty complex reasoning with high-performance NPUs Hardware advantages: The processor with an NPU acceleration unit has an integrated NPU with up to 50 TOPS of AI computing power and has been optimized for hardware-level instructions specifically for matrix convolution operations.

[0050] System benefits: This abundant computational power surplus allows this embodiment to incorporate complex operators such as the LCA attention module into BiSeNetV2 without sacrificing inference speed. Real-world testing shows that even with the increased computational load, the NPU can still stably maintain a high frame rate output of 20Hz+.

[0051] 2) Zero-copy and deterministic latency under UMA architecture Hardware advantages: Unlike the traditional "CPU + discrete graphics card" architecture, which requires data to be transferred through the PCIe bus (which is not only limited in bandwidth but also susceptible to millisecond-level jitter due to OS scheduling), the processor with NPU acceleration unit adopts a unified memory architecture (UMA), where the CPU and NPU share the same physical video memory.

[0052] System benefits: Zero-copy: NPU inference results are written directly to shared memory, and the CPU-side SLAM thread reads them directly through pointers, completely eliminating data transfer overhead.

[0053] Deterministic Latency: This tightly coupled architecture guarantees extremely low jitter-free inference response time. This is crucial for the ESIKF filter, ensuring strict alignment between visual / radar semantic tags and IMU timestamps, and avoiding anomalies in dynamic residual calculation caused by transmission delay jitter.

[0054] 3) Compute Isolation System benefits: Offloading the heavy convolution operations to the NPU reduces CPU load by approximately 20%. The saved CPU resources are fully allocated to SLAM backend optimization threads (such as LM optimization) that require highly logical judgments, avoiding contention for computing resources.

[0055] 2.3 Mathematical Model: Decoupled Error State Iterative Kalman Filter (Decoupled ESIKF) To address non-rigid body interference at the mathematical level, this embodiment utilizes semantic tags to deeply reconstruct the observation model within a laser inertial odometry framework based on error state iterative Kalman filtering (ESIKF).

[0056] 2.3.1 State Update Equation and Semantic Intervention In the update step of ESIKF, the optimal state estimate depends on the Kalman gain. This embodiment introduces a semantically adaptive observation noise covariance matrix. The gain formula can be rewritten as follows: , in To measure the Jacobian matrix, This is the prior error state covariance matrix. The innovation lies in... It is no longer a constant matrix, but rather composed of semantic tags. Dynamically determined diagonal matrix: , This embodiment constructs differentiated residual models and variance functions for three typical objects in the power scenario. : 1) Tower point: rigidly constrained The tower is the only absolutely rigid body in the scene. This embodiment constructs a standard point-to-plane residual: , in, It is a plane normal vector. For the corresponding point on the map, This is the current observation point.

[0057] Variance strategy: Assign it an extremely high confidence level, and set the variance to the physical ranging noise benchmark of the LiDAR sensor (e.g., 3cm). , Mathematical derivation consequences: In Kalman gain calculation, Minimal, making the corresponding dimension The value is relatively large. The filter will primarily utilize the geometric residuals at the tower points to correct the state vector. This allows the robot's absolute pose to be locked.

[0058] 2) Electric Power Line Points: Elastic Fault Tolerance Constraints and Covariance Expansion Power lines are semi-rigid objects and exhibit the physical characteristic of being subject to wind swaying. This embodiment constructs a point-to-line residual: , In the formula, : The perpendicular distance residual from a point to a line. : Identity matrix. : The unit direction vector of the line. : The current measured coordinates of the power line point cloud. : Corresponds to any point on the straight line. Projection matrix, used to calculate points To the straight line direction The vertical component. Wherein Let be the unit direction vector of the line. For online purposes.

[0059] Variance strategy (covariance expansion): To prevent power line galloping from deviating from its trajectory, this embodiment designs a wind speed-based strategy. Adaptive covariance inflation function: , In the formula, : The observation variance ultimately used for Kalman filter updates. The fundamental physical ranging noise of LiDAR sensors. : Coefficient of thermal expansion. Wind speed. : A function based on wind speed.

[0060] Function: When the wind speed is high, increase the variance and reduce the weight of the point in the location update to prevent the dancing power line from deviating from the trajectory.

[0061] here is the coefficient of thermal expansion.

[0062] Robust kernel function (HuberKernel): To further suppress outliers caused by large fluctuations, the Huber kernel function is introduced to reweight the residuals. , In the formula, : The residual function after reweighting. : Original residual value. Threshold parameter.

[0063] Function: When the residual |r| exceeds the threshold When outliers (such as those caused by large fluctuations) occur, the error term is transformed from a quadratic function into a linear function to suppress its excessive influence on optimization.

[0064] Mathematical derivation consequence: increased The weight of wire points in location updates has been reduced, but in direction (Jacobi) The constraints are still maintained on the rotational component (in the circuit). This achieves the decoupling effect of "using wires to constrain the heading angle, but tolerating its positional jitter".

[0065] 3) Invalid background points: Remove completely High-frequency noise sources such as vegetation not only have no geometric constraint value, but also disrupt the optimization process.

[0066] Variance strategy: , in The variance of background observations.

[0067] Mathematical derivation consequence: When the variance approaches infinity, the observation information matrix... This means that the residuals of the background points contribute zero to the state update, fundamentally cutting off the path for dynamic environmental noise to enter the state estimator mathematically.

[0068] This step mainly involves iteratively looping within a decoupled error state iterative Kalman filter framework to obtain the UAV's six-degree-of-freedom pose. Specific steps include: a. Based on point cloud data with semantic labels, construct corresponding observation models according to different semantic labels; b. For the observation model, perform iterative loops within the decoupled error state iterative Kalman filter framework and calculate the Kalman gain; c. Based on Kalman gain The observation residuals calculated from the integrated observation model are used to calculate the error state update amount and correct the UAV state. d. Determine whether the iteration termination condition has been met. If yes, the corrected UAV state is the UAV's six-degree-of-freedom pose. If not, return to step a to update the observation model and perform iterative calculations until the termination condition is met.

[0069] 3. Tightly Coupled LIVO Based on Catenary Geometric Constraints 3.1 Geometric Degradation and Catenary Physical Model The geometry of power line corridors exhibits strong degradation, which is the root cause of drift along the line in traditional SLAM.

[0070] From a geometric perspective, when a drone flies away from the tower, the main features scanned by LiDAR are only linear objects. Mathematically, straight lines in space possess "axial translation invariance," meaning that movement along a straight line does not cause changes in the geometric residuals. The direct consequence of this mathematical property is that the state along the line is unobservable; the system cannot perceive its own distance traveled, thus falling into unbounded "corridor effect" drift.

[0071] To address this degradation problem, this scheme introduces physical constraints to break the "straight-line assumption." Physical facts show that flexible transmission lines under gravity are not straight lines, but strictly follow the catenary equation.

[0072] The core basis for restoring observability using the catenary model lies in its unique curvature characteristics: The slope of the tangent to a straight line is constant, leading to ambiguity in its position; the slope of the tangent to a catenary... It is a location The function is such that the curvature is different at different locations.

[0073] Therefore, this embodiment establishes the following causal logic: as long as a small segment of the wire's curvature is observed, the position of that segment in the global model can be uniquely determined using the change in its curvature. This makes the previously unobservable displacement along the line observable, thereby correcting the cumulative drift of SLAM through physical laws.

[0074] 3.2 State Augmentation and Parameterization To utilize this constraint in SLAM, this embodiment augments the wire parameters in the ESIKF state vector. To ensure numerical stability and computational efficiency, the first... The wire parameterization is : , In the formula, : World coordinates of the catenary vertex. : The azimuth angle of the vertical plane in which the power line is located. Shape parameters (determined by tension and linear density).

[0075] 3.3 Derivation of Catenary Residuals and Jacobian To accurately describe the geometry of the transmission lines in three-dimensional space, we construct observation points. Composite residual function to theoretical catenary model The function consists of two orthogonal components: the out-of-plane residual (Geometric Residual) of the constraint traverse in the vertical plane, and the in-plane height residual (Physical Residual) of the constraint traverse conforming to the physical sag law.

[0076] (Observation input): For the first The three-dimensional coordinates of a power line lidar point cloud in the world coordinate system.

[0077] (State Vector to be Optimized): A core parameter vector describing the current geometry of the transmission line, containing the following three physical dimensions: : Coordinates of the catenary's vertex (lowest point). That is, the three-dimensional spatial coordinates of the entire conductor when it sags to its lowest position.

[0078] (Azimuth): Azimuth. The angle between the vertical plane containing the traverse and the X-axis of the world coordinate system. This parameter determines the direction of the traverse and constrains the coplanarity of the point cloud.

[0079] (Catenary Constant): Catenary shape parameter.

[0080] Mathematical definition: ,in For the horizontal tension of the conductor, This is the weight per unit length.

[0081] Geometric Intuition: This determines the size and curvature of the opening. The larger the value, the greater the tension, and the straighter the wire is pulled. The smaller the value, the looser the conductor and the more pronounced the sag.

[0082] Let the state vector to be optimized be... ,in These are the coordinates of the vertex.

[0083] 1) Local coordinate projection First, the observation point in the world coordinate system Projected onto a point with the catenary vertex as the origin and a direction angle of . In the local coordinate system.

[0084] The horizontal projection distance of the point in the vertical plane containing the traverse for: , 2) Out-of-Plane Residual: This constraint requires the observation point to be located within the range defined by the azimuth angle. Within a defined two-dimensional vertical plane, calculate the perpendicular distance from a point to this plane: , Note: This residual is used to correct the azimuth angle. The estimation bias.

[0085] 3) In-plane height residual (Sag Residual) Within a local vertical plane, the height of a point should conform to the catenary equation. The residual is defined as the difference between the observed height and the theoretical height. , Note: This residual is mainly used to correct shape parameters. and vertex height It is used for drift correction along the line position in ESIKF.

[0086] 4) Definition of composite residual In summary, the residual function Defined as the residual vector of the above two terms: , The corresponding least-squares optimization objective is to minimize the sum of squared orthogonal distances: , This embodiment uses the existing catenary model, whose expression is: , In the formula, For the horizontal coordinate The theoretical calculated height of the conductor. It is a hyperbolic cosine function; 3.4 Sliding Window Joint Fitting To address the problem of unstable parameter estimation caused by sparse single-frame LiDAR scanning and to reverse-correct UAV pose using catenary geometric constraints, this scheme designs a nonlinear optimization and drift correction mechanism based on a sliding window.

[0087] 1) Sliding window and world coordinate system transformation The system maintains a length of Keyframe sliding window , For the first t The power line point cloud data frame at each moment. Since the point cloud of a single frame is acquired in the body coordinate system, the current prior pose must be used in order to perform uniform fitting. All power line point clouds within the window Transform to the World Frame: , This step constructs a dense, globally consistent set of power line point clouds, providing ample data support for parametric reconstruction.

[0088] 2) Nonlinear least squares parameter fitting Based on the transformed global point cloud, an objective function based on nonlinear least squares (NLLS) is constructed to solve for the optimal catenary parameters. This is based on the state vector defined in Section 3.3. The objective function is defined as minimizing the sum of squares of the composite residuals (i.e., orthogonal distances) from all observation points to the theoretical catenary model: , Parameter description: : The catenary parameter vector to be optimized, including vertices Azimuth With shape parameters .

[0089] The composite residual function derived in Section 3.3 includes out-of-plane residuals. and in-plane height residual .

[0090] The total number of point clouds within the sliding window.

[0091] Parameter details: (Optimal Parameter Vector): Meaning: The optimal parameter vector obtained after optimization calculation that best matches the current power line geometry.

[0092] Physical meaning: This is the mathematical expression of the reconstructed high-precision conductor model.

[0093] (State Vector): Meaning: The state variables to be optimized, containing the five core parameters required to describe a catenary: . : The three-dimensional coordinates of the catenary vertex (lowest point) in the world coordinate system. The azimuth angle of the vertical plane containing the catenary relative to the X-axis of the world coordinate system. The shape parameter (Catenary Constant) of the catenary, physically... ,in For horizontal tension, This is the weight per unit length. The larger the value, the straighter the wire is pulled; The smaller the value, the more pronounced the sagging.

[0094] (Window Size): Meaning: The total number of power line point clouds contained within the sliding window.

[0095] Function: By accumulating data from multiple frames (such as the past 20 frames), it solves the problem of sparse radar beams in a single frame and provides sufficient constraints to accurately fit nonlinear curves.

[0096] (Point in World Frame): Meaning: The first one in the sliding window The point cloud of electric lines has been transformed from the body coordinate system to the world coordinate system through a pose transformation matrix.

[0097] (Residual Function): Meaning: Residual function. Calculated from observation points. To the parameter The orthogonal distance of the defined theoretical catenary model (including in-plane height residuals and out-of-plane distance residuals).

[0098] in The residual function is derived in Section 3.3. This optimization problem is solved iteratively using the Levenberg-Marquardt (LM) algorithm to obtain high-precision conductor geometry parameters.

[0099] 3) Z-axis Drift Correction This is the key closed loop for eliminating accumulated errors in SLAM using geometric priors. Once the accurate catenary parameters are solved using NLLS... The system calculates the altitude residual (SagResidual) between the observation point and the theoretical model under the current UAV pose.

[0100] Due to the curvature of the catenary (as described in Section 3.1), the altitude residual exhibits a one-to-one functional relationship with the UAV's position error along the catenary. The system calculates the Jacobian matrix of the altitude residual relative to position. This is then incorporated as an observation constraint into the ESIKF update process: , Parameter details: (Position Correction): Meaning: The calculated position correction along the direction of the transmission line.

[0101] Function: This value is fed back directly to the SLAM system to “pull back” the accumulated drift caused by geometric degradation.

[0102] (Gain / Jacobian Factor): Meaning: Correction gain, essentially the same as the Jacobian matrix. Related.

[0103] Physical logic: Utilizing the curvature characteristics of a catenary (slope varies with position). If the tangent slope of the conductor at the current position is large, the height residual becomes highly sensitive to positional errors. A larger value indicates a stronger correction; however, if the value is near the lowest point (where the slope is 0), no correction can be made.

[0104] (Measured Height): Meaning: The actual altitude of the UAV (or observation point) in the world coordinate system at the current moment (provided by barometer or SLAM state estimation).

[0105] (Theoretical Height): Meaning: Based on the fitted optimal catenary parameters Substitute the current estimated horizontal position into the calculated theoretical height.

[0106] Logic: If the "actual height" does not match the "theoretical height", and the shape of the conductor is known to be accurate, then the "horizontal position" must be estimated incorrectly (because we are on a slope, and if the position is wrong, the height will be wrong).

[0107] In this way, the originally negligible positional drift along the line is transformed into a considerable height error, thereby forcing the SLAM system to correct its cumulative drift along the corridor direction and achieving a "soft loop" at the geometric level.

[0108] SLAM of 4 Insulator "Skeleton Node" Topology 4.1 Insulator aliasing sensing and skeleton extraction The widespread insulator strings have highly repetitive geometry and texture, which is the root cause of "perceptual aliasing" in vision and lidar.

[0109] Within the traditional SLAM framework, this repetitiveness can lead to a fatal causal chain reaction: 1) Feature collapse: The first Film and the first The feature descriptors of the slices are indistinguishable in the feature space.

[0110] 2) Mismatched Matching: Front-end algorithms are highly prone to incorrect associations (such as mismatching the first...). Piece matched to the first piece).

[0111] 3) Axial drift: This misalignment directly translates into cumulative error along the insulator axis, and traditional bag-of-words loop closure detection fails due to a large number of false positives, making it unable to correct this error. To address this pain point, this solution proposes a "discard texture, re-topology" strategy, the core of which lies in using "frequency domain analysis" to transform repetitive interference into location information. This embodiment constructs a structure-frequency descriptor (SFD), the extraction process of which embodies profound physical and mathematical logic: Structure-Frequency Descriptor (SFD) Extraction Process: 1) Axis fitting: Using RANSAC in conjunction with surface normal constraints of insulator point cloud points, the central axis of the insulator is robustly extracted from the insulator point cloud collected by UAV and used as the insulator skeleton.

[0112] 2) Contour waveform generation: Project the point cloud onto the axis coordinate system and calculate the radius distribution function along the axis. , is represented as: , In the formula, The projection coordinates of the point cloud points in a plane perpendicular to the central axis; 3) Frequency domain fingerprint extraction: for Perform a Fast Fourier Transform (FFT) to obtain the frequency domain fingerprint, including: clock speed : Corresponding to the spacing of the insulator discs .

[0113] phase : Corresponds to the axial offset of the insulator string relative to the starting point.

[0114] Amplitude spectrum: serving as the unique fingerprint of the insulator string, used to distinguish different models (such as double-umbrella and triple-umbrella).

[0115] 4.2 Phase Locking Anti-Drift Mechanism This is the core innovation of our solution in handling repetitive textures. Traditional SLAM produces unbounded cumulative drift under repetitive textures. This solution utilizes "phase" to make the error bounded.

[0116] Assume the displacement estimated by the odometer is The observed phase extracted by FFT is This embodiment uses the following formula to correct the displacement: , Parameter details: (Corrected Position): Meaning: Precise axial displacement after phase-locking correction.

[0117] Accuracy: Documentation can be described down to the centimeter level.

[0118] (Estimated Position): Meaning: Coarse displacement estimated by front-end odometer (LIO).

[0119] Feature: Includes cumulative drift over time.

[0120] (Integer Ambiguity Resolution): Meaning: Rounding function.

[0121] Function: To calculate how many complete insulator discs the robot traversed (integers) This is the core of the formula, used to solve "integer ambiguity".

[0122] (Wavelength / Pitch): Meaning: The physical distance (wavelength) between insulator discs.

[0123] Source: The inverse of the dominant frequency obtained by performing FFT (Fast Fourier Transform) on the point cloud distribution.

[0124] (Observed Phase): Meaning: The current observation phase angle extracted by FFT (unit: radians, range). ).

[0125] Function: Indicates the specific percentage position of the current location within a disk cycle.

[0126] (Scale Factor): Meaning: A scaling factor for converting phase angle (radians) into physical distance (meters).

[0127] (Intra-period Displacement): Meaning: Displacement within a period. That is, the precise distance ("decimal part") between the current position and the zero point of the previous disk.

[0128] Explanation of principle: As long as the odometer drift between two observations is less than half a cycle ( In this embodiment, the number of disks the robot has traversed can be determined by rounding (integer ambiguity resolution), thereby using phase information to lock the axial positioning accuracy at the centimeter level. This essentially transforms unbounded cumulative drift into bounded phase observation noise.

[0129] Intuitive analogy: The Vernier Caliper Effect In this embodiment, the insulator string can be likened to a precision ruler, with each disc being a "graduation".

[0130] Traditional SLAM (odometer): It's like reading a distance with the naked eye. As the distance increases, the reading error accumulates, and it may be seen as 10.1cm or 10.9cm (there is cumulative drift).

[0131] Phase-locked function (SFD): This is like the "alignment line" of a vernier caliper. Although the odometer can only tell this embodiment that it is approximately "around the 10th mark", the phase information extracted by SFD can accurately tell this embodiment "the current waveform corresponds to the 0.05 position within the mark".

[0132] The system combines the two: using the odometer to determine "which mark it is (integer part)" and using the phase to determine "the precise position within the mark (decimal part)", thereby achieving centimeter-level axial positioning.

[0133] 4.3 Topological loops based on sequence alignment An insulator topology graph (ITG) is constructed. Nodes are insulator skeletons, and also include the aforementioned frequency domain fingerprint information; edges represent connections. Loop closure detection no longer compares image textures, but rather compares "skeleton sequences." Specifically, when a drone flies over an area, it records the current position and a series of insulator skeleton sequences. When flying to a new position, the current skeleton sequence is used as the query sequence. The Needleman-Wunsch sequence alignment algorithm from bioinformatics is used to compare the query sequence with all historical insulator skeleton sequences in the insulator topology graph. If the similarity score between a historical insulator skeleton sequence and the query sequence exceeds a threshold, a loop closure is determined. Geometric correction is then performed using the aforementioned modified displacement formula to achieve global optimization and complete the loop closure detection process. This algorithm allows for gaps in the sequence, tolerating missed detections due to occlusion, and only matches the overall topology. This provides robustness far exceeding that of traditional bag-of-words models in extremely similar environments.

[0134] In this step, the ITG topology map finds the 'missing location' through sequence alignment (solving similarity interference), while SFD phase locking calculates the 'precise error' using the periodic characteristics of the insulator (solving axial drift). The combination of the two achieves robust and high-precision closed-loop detection.

[0135] 5. System Integration and Engineering Implementation 5.1 Software Architecture and Thread Scheduling The system is developed based on the ROS2 architecture and makes full use of the multi-core performance of processors with NPU acceleration units for thread-level parallel scheduling.

[0136] Thread1(Real-timeCritical): SensorI / O&Sync Responsible for driving and acquiring data from LiDAR, IMU, and cameras.

[0137] Process the timestamps reported by the microcontroller and perform hard synchronization compensation.

[0138] Thread2(ComputeHeavy):Front-end&LIO LiDAR processing: point cloud distortion correction and catenary sliding window fitting.

[0139] Semantic reasoning: Invoking the NPU to perform BiSeNetV2 inference.

[0140] ESIKF update: Perform decoupled state estimation, fusing tower, power line (catenline constraint) and IMU data.

[0141] Thread3(Vision&Control):Exposure Loop Perform point cloud projection, mask generation, and exposure parameter calculation.

[0142] Exposure commands are sent to the microcontroller via serial port.

[0143] Thread4(Background):Topology&Optimization Perform insulator skeleton extraction, FFT analysis, and phase locking.

[0144] Maintain the global topology graph and perform sequence alignment loop closure detection and global pose graph optimization (PGO).

[0145] 5.2 Critical Data Flow and Memory Management To ensure real-time performance of 20Hz+, the system adopts a zero-copy architecture.

[0146] ROS2 shared memory: When transferring large amounts of data (such as point clouds and images) between nodes, ROS2's Loaned Messages mechanism is used to avoid the overhead of serialization and deserialization.

[0147] NPU-CPU Shared Memory: The UMA architecture of a processor platform with an NPU acceleration unit allows the CPU pointer to directly access the NPU's inference output buffer, resulting in negligible latency in semantic mask acquisition.

[0148] 5.3 Robust Design Table The robustness design table is shown in Table 1 below: Table 1 Robust Design Table 6. Overall Implementation Steps of the Parametric Reconstruction Method for Transmission Corridors This invention proposes a parametric reconstruction method for power transmission corridors that integrates nonlinear geometric priors and semantic topology. The specific implementation process includes the following five steps, the flow of which is as follows: Figure 1 As shown: Step S1: Multimodal data acquisition and adaptive enhancement (corresponding to Section 1) Inputs: Raw point cloud from LiDAR, raw images from the camera, and inertial data from the IMU.

[0149] Processing: A "geometric saliency mask" is constructed using the point cloud from the previous frame. The weighted average reflectance of the tower area is calculated, and the optimal exposure time is determined through a "reflectance-exposure feedforward control loop". It also controls the camera to capture the next frame image via hardware triggering.

[0150] Output: High dynamic range image with appropriate exposure, and time-synchronized raw point cloud data.

[0151] Step S2: Semantic perception and feature decoupling (corresponding to the first half of Section 2) Input: Image data obtained in step S1.

[0152] Processing: The image is input into the NPU-accelerated Wire-AwareBiSeNetV2 network for inference to generate pixel-level semantic segmentation results; combined with LiDAR depth information, the scene point cloud is divided into three categories: rigid tower points, semi-rigid power line points, and invalid background points.

[0153] Output: A 3D point cloud with semantic labels (tower, line, background).

[0154] Step S3: Semantically guided decoupling state estimation (corresponding to the second half of Section 2) Input: Point cloud and IMU data with semantic labels.

[0155] Processing: Enter the decoupled error state iterative Kalman filter (DecoupledESIKF).

[0156] For tower points: construct high-weight point-to-plane residuals to lock the absolute pose.

[0157] For power line points: construct point-to-line residuals with covariance expansion, and use a wind speed model to soften constraints, limiting only the heading and tolerating galloping.

[0158] For background points: set the variance to infinity to completely eliminate them.

[0159] Output: High-frequency, high-precision 6-DoF pose of UAVs .

[0160] Step S4: Parametric mapping based on catenary constraints (corresponding to Section 3) Input: Pose and power line point cloud from step S3.

[0161] Processing: Maintain a sliding window to transform the multi-frame power line point cloud to the world coordinate system. Utilize the nonlinear least squares method to jointly fit the conductor parameters (vertices) based on the catenary model. Azimuth Shape parameters ), and calculate the height residual reverse correction along the line position drift (Z-axis Drift Correction).

[0162] Output: Vectorized transmission line parameter model.

[0163] Step S5: Loop closure detection and global optimization based on topological fingerprint (corresponding to Section 4) Input: Insulator point cloud fragment.

[0164] Processing: Extract the insulator skeleton and generate a structure-frequency descriptor (SFD); perform phase locking through FFT frequency domain analysis to eliminate axial aliasing errors; construct an insulator topology graph (ITG), use a sequence alignment algorithm to detect loop closures, and perform factor graph optimization (PGO).

[0165] Output: A globally consistent high-precision 3D map of the power transmission corridor.

[0166] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0167] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0168] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0169] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0170] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0171] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0172] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A parametric reconstruction method for power transmission corridors that integrates nonlinear geometric priors and semantic topology, characterized in that, include: The point cloud data, image data, and IMU inertial data of the power transmission corridor are acquired and adaptively enhanced to obtain high dynamic range images and time-synchronized point cloud data. Based on the high dynamic range image, semantic segmentation is performed using the Wire-Aware BiSeNetV2 network to obtain pixel-level semantic segmentation results, and the time-synchronized point cloud data is divided into multiple semantic labels; Based on point cloud data with semantic labels, a semantically guided decoupled state estimation method is used to process the data to obtain the six-degree-of-freedom pose of the UAV. Based on the six-degree-of-freedom pose of the UAV, a parametric mapping method with catenary constraints is used for fitting and drift correction to obtain the parameters of the power transmission corridor conductor.

2. The method for parametric reconstruction of power transmission corridors integrating nonlinear geometric priors and semantic topology as described in claim 1, characterized in that, The steps for adaptive enhancement include: Combining the IMU inertial data, the point cloud from the previous frame is... Transform from radar coordinate system to current camera coordinate system The three-dimensional coordinates under the current camera coordinates are obtained, and are represented as follows: ， In the formula, For the present 3D coordinates of the point cloud at any given time in the current camera coordinate system , It is an extrinsic parameter matrix. The pose at the current moment is predicted from IMU inertial data. The pose of the previous frame. This is the extrinsic parameter matrix; Based on the aforementioned three-dimensional coordinates, the pinhole model is used to extract the point cloud from the previous frame. Projecting the image onto a two-dimensional image plane yields the projected pixel coordinates, forming a sparse mask image with the same resolution as the image data. The expression for calculating the projected pixel coordinates is as follows: ， In the formula, These are the projected pixel coordinates. For projection function, This is the camera intrinsic parameter matrix; The sparse mask image Based on the reflectivity intensity of the projection point Determine the effective statistical domain ROI: And calculate the weighted average reflectance of the effective statistical domain ROI. The effective statistical domain represents the tower area; A reflectivity-exposure feedforward control loop is used to calculate the optimal exposure time for the next frame. The calculation expression is: ， In the formula, To determine the optimal exposure time for the next frame, The exposure time currently being used in the current frame. The preset target brightness value, The sparse mask image from the previous frame. Average gray level within, / The LiDAR-weighted average reflectance of the previous and current frames. This is the sensitivity coefficient; Based on the optimal exposure time of the next frame Data acquisition is performed to obtain high dynamic range images and time-synchronized point cloud data.

3. The method for parametric reconstruction of power transmission corridors integrating nonlinear geometric priors and semantic topology as described in claim 1, characterized in that, The semantic tags are categorized as follows: rigid tower point, semi-rigid power line point, and invalid background point.

4. The method for parametric reconstruction of power transmission corridors integrating nonlinear geometric priors and semantic topology as described in claim 3, characterized in that, The steps for obtaining the six-degree-of-freedom pose of the UAV include: 1) Based on the point cloud data with semantic labels, construct corresponding observation models according to different semantic labels; 2) For the observation model, iterative loops are performed within the decoupled error state iterative Kalman filter framework, and the Kalman gain is calculated. The Kalman gain mentioned above The calculation expression is: ， in: ， In the formula, Let the prior error state covariance matrix be... To measure the Jacobian matrix, It is a diagonal matrix. It is the variance function. For the first semantic tags; 3) Based on the Kalman gain The observation residuals calculated by the observation model are integrated to calculate the error state update amount and correct the UAV state. 4) Determine whether the iteration termination condition has been met. If yes, the corrected UAV state is the UAV's six-degree-of-freedom pose. If no, return to step 1) to update the observation model and perform iterative calculations until the termination condition is met.

5. The method for parametric reconstruction of power transmission corridors integrating nonlinear geometric priors and semantic topology as described in claim 4, characterized in that, The observation model consists of two parts: a residual model and a variance function. For rigid tower points, the residual model is a point-to-surface residual model, expressed as: ， In the formula, This is a point-to-plane residual model, representing the distance residual from a point to a plane. It is a plane normal vector. For the current observation point, The corresponding point on the map; Its variance function is expressed as: ， In the formula, To observe the variance, The fundamental physical ranging noise of LiDAR sensors, For the first semantic tags, For rigid tower points; For a semi-rigid electric line point, its residual model is a point-to-line residual model, expressed as: ， In the formula, For the point-to-line residual model, it represents the perpendicular distance residual from a point to a line. It is the identity matrix. Let be the unit direction vector of the line. The coordinates of the current measured power line point cloud. For any point on the corresponding line, The projection matrix; Its variance function is expressed as: ， In the formula, To observe the variance, The fundamental physical ranging noise of LiDAR sensors, The coefficient of thermal expansion is 1 / 3. For wind speed, A function based on wind speed. For semi-rigid electric field line points; For invalid background points, their variance function is set to To completely remove invalid background points, among which Invalid background point.

6. The method for parametric reconstruction of power transmission corridors integrating nonlinear geometric priors and semantic topology as described in claim 5, characterized in that, For semi-rigid electric field points, the method further includes: introducing a robust kernel function to reweight the variance function, wherein the reweighted residual function is expressed as: ， In the formula, The residual function after reweighting. The original residual value, This is the threshold parameter.

7. The method for parametric reconstruction of power transmission corridors integrating nonlinear geometric priors and semantic topology as described in claim 1, characterized in that, The steps for obtaining the parameters of the power transmission corridor conductors include: Set up a sliding window ,in The length of the sliding window. For the first t Power line point cloud data frames at any given time; Using the prior pose at the current moment Transform all electric field point clouds within the sliding window to the world coordinate system to obtain: ， In the formula, For the first transformation to the world coordinate system within the sliding window i A point cloud of power lines, Before the conversion within the sliding window i A point cloud of power lines; Based on the point cloud of all electric power lines to the world coordinate system, a nonlinear least squares-based objective function is constructed to minimize the sum of orthogonal distances from all observation points to the catenary model. The expression for the objective function is: ， in: ， ， ， In the formula, This is the optimal catenary parameter vector. The state variables to be optimized are: The coordinates of the lowest point of the catenary in the world coordinate system are: Let be the azimuth angle of the vertical plane containing the catenary relative to the X-axis of the world coordinate system. These are the shape parameters of the catenary. For horizontal tension, Weight per unit length This represents the total number of power line point clouds contained within the sliding window. For the residual function, Let be the perpendicular distance from the point to the two-dimensional vertical plane. This refers to the height residual between the observed altitude of the UAV in its current pose and the theoretical catenary model during the ESIKF update. The horizontal projection distance of the point in the vertical plane containing the traverse; The expression for the catenary model is as follows: ， In the formula, For the horizontal coordinate The theoretical calculated height of the conductor. It is a hyperbolic cosine function; In the error state iterative Kalman filter framework update, the objective function based on nonlinear least squares is solved iteratively to obtain the optimal catenary parameter vector. ; This also includes the following: Based on the height residual Calculate its Jacobian matrix relative to the UAV's position along the line, expressed as: ， The Jacobian matrix is ​​introduced as an observation constraint into the ESIKF update process: ， In the formula, For Jacobian matrices, The height residual in the plane, This indicates the location of the drone along the route. This is the positional correction amount along the direction of the transmission line. To correct the gain, This represents the actual altitude of the drone or observation point in the world coordinate system at the current moment. To determine the optimal catenary parameters based on the fitted data .

8. The method for parametric reconstruction of power transmission corridors integrating nonlinear geometric priors and semantic topology according to claim 1, characterized in that, It also includes loop closure detection and global optimization steps based on topological fingerprinting, specifically including: Using the RANSAC algorithm combined with the surface normal direction constraint of the insulator point cloud, the central axis of the insulator is extracted from the insulator point cloud and used as the insulator skeleton. Project the insulator point cloud onto the coordinate system of the insulator's central axis, and calculate the radius distribution function along the insulator's central axis. , is represented as: ， In the formula, The projection coordinates of the point cloud points in a plane perpendicular to the central axis; For the radius distribution function Perform a Fast Fourier Transform to obtain the frequency domain fingerprint, including the dominant frequency. Phase Amplitude spectrum, of which the dominant frequency Corresponding insulator disc spacing phase The amplitude spectrum corresponding to the axial offset of the insulator string relative to the starting point is determined by the specific shape of the disc, serving as the unique fingerprint of the insulator string; Construct an insulator topology graph, where the insulator skeleton serves as a node, the node also includes the frequency domain fingerprint, and the connection relationship serves as an edge; When the drone flies over an area, it records the current position and a sequence of insulator skeletons of a series of insulators. When it flies to a new position, it uses the current skeleton sequence as the query sequence. Using the Needleman-Wunsch sequence alignment algorithm, the query sequence is compared with all historical insulator skeleton sequences in the insulator topology map. If the similarity score between a historical insulator skeleton sequence and the query sequence exceeds a threshold, a loop closure is determined to have occurred. Global optimization is then performed through geometric correction to complete the loop closure detection process. Geometric correction is performed using the following formula: ， In the formula, This represents the precise axial displacement after phase-locking correction. This is a rough displacement estimated by the front-end odometer. This is the rounding function. The physical spacing between insulator discs. The current observed phase angle is extracted using FFT. To convert the phase angle into a scaling factor for physical distance, It represents the displacement within the period.

9. The method for parametric reconstruction of power transmission corridors integrating nonlinear geometric priors and semantic topology as described in claim 1, characterized in that, The Wire-Aware BiSeNetV2 network embeds a lightweight coordinate attention mechanism at the end of the standard detail branch, which performs global pooling on the feature map obtained at the end of the standard detail branch along the X and Y axes respectively.

10. The method for parametric reconstruction of power transmission corridors integrating nonlinear geometric priors and semantic topology according to claim 1, characterized in that, The Wire-Aware BiSeNetV2 network is trained using a boundary-class weighted hybrid loss function, the expression of which is: ， In the formula, For boundary-class weighted mixed loss function, To uncover losses in online difficult cases, For balance coefficient, For boundary loss; in: ， ， In the formula, The total number of pixels in the difficult example. For the selected set of difficult-to-find pixels, For pixels Category The true label, Predict pixels for the network Category The probability, This is the edge probability map predicted by the network. This is a true edge label map.

Images