Power equipment inspection path planning method and device based on laser and vision

Abstract

The invention discloses a power equipment inspection path planning method and device based on laser and vision, and relates to the technical field of intelligent robots and power inspection, and the method comprises the steps: constructing a multi-sensor system, carrying out the external parameter calibration and time synchronization of each sensor, and carrying out the environment modeling and positioning through employing an improved LIO-SAM algorithm; performing power equipment identification; generating an optimal inspection path based on the calculated coordinate set of the power equipment; according to the optimal inspection path, dynamic obstacles are detected in combination with semantic and geometric information, obstacle avoidance is achieved through local track adjustment and path re-planning, and a planned path is obtained; and driving the inspection robot to autonomously move according to the planned path, acquiring an image and infrared data at the target power equipment, and feeding back a result to the background system to complete closed-loop inspection. According to the invention, the autonomous inspection of the whole process of environment modeling, target equipment identification, path generation, dynamic obstacle avoidance and the like can be effectively realized.

Description

Technical Field

[0001] This application relates to the field of intelligent robots and power inspection technology, specifically to a method and device for power equipment inspection path planning based on laser and vision. Background Technology

[0002] With the continuous expansion of power systems, substations have a wide variety of equipment and complex spatial structures, and the stability of equipment operation is directly related to power grid safety. Currently, power inspection mainly relies on manual methods, which suffers from high labor intensity, high risk, and low efficiency. Although some mobile robots or robot dogs have been applied to power inspection, their core algorithms still face the following challenges: the dense metal equipment and highly regular structures in substations make conventional laser SLAM (Simultaneous Localization and Mapping) prone to matching ambiguities and cumulative drift, making it difficult to guarantee long-term stable positioning; existing deep learning-based detection models have insufficient recognition accuracy under complex lighting and occlusion conditions and lack accurate mapping to 3D coordinates; traditional algorithms such as A* and D*Lite mainly focus on the geometric shortest path and fail to combine the task characteristics of power inspection (priority, timeliness, and fixed point order), resulting in planning results that are difficult to meet engineering requirements; at the same time, personnel and vehicles often enter during the inspection process, and existing obstacle avoidance algorithms are often limited to geometric-level real-time avoidance, lacking the utilization of semantic categories and temporal movement patterns.

[0003] Therefore, how to combine laser SLAM and AI visual recognition to realize autonomous inspection path planning for power equipment, so as to improve positioning robustness, equipment recognition accuracy and task execution efficiency, and thus realize the true autonomy and engineering usability of power inspection robots, has become an urgent problem to be solved. Summary of the Invention

[0004] This application provides a method and device for power equipment inspection path planning based on laser and vision, which can effectively realize autonomous inspection of the entire process, including environmental modeling, target equipment identification, path generation and dynamic obstacle avoidance.

[0005] In a first aspect, embodiments of this application provide a method for power equipment inspection path planning based on laser and vision, the method comprising: A multi-sensor system was constructed based on 3D LiDAR, IMU, camera and odometer. The external parameters of each sensor were calibrated and time was synchronized. An improved LIO-SAM algorithm was used for environmental modeling and localization. In this process, prior constraints on power structure and dynamic point cloud removal mechanism were introduced into the factor graph. Power equipment identification is performed using an improved YOLOv8 model. The improved YOLOv8 model embeds a structure-enhanced attention mechanism in the feature extraction stage, adopts an occlusion perception training strategy and combines a difficult example contrast distillation mechanism in the training stage, and calculates the three-dimensional coordinates of the power equipment in the map by combining multi-sensor projection mapping and point cloud fitting. Based on the calculated set of coordinates of the power equipment, an improved path planning algorithm with a composite cost function and a dynamic task reordering mechanism is used to generate the optimal inspection path that satisfies the constraints of inspection priority, timing and sequence. Based on the optimal inspection path, dynamic obstacles are detected by combining semantic and geometric information. Obstacle avoidance is achieved by using local trajectory adjustment and path replanning, and the planned path is obtained. The inspection robot is driven to move autonomously along the planned path, collect images and infrared data at the target power equipment, and feed the results back to the background system to complete the closed-loop inspection.

[0006] In conjunction with the first aspect, in one implementation method, The prior constraint of the power structure is to identify power equipment with regular geometric features in the substation by camera, and obtain geometric primitives by fitting them in the point cloud. The geometric primitives are then injected as static landmarks into the factor map optimization process to enhance the geometric consistency of the map. The dynamic point cloud culling mechanism detects dynamic objects visually and analyzes their movement based on time-series trajectories to remove the corresponding point cloud data, thus retaining only the static environmental point cloud for mapping and localization.

[0007] In conjunction with the first aspect, in one implementation method, The detection head of the improved YOLOv8 model is a structural constraint orientation regression head. In addition to regressing the traditional center and scale, the detection head also regresses the target orientation and includes a direction regularization term in the loss function. The improved YOLOv8 model has a geometrically consistent cross-modal attention module between its backbone and neck. This module generates a depth or normal cue map by projecting sparse point clouds onto the image and calculates attention weights based on point density and normal consistency. This enhances the feature response of geometrically supported pixels during the feature fusion stage and automatically reduces the weights of pure texture or regions without geometric support.

[0008] In conjunction with the first aspect, in one implementation method, The hard example contrastive distillation is a training strategy that includes constructing a feature prototype memory for each type of power equipment, incorporating highly confusing hard negative samples that are manually labeled or rule-mined into the feature prototype memory, applying contrastive loss to the classification branch to encourage sample features to move closer to their positive class prototypes and away from hard negative prototypes, and combining the soft labels of the teacher network for distillation training.

[0009] In conjunction with the first aspect, in one implementation method, For the three-dimensional coordinates of power equipment, the geometric parameters are obtained by projecting the orientation box onto the laser coordinate system according to the camera intrinsic parameters and selecting the point cloud return point closest to the projection ray. Then, a robust fitting of a straight line, plane, or cylinder is performed on the subset of the point cloud to determine the coordinates.

[0010] In conjunction with the first aspect, in one implementation, the composite cost function includes a priority penalty term, a due date penalty term, and a sequential default penalty term.

[0011] In conjunction with the first aspect, in one implementation, the dynamic task rescheduling mechanism updates the task sequence according to a strategy based on the comparison between the estimated future arrival time and the task deadline, and performs only local improved A* replanning on the affected interval to ensure real-time performance.

[0012] In conjunction with the first aspect, in one implementation method, For the detection of dynamic obstacles, Kalman filtering is used to predict the two-dimensional position and velocity of the obstacles. Based on the prediction, a probabilistic safe domain penalty term is added to the local spatiotemporal trajectory optimization cost to achieve prediction-driven obstacle avoidance.

[0013] In conjunction with the first aspect, in one implementation method, For local trajectory adjustment, a spatiotemporal constraint optimization based on time elastic band is adopted. The spatiotemporal distance penalty of predicting dynamic obstacles is added to the objective function, and soft constraints are applied to velocity, angular velocity and curvature to obtain a smooth and executable obstacle avoidance trajectory. For path replanning, the triggering condition is that the main channel is predicted to be blocked for a period of time exceeding a preset threshold, or the task completion time is expected to exceed the allowed time window after local replanning is triggered.

[0014] Secondly, embodiments of this application provide a power equipment inspection path planning device based on laser and vision, the power equipment inspection path planning device based on laser and vision includes: The environment modeling and localization module is used to build a multi-sensor system based on 3D LiDAR, IMU, camera and odometer, to perform external parameter calibration and time synchronization of each sensor, and to use the improved LIO-SAM algorithm for environment modeling and localization. In this module, the prior constraints of power structure and dynamic point cloud removal mechanism are introduced into the factor graph. The target device identification and localization module is used to identify power equipment using an improved YOLOv8 model. The improved YOLOv8 model embeds a structure-enhanced attention mechanism in the feature extraction stage, adopts an occlusion perception training strategy and combines a difficult example contrast distillation mechanism in the training stage, and calculates the three-dimensional coordinates of the power equipment in the map by combining multi-sensor projection mapping and point cloud fitting. The inspection path generation module is used to generate the optimal inspection path that satisfies the inspection priority, timing and sequence constraints based on the calculated set of coordinates of the power equipment and an improved path planning algorithm with a composite cost function and dynamic task reordering mechanism. The dynamic obstacle avoidance and path replanning module is used to detect dynamic obstacles based on the optimal inspection path, combined with semantic and geometric information, and to achieve obstacle avoidance by using local trajectory adjustment and path replanning to obtain the planned path. The autonomous inspection execution and data acquisition module is used to drive the inspection robot to move autonomously along the planned path, collect images and infrared data at the target power equipment, and feed the results back to the background system to complete the closed-loop inspection.

[0015] The beneficial effects of the technical solutions provided in this application include: (1) Improve positioning accuracy and robustness: By introducing prior constraints on power structure and dynamic point cloud removal mechanism in the improved LIO-SAM, the drift caused by matching ambiguity and dynamic interference in regular scenarios is effectively reduced, the trajectory drift is reduced by about 45%, and the environmental modeling and positioning accuracy is improved. (2) Achieve high-precision equipment identification and positioning: The improved YOLOv8 model with structure-enhanced attention mechanism is adopted, which improves the average accuracy (mAP0.5) of identification of typical substation equipment (circuit breakers, disconnectors, surge arresters, insulators, etc.) from 92.1% to 96.8%, and the identification recall rate under occlusion conditions is improved by 12%, which significantly improves the identification accuracy of power equipment and the robustness under occlusion scenarios. In addition, the three-dimensional coordinates of power equipment are calculated by multi-sensor calibration, providing reliable target input for inspection path planning. (3) Path optimization that satisfies power inspection constraints: In the path planning, a composite cost function and a dynamic task reordering mechanism are introduced. This not only considers the shortest path and the optimal energy consumption, but also satisfies the actual constraints of power inspection, such as the priority of power equipment, timing and inspection sequence, so as to generate the optimal inspection path. (4) Ensure the continuity and safety of inspection: Through the dynamic obstacle avoidance and path replanning method that integrates semantics and geometry, the continuity and safety of the task can be maintained in environments with moving obstacles such as personnel and vehicles. (5) Forming a closed-loop autonomous inspection process: The inspection robot can autonomously complete environmental modeling, equipment identification, path generation, dynamic obstacle avoidance and data collection, forming a complete closed-loop inspection process, significantly reducing manual workload and improving the intelligence and automation level of power inspection. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the laser and vision-based power equipment inspection path planning method of this application. Figure 2 This is a schematic diagram illustrating the specific implementation process of the laser and vision-based power equipment inspection path planning method of this application; Figure 3 This is a schematic diagram of the functional modules of the power equipment inspection path planning device based on laser and vision in this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0019] In the first aspect, the embodiments of this application provide a power equipment inspection path planning method based on laser and vision, namely, a power equipment autonomous inspection path planning method based on laser SLAM and AI visual recognition, which is applicable to intelligent inspection robots in power scenarios such as substations and power plants, and is used to realize autonomous inspection of the entire process, including environmental modeling, target equipment identification, path generation and dynamic obstacle avoidance.

[0020] Specifically, this system integrates data from LiDAR, IMU (Inertial Measurement Unit), and odometers, utilizing an improved LIO-SAM model that incorporates power structure constraints and a dynamic point cloud culling mechanism to achieve environmental modeling and localization. An improved YOLOv8 model with a structure-enhanced attention mechanism is employed to identify power equipment and determine its 3D coordinates. Based on a set of target points, an optimal inspection path is generated using a composite cost function and a dynamic task rearrangement mechanism. Furthermore, dynamic obstacle avoidance and replanning through semantic-geometric fusion ensure inspection continuity, ultimately achieving high-precision localization, intelligent identification, and autonomous execution of power inspection tasks. LIO-SAM is a tightly coupled localization framework based on the fusion of LiDAR and inertial navigation systems, achieving high-precision real-time trajectory estimation and mapping through factor graph optimization; YOLOv8 is a computer vision framework.

[0021] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating the laser and vision-based power equipment inspection path planning method of this application. Figure 1 As shown, the power equipment inspection path planning method based on laser and vision includes: S1: A multi-sensor system is constructed based on 3D LiDAR, IMU, camera and odometer. The external parameters of each sensor are calibrated and time is synchronized. An improved LIO-SAM algorithm is used for environmental modeling and localization. In this process, a priori constraints on power structure and a dynamic point cloud removal mechanism are introduced into the factor map to achieve high-precision mapping and global localization. Among them, the prior constraint of the power structure is to identify power equipment (busbars, cabinets, insulators, etc.) with regular geometric features in the substation by using cameras, and to obtain geometric primitives (straight lines, planes, cylinders, etc.) by fitting them in the point cloud. The geometric primitives are then injected as static Landmarks into the factor map optimization process to enhance the geometric consistency of the map. Among them, the dynamic point cloud elimination mechanism is to visually detect dynamic objects (people, vehicles, etc.) and combine them with time-series trajectory analysis to judge the mobility of dynamic objects, and then eliminate the corresponding point cloud data, so as to retain only the static environmental point cloud for mapping and positioning. S2: Power equipment identification is performed using an improved YOLOv8 model. The improved YOLOv8 model embeds a structure-enhanced attention mechanism in the feature extraction stage, adopts an occlusion perception training strategy and combines a difficult example contrast distillation mechanism in the training stage to reduce the false detection rate in complex backgrounds, and combines multi-sensor projection mapping and point cloud fitting to calculate the three-dimensional coordinates of power equipment in the map. Among them, the detection head of the improved YOLOv8 model is the Structural Constraint Orientation Regression Head (SCA-Head). In addition to regressing the traditional center and scale, the detection head also regresses the target orientation and includes a direction regularization term in the loss function to enhance the detection stability of slender oriented devices (such as busbars and switch rods). The improved YOLOv8 model has a geometrically consistent cross-modal attention module between the backbone and neck. The geometrically consistent cross-modal attention module generates a depth or normal cue map by projecting sparse point clouds onto the image and calculates attention weights based on point density and normal consistency. This enhances the feature response of geometrically supported pixels and automatically reduces the weights of pure texture or regions without geometric support during the feature fusion stage. Distillation of hard examples is a training strategy that includes building a feature prototype memory for each type of power equipment, incorporating highly confusing hard negative samples that are manually labeled or rule-mined into the feature prototype memory, applying contrastive loss to the classification branch to encourage sample features to move closer to their positive class prototypes and away from hard negative prototypes, and combining the soft labels of the teacher network for distillation training. For the three-dimensional coordinates of power equipment, the geometric parameters are obtained by projecting the orientation box onto the laser coordinate system according to the camera intrinsic parameters and selecting the point cloud return point closest to the projection ray. Then, a robust fitting of a straight line, plane or cylinder (such as RANSAC, PROSAC) is performed on the subset of the point cloud to determine the geometric parameters and coordinates accordingly. S3: Based on the calculated set of coordinates of the power equipment, an improved path planning algorithm with a composite cost function and a dynamic task reordering mechanism is used to generate the optimal inspection path that satisfies the constraints of inspection priority, timing and sequence. In this application, the composite cost function includes a priority penalty term, a due date penalty term, and a sequential default penalty term; The dynamic task reordering mechanism updates the task sequence at runtime based on the comparison between the estimated future arrival time and the task deadline, according to the strategy (e.g., skipping, delaying, or skipping and re-inserting), and performs only local improved A* replanning on the affected interval to ensure real-time performance. S4: Based on the optimal inspection path, dynamic obstacles are detected by combining semantic and geometric information. Obstacle avoidance is achieved by using local trajectory adjustment and path replanning to obtain the planned path. In this application, for the detection of dynamic obstacles, Kalman filtering is used to predict the two-dimensional position and velocity of the obstacle. Based on the prediction, a probability safety domain penalty term is added to the local spatiotemporal trajectory optimization cost to achieve prediction-driven obstacle avoidance. For local trajectory adjustment, a spatiotemporal constraint optimization based on time elastic band is adopted. The spatiotemporal distance penalty of predicting dynamic obstacles is added to the objective function, and soft constraints are applied to velocity, angular velocity and curvature to obtain a smooth and executable obstacle avoidance trajectory. For path replanning, the triggering condition is that the main channel is predicted to be blocked for a period of time exceeding a preset threshold, or the task completion time is expected to exceed the allowed time window after local replanning is triggered. S5: Drive the inspection robot to move autonomously along the planned path, collect images and infrared data at the target power equipment, and feed the results back to the background system to complete the closed-loop inspection.

[0022] For step S1 above, for environmental modeling and localization, a 3D LiDAR is used as the main sensor, fused with IMU and odometer, and an improved LIO-SAM with a factor graph framework is used to complete mapping and localization. In view of the structural regularity of substations and the characteristics of high personnel and vehicle mobility, two special mechanisms are introduced: (1) Power structure prior constraints - extract stable geometric priors for equipment such as busbars, cabinets, and disconnectors and write them into the factor graph to improve local and global geometric consistency; (2) Dynamic point cloud removal - use visual semantics and temporal consistency to jointly determine dynamic points, continuously maintain static sub-maps, and suppress drift and error closure. Output the pose flow and dense / sparse map of the inspection robot in the global coordinate system of the substation to provide a basis for subsequent path planning.

[0023] Specifically, when applied, step S1 includes sensor system, coordinate system and calibration requirements, as well as time synchronization and data management, point cloud preprocessing and feature extraction, laser-inertial tight coupling and factor graph baseline, power structure prior constraints, vision-temporal joint dynamic point cloud culling, factor graph construction, sliding window management and loop closure verification.

[0024] Regarding sensor system, coordinate system, and calibration requirements, the multi-sensor system in this application includes lidar. IMU ,camera With chassis odometer Define the world coordinate system Coordinate system with inspection robot Each external reference is Before deployment, hand-eye calibration and checkerboard grid joint point cloud planar registration were used to solve the problem. Then switch to The calibration residuals are controlled within the range of rotation error <0.5° and translation error <5mm to ensure consistency between visual and point cloud projection. This represents the coordinate transformation matrix from the camera coordinate system to the lidar coordinate system, used to describe the rotation and translation relationship of the camera in the lidar coordinate system. This matrix is ​​obtained by hand-eye calibration and checkerboard-point cloud plane registration to ensure that the visual detection results are consistent with the spatial coordinate projection of the point cloud.

[0025] For time synchronization and data management, the laser frame time is used as the master clock. The IMU uses forward pre-integration interpolation to the laser point time to eliminate motion distortion of laser scanning (corrected by the beam timestamp). Camera and odometer signals are mapped to the laser time axis through linear interpolation. The time delay between multiple sensors is estimated online (by minimizing the path difference between the radar odometer and the IMU) to control the delay error to <3ms.

[0026] For point cloud preprocessing and feature extraction, each frame of point cloud is first denoised by statistical filtering, then downsampled by voxel filtering (voxel size 5~10cm), the point curvature is calculated and corner points and planar points are extracted layer by layer according to the LOAM approach, RANSAC plane fitting is used on the ground to determine the direction of gravity and assist in registration, and high reflective metal surfaces (which are prone to generating ghost points) are removed by intensity-distance joint thresholding and given small weights in subsequent matching.

[0027] For laser-inertial tight coupling and factor map baseline, the state of the positioning system in a multi-sensor architecture within a sliding window is represented as follows: ; Indicates the first [number]th ... The state quantity at each moment Indicates the IMU coordinate system at the th The rotational attitude relative to the world coordinate system at any given moment is used to describe the orientation of the machine. This represents the IMU's position coordinates in the world coordinate system, used to describe the system's translational position in space. This represents the linear velocity of the IMU in the world coordinate system. This describes the systematic bias in IMU angular velocity measurements and is used to compensate for IMU angular velocity drift. Describe the systematic bias in IMU acceleration measurements; The optimization objective is to minimize the IMU pre-integration, laser matching, and loop closure residuals. ; in, Indicates the IMU measurements at adjacent times. and The constraint residuals formed between the state variables are used to reflect the difference between the pre-integration measurement and the state estimate, and to constrain the temporal continuity of attitude, velocity and bias. , These represent the first and second parts of the sliding window, respectively. , No. The state vector of the system at each moment; This represents the geometric residual obtained from lidar point cloud registration, including corner-to-line residuals and planar point-to-surface residuals, used to constrain the spatial consistency between the current pose and the observation data. For the first Time-lapse laser observation; This indicates when a historical frame is detected. With the current frame Pose constraints are introduced when laps exist to suppress cumulative drift and achieve global consistency optimization.

[0028] The laser residual is calculated using point-to-line or surface distance, combined with Huber kernel to suppress outliers; the sliding window size is typically 60~120 frames, and iSAM2 is used for optimization to ensure real-time performance.

[0029] For the prior constraints of the power structure, the parallel busbars, cabinet arrangement, insulator strings, etc. in the substation generate highly regular geometric primitives. Conventional SLAM is prone to matching ambiguity and cumulative drift when facing regular structures. By explicitly injecting these structures as long-term static landmarks into the optimization, the map can be "anchored" by geometric constraints. First, the improved YOLOv8 model, pre-trained on the camera, is used for semantic segmentation to identify the category of power equipment. Then, the visual mask is based on extrinsic parameters. , Projecting laser frames yields a subset of point clouds belonging to the current power equipment category. ,right Using PANSAC to fit the corresponding geometric primitives: straight lines (busbars, crossarms), planes (cabinets, walls), and cylinders (insulator segments), a set of parameters is obtained. For example, for straight lines... ,flat ,cylinder ; The constraint forms are as follows: Line factor (residual distance from point to line): ; This represents the coordinates of a point in the point cloud dataset belonging to the power equipment target. Its coordinates are defined in the lidar coordinate system and are used to calculate the geometric residual. Represents the center point of a geometric primitive, corresponding to the coordinates of the center point of the geometric primitive (such as a line, cylinder, or rod). Represents the direction vector of the primitive element, and represents the unit direction vector of the axis of a line or cylinder; Plane factor (algebraic distance from a point to a plane): ; This represents the plane normal vector, specifically the unit normal vector of a planar geometric primitive (such as a wall or frame), used to calculate the algebraic distance residual from a point to a plane. The set of parameters representing the equation of a plane or a geometric surface. The constant term in the plane equation determines the offset of the plane in space; Cylindrical factor (radius deviation): ; The fitting radius represents the radius of the cylinder (such as insulator strings, tower supports, etc.), which is used to calculate the radial distance residual from the point cloud to the cylinder surface; These residuals constitute new factors for the keyframe pose and primitive parameters, and are added to the overall cost with weights:

[0030] in, The structural constraint weight coefficient represents the weight of the structural geometric factor in the overall optimization objective, and is used to balance the influence between the geometric consistency residual and the sensor constraint residual (IMU, LiDAR, Loop). The structural factor is activated only when the following conditions are met simultaneously: visual confidence. And the consistency of visual-point cloud projection Weights are based on Adaptive settings This represents a global adjustment factor for structural factors, which adjusts the overall influence of different types of structural factors (such as planar constraints, edge constraints, and corner constraints). The geometric consistency index between the visual detection box and the point cloud projection box is defined as the intersection-union ratio of the two regions, reflecting the degree of spatial overlap between the visually detected target and the point cloud spatial projection target. If the detection is occluded or the center point is lowered, it will be automatically downweighted or deactivated to avoid incorrect constraints.

[0031] For visual-temporal joint dynamic point cloud removal, substation inspections may encounter short-term personnel, parked or moving vehicles, construction equipment, etc. If these dynamic echoes are written into the static map, it will lead to closed-loop errors and positioning jumps. The judgment and elimination process is as follows: First, the dynamic categories (people, vehicles, mobile machinery) are identified using the improved YOLOv8 model. The detection bounding boxes are then projected onto the point cloud and the corresponding point sets are labeled. Then in the time window For each candidate point cluster, calculate the centroid trajectory and velocity. ,like If the centroid variance is significantly greater than the background, it is judged as dynamic; otherwise, it is retained as static (e.g., stationary vehicles should not be immediately removed); if the point cluster remains relatively fixed at multiple different times and is rigidly matched with the nearby structure (e.g., falling on the surface of a cabinet), it is not marked as dynamic. This indicates the speed threshold for determining dynamic targets, used to distinguish between "stationary" and "moving" targets; Points identified as dynamic are not included in the current frame and global registration, nor are they written to the static submap; these points are written to a short-term "dynamic cache" for obstacle avoidance and short-term tracking. The static submap uses voxel hash storage; if a voxel is marked as dynamic N times consecutively, the confidence level of that voxel is exponentially decayed to prevent long-moving objects from being mistakenly written as static.

[0032] For factor graph construction, sliding window management, and loop closure verification, the factor graph includes IMU pre-integration factors, laser matching factors, structural factors, and loop closure factors. Keyframes are triggered by pose changes (translation > 0.5 m or rotation > 2°). Loop closure candidates are retrieved using Scan-Context semantic-geometric descriptors. After verification by ICP / GICP and structural consistency checks (primitive alignment error threshold < 5 cm), candidates are added to the loop closure factor to trigger global optimization. The structural factor provides additional geometric anchoring in loop closure optimization, avoiding incorrect matching caused by duplicate structures.

[0033] For step S2 above, which involves target equipment identification and localization, step S2, based on the global map and pose provided in step S1, completes the automatic identification and 3D localization of key power equipment in the substation. The identification side employs an improved YOLOv8 depth detection network, addressing the challenges of similar appearances, slender structures, and densely packed small targets in power scenarios through structural innovation. The localization side uses camera-laser cross-modal projection to extract the geometric subset corresponding to the target from the point cloud and perform geometric fitting, outputting the 3D coordinates and uncertainty of the equipment in the global coordinate system. Step S2 only relies on the interface of step S1: the pose after timestamp alignment. External reference Camera internal parameters The calibration process is not repeated.

[0034] For the overall process of step S2, at time... The camera acquires images and feeds them into the improved YOLOv8 to obtain the device category, bounding box (or mask), and confidence score. The detection results are then processed via extrinsic parameters. Projecting onto the lidar coordinate system, a subset of the target point cloud consistent with the detection area is selected from the full point cloud. Then, geometric fitting (linear, planar, or cylindrical) is performed on this subset to obtain the device's geometric parameters and 3D reference points. Finally, the output from step S1 is used... The results are transformed to a global coordinate system to generate a description of the device's position and attitude in the world coordinate system, and the localization covariance is estimated. To ensure cross-frame stability, multi-frame confirmation and temporal consistency filtering are used, and the final target is output only when threshold conditions are met for multiple consecutive frames.

[0035] Specifically, step S2, when applied, includes structure prior-guided bounding box regression, geometrically consistent cross-modal attention, hard example contrastive distillation and prototype memory (HNCD), cross-modal projection and target point cloud extraction.

[0036] For structure-constrained oriented head (SCA-Head) regression, to accurately identify slender and significantly directional electrical equipment such as busbars and disconnector links, this application introduces a "Structure-Constrained Oriented Head" (SCA-Head) into the YOLOv8 detection head. This detection head simultaneously regresses the object's center, size, and orientation parameters, and outputs the results as follows: ,in This represents the direction of the major axis in the image plane. Incorporating prior knowledge of the power structure (e.g., busbars tend to be horizontal, insulators are mostly approximately vertical), a direction regularization term is added to the loss function. The joint loss of classification, box regression, and direction estimation is defined as:

[0037] in, To represent the category classification loss, BCE is used. , , This represents the weighting coefficient, used to balance the contributions of various losses; in, CIoU loss is used; Using cosine distance form ; Used for injecting structural priors, such as horizontal priors for busbars. , written as This object-oriented regression approach reduces bounding box jitter for slender targets and improves geometric consistency with subsequent line / cylinder fittings.

[0038] To suppress false detections and improve the separability of small targets under complex backgrounds and reflective conditions, this application adds "Geometry-Consistent Cross-Modal Attention" (GCCA) between the backbone and neck regions for geometrically consistent cross-modal attention. Specifically, this involves using the current sparse laser point cloud... By projecting onto the image plane, a sparse depth and normal cue map is formed. For any intermediate feature map, compute an attention weight driven by depth sparsity and normal stability:

[0039] in, , This represents the modal fusion balance coefficient (empirically set to 0.6 and 0.4). It is point density normalization. It is a measure of local normal consistency. It is a sigmoid function. The cross-modal weighted feature is... Relying solely on geometric consistency signals without introducing additional supervision, it can improve semantic resolution in areas supported by point clouds and automatically reduce weights in texture-disturbed areas without geometric support, thereby reducing the response of false targets such as metal grids and cable shadows.

[0040] To address the false detection problem of "similar device appearance and blurred class boundaries," this application introduces "Hard-Negative Contrastive Distillation" (HNCD) during the training phase, constructing a small class prototype memory (the feature mean of each device class), and applying a contrastive loss to the penultimate layer of features in the classification branch.

[0041] in, This represents the function for calculating cosine similarity between features. The feature mean of the category, Prototype features representing categories This represents the temperature coefficient. "Strongly interfering structures" (such as steel components and cable shadows) identified manually or through rules are added to the memory bank as difficult-to-bear samples. Combined with the soft labels of the distillation teacher network, this significantly reduces the false detection rate of "appearing devices." This learning strategy is tailored to the power industry scenario and addresses the easily confused nature of repetitive metal structures in substations.

[0042] For cross-modal projection and target point cloud extraction, after detecting the device's bounding box or mask, extrinsic parameters are used. Project the image coordinates onto the LiDAR coordinate system, and select the set of point clouds that intersects with this region as the subset of the target point cloud. For each pixel... If a point cloud is returned by the projected ray, that point is included in the target; otherwise, the confidence is attenuated by the geometric attention weights of GCCA to avoid mistakenly incorporating "pure texture" regions into the subset. Through this one-to-one semantic-geometric constraint, the target point cloud subset has high purity, providing clean data for subsequent geometric fitting.

[0043] Different geometric models are used for different power equipment: the busbar and disconnector linkage are modeled as linear models. The cabinet and the wall are modeled in two planes. Insulators and surge arresters are modeled in cylindrical form. Robust fitting was performed using RANSAC / PROSAC, and the residuals were defined as the distance from the point to the line, the algebraic distance from the point to the plane, and the deviation from the point to the cylinder radius, respectively. Three-dimensional reference points for power equipment. Based on the model, the centroids are determined as follows: for linear types, the centroids are taken from the midpoint or endpoints within the assembly length range; for planar types, the centroid projection of the visible contour is taken; and for cylindrical types, the intersection of the axis and the mid-section is taken. Finally, the global coordinates are obtained using the pose transformation in step S1. And the uncertainty is propagated through the first-order error. ,in, This is obtained by linearizing the pose using external participants. This combines the point cloud fitting residuals with the sensor measurement variance. (Record) As a confidence output for subsequent steps.

[0044] For step S3 above, which generates the inspection path, step S3 is the global three-dimensional coordinate set of the target power equipment output in step S2. Based on this, the optimal motion path of the inspection robot that meets the needs of power line inspection is generated. Unlike traditional path planning that only considers geometric optimality, this application focuses on introducing power line inspection task constraints and dynamically adjusts the order of objectives during the planning process, thereby ensuring that the robot can not only complete the path but also conforms to the inspection business logic.

[0045] Specifically, when applied, step S3 includes inputs and outputs, constraint-driven composite cost functions, and DTR-A*.

[0046] For input and output, the global raster map output in step S1 is... The set of three-dimensional coordinates of the device output in step S2 Task scheduling parameters (device priority) ,time limit Sequence dependency As input, output the globally optimal inspection path. Each sub-path represents the optimal travel path for the inspection robot between two devices, and includes task scheduling information (estimated arrival time, remaining battery power, etc.).

[0047] For constraint-driven composite cost functions, in the traditional A* algorithm, the cost function mainly consists of geometric distance or energy consumption. This application proposes a constraint-driven composite cost function that incorporates the unique requirements of power line inspection into the heuristic evaluation. The total cost function is defined as follows: ,in, From the starting point to the node The cumulative cost, This is a heuristic estimation (using Euclidean distance). The constraints on the task include: Priority penalty: If the node corresponds to a higher-priority device, a late arrival penalty is added to the path cost. ; Timed penalty: If the estimated arrival time exceeds the task deadline, the cost increases exponentially. ; Indicates the estimated arrival time; Sequential dependencies: if constraints exist Requirements for electrical equipment exist Previously, during inspections, violations would result in immeasurable penalties. In summary, the constraint cost is defined as ; This represents a sequence dependency penalty term, used to constrain the inspection sequence logic between tasks or devices. When the sequence logic requirement between devices is violated, this term incurs a positive penalty; otherwise, it does not. =0; This cost function enables the planning path to not only consider geometric optimality but also automatically comply with the business logic of power inspection.

[0048] For DTR-A*, the traditional A* assumes a fixed objective, which makes it difficult to handle multiple device objectives and task dynamism. This application proposes DTR-A*: introducing a dynamic task reordering strategy under the A* framework to transform the inspection task into a constrained traveling salesman problem (TSP with constraints). The algorithm steps are as follows: First, generate a shortest path cost table between all devices based on the cost function. Second, use a heuristic sorter (priority, timing) to generate the initial inspection order. Third, dynamically adjust the order during the search process: if the estimated arrival time at a device has exceeded the time limit, skip it and re-insert it into a suitable position; if a high-priority task suddenly appears, insert it immediately. For order adjustments, use a local restart (A*) to replan only the affected local paths, avoiding global redundant calculations. While ensuring scalability, it can handle common high-priority queue jumping and timed inspection windows in power inspection tasks.

[0049] For step S4 above, which is the dynamic obstacle avoidance and path replanning module, specifically, when applied, step S4 includes dynamic obstacle detection and recognition, obstacle motion modeling and trajectory prediction, and prediction-based local path optimization.

[0050] For dynamic obstacle detection and recognition in substation scenarios, inspection robots may encounter dynamic obstacles such as workers, maintenance vehicles, or temporary construction equipment. This application first employs a multi-sensor fusion mechanism for obstacle detection: an improved YOLOv8 model identifies dynamic categories such as "people, vehicles, and construction machinery" in real time and outputs a two-dimensional detection box; using camera-laser extrinsic parameters, the detection box is projected onto a point cloud frame, and the corresponding obstacle point cloud clusters are extracted to obtain their precise three-dimensional position and size; within a time window, the motion trajectory of the centroid of the obstacle point clusters is tracked, and the velocity vector is calculated. ,like or trajectory variance If the visual perception is affected by lighting or occlusion and the dynamic object is not identified, the identification is supplemented by the density change and motion consistency detection of the point cloud. This indicates the speed threshold, used to distinguish between static and dynamic obstacles. This represents the trajectory variance, indicating the degree of positional fluctuation of the centroid of the target point cluster within the time window. If the variance is significantly greater than the stability variance of the background point cloud, it indicates that the target is continuously moving.

[0051] For obstacle motion modeling and trajectory prediction, to avoid the passive detours of traditional obstacle strategies, this application introduces an obstacle trajectory prediction mechanism: the obstacle state is defined as... ,in For two-dimensional position, The velocity component is used. A uniform motion model is adopted, combined with Kalman filtering iterative prediction of the future state:

[0052] in, Given Gaussian process noise, we obtain the obstacle position distribution for the next k time steps and generate a probabilistic safety region as a subsequent obstacle avoidance constraint. Represents the state vector. This represents the state vector at the next moment. This represents the time interval, indicating the time difference between two observation frames. Therefore, the inspection robot can avoid obstacles it's about to enter the passage in advance, instead of waiting until it encounters them to temporarily detour.

[0053] For prediction-based local path optimization, the inspection robot employs an improved Timed Elastic Band (TEB) local optimization method when executing the global path. The trajectory is represented as a set of time-sequential nodes. The optimization objective function is:

[0054] in, This comes at the cost of trajectory smoothness. To ensure smooth steering, even at the cost of changes in angle; To compensate for the spatiotemporal distance penalty with respect to the predicted obstacle safety domain, it is defined as... , As weights, adaptive adjustment (when obstacles approach, (Automatic elevation); Indicates at time The predicted obstacle position obtained by the dynamic obstacle trajectory prediction module is used to calculate the spatiotemporal distance constraint between the trajectory node and the predicted obstacle, so that the local path can maintain obstacle avoidance safety in the dynamic environment. If obstacle prediction shows that a critical passage is blocked for a long time (such as a bus corridor being blocked by large vehicles), and local optimization cannot bypass the blockage, a global replanning is triggered: the local module reports the blocked area to the global planner, and the global module regenerates the target point access order and path, giving priority to avoiding the long-term blocked area, so as to ensure that the inspection task can still be completed under time constraints.

[0055] For step S5 above, which involves autonomous inspection, after path planning and dynamic obstacle avoidance, the inspection robot needs to accurately execute the predetermined trajectory. The execution layer receives the locally corrected path point sequence and converts it into speed commands. Motion control is achieved through chassis drive. The control strategy combines error feedback and predictive control to ensure the inspection robot can operate smoothly in complex substation environments, avoiding overshoot and jitter.

[0056] When the inspection robot approaches the coordinates of the target power equipment, it performs fine-tuning actions (such as rotation and lateral movement) based on the positioning results and the equipment's spatial posture to ensure that the camera and infrared sensor are at a suitable viewing angle, avoiding data loss due to obstruction or excessive angle. Simultaneously, it monitors the power equipment identification confidence level; if it falls below a threshold, it triggers a small-scale position correction until the data acquisition conditions are met.

[0057] The following combination Figure 2 The laser and vision-based power equipment inspection path planning method of this application is described in detail.

[0058] Step 1: Sensor system calibration and time synchronization The inspection robot is equipped with a 3D LiDAR, IMU, camera, and odometry. Before deployment, the extrinsic parameters of the camera-laser and IMU-laser are obtained through checkerboard calibration and point cloud planar registration. The calibration error is controlled within rotation <0.5° and translation <5 mm. During operation, the laser frame is used as the master clock. The IMU data is interpolated to the laser point time using forward pre-integration. The camera and odometry signals are synchronized to the laser time axis through linear interpolation, thereby ensuring the spatiotemporal alignment of multi-sensor data.

[0059] Step 2: Environment Modeling and Global Localization Laser-inertial tightly coupled mapping and positioning are performed based on an improved LIO-SAM system. To adapt to the substation environment, the system introduces two dedicated mechanisms: Prior constraints on power structure: Improve the YOLOv8 model at the camera end to identify equipment such as busbars, cabinets, and insulators, and fit linear, planar, and cylindrical geometric primitives in the point cloud. Use these primitives as long-term static landmarks to inject into the factor map to enhance the geometric consistency of the map. Dynamic point cloud removal: Dynamic objects such as people and vehicles detected are removed by combining visual recognition and temporal trajectory analysis, while only static sub-maps are maintained, thereby suppressing drift and erroneous loop closure.

[0060] The final output includes the pose trajectory of the inspection robot in the global coordinate system of the substation and a dense / sparse map, and provides a set of structural primitives and static sub-map interfaces for downstream modules to call.

[0061] Step 3: Target device identification and 3D coordinate calculation An improved YOLOv8 model is used for device identification, and the following three scenario-based innovations are implemented to improve the robustness and accuracy of identification and localization: Structural-Aware Attention: A structural-aware attention submodule is embedded in the neck part of YOLOv8. This module uses convolutional attention and spatial geometric cues (such as combining local edge strength or projection depth cues) to emphasize pixel responses consistent with the geometric features of electrical equipment, thereby suppressing spurious responses in similar backgrounds (steel structures, cable networks) and improving the detection stability of slender and regular equipment (such as busbars and disconnectors). Occlusion-aware training strategy: During the training phase, simulated occlusion (random rectangular / irregular occlusion), partial visibility annotations, and visibility weight adjustments are added to the dataset. By implementing weighted learning for samples with low visibility in the loss function, the recall ability of the model in partially occluded scenarios is improved. During deployment, a multi-frame verification strategy is combined (the target must appear in several consecutive frames or pass geometric consistency verification) to reduce the impact of single-frame false detections on the system. Hard Example Contrastive Distillation and Prototype Memory (HNCD): During the training phase, a category prototype memory bank is established to store the feature centers of various types of equipment and typical hard negative sample features (such as steel structures and cable shadows). Contrastive loss is introduced in the classification branch to bring samples closer to their correct prototypes and away from hard negative prototypes. At the same time, teacher network soft label distillation is used to improve the model's ability to distinguish hard examples. This strategy helps to reduce false alarms of "equipment with similar styles but not equipment". Subsequently, the 2D mask / orientation box is projected onto the point cloud coordinate system using extrinsic parameters. A subset of the point cloud is extracted, and a geometric model (line / plane / cylinder) is selected according to the equipment type for robust fitting to obtain the 3D geometric parameters and reference points of the power equipment.

[0062] Step 4: Inspection Route Planning Using the coordinates of the target power equipment as the set of inspection target points, an improved path planning method is employed to generate the optimal inspection path that satisfies business constraints. This process mainly consists of two parts: A composite cost function is introduced for path search, which comprehensively considers geometric cost (path length, turning smoothness), energy cost (estimated energy consumption), and task constraint cost (equipment priority, arrival time limit, sequence dependency, no-passage / speed-limit zone penalty). The task constraint cost is used to guide the search in heuristic evaluation. If a path may cause serious violation of sequence or time limit, it will be given a higher penalty or marked as infeasible, thus directly reflecting the inspection business logic in the global planning stage. Treating the multi-objective inspection problem as a constrained route scheduling problem, we first calculate the shortest path cost table between power equipment, and then generate an initial order based on the cost and task constraints. During the inspection process, we evaluate the expected arrival time and task status in real time. If a high-priority emergency task or a target is about to be overdue, we trigger task rescheduling. We adopt a local restart-based improvement A* that only replans the affected intervals to avoid global recalculation and ensure real-time performance.

[0063] Step 5: Dynamic obstacle avoidance and path replanning During execution, dynamic obstacles (people, vehicles, construction equipment, etc.) are perceived through semantic-geometric fusion, and prediction-driven local obstacle avoidance and global replanning when necessary are achieved.

[0064] Obstacle clusters are extracted based on visual inspection results and point cloud projection. Their speed and motion patterns are estimated using temporal tracking to determine whether they are dynamic or static. Visual failures can be compensated for by changes in point cloud density and motion detection. A simplified dynamic model is established for obstacles identified as dynamic, forming a probabilistic safety region. An improved local optimization is employed based on predictive information, incorporating a penalty term for the spatiotemporal distance to the obstacle into the cost function to generate a smooth and preventative obstacle avoidance trajectory, prioritizing continuity and comfort under safe conditions. If prediction or observation indicates that the main passage is blocked for an extended period or local obstacle avoidance cannot complete the task within the time window (e.g., the continuous blocking time exceeds a threshold or the estimated task completion time after replanning exceeds the allowable range), the blocking information is reported to the global path generation module, triggering a constrained global replanning to update the subsequent inspection sequence and path.

[0065] Step 6: Autonomous Inspection and Data Collection The inspection robot moves to the vicinity of the target power equipment according to the planned path and collects images and infrared data at the designated observation pose. After identifying abnormal signals (such as temperature rise or equipment surface damage), it sends the results back to the backend system. After completing the inspection task, the inspection robot outputs a complete map, pose trajectory, and inspection data report, forming a closed-loop autonomous inspection process.

[0066] Secondly, embodiments of this application also provide a power equipment inspection path planning device based on laser and vision.

[0067] In one embodiment, reference is made to Figure 3 , Figure 3 This is a schematic diagram of the functional modules of the power equipment inspection path planning device based on laser and vision according to this application. Figure 3 As shown, the laser and vision-based power equipment inspection path planning device includes: an environment modeling and positioning module, a target equipment identification and positioning module, an inspection path generation module, a dynamic obstacle avoidance and path replanning module, and an autonomous inspection execution and data acquisition module.

[0068] The environment modeling and localization module is used to construct a multi-sensor system based on 3D LiDAR, IMU, camera, and odometer, perform extrinsic parameter calibration and time synchronization of each sensor, and adopt an improved LIO-SAM algorithm for environment modeling and localization. This algorithm incorporates prior constraints on power structure and a dynamic point cloud culling mechanism into the factor graph. The target equipment identification and localization module is used to identify power equipment using an improved YOLOv8 model. This improved YOLOv8 model embeds a structure-enhanced attention mechanism in the feature extraction stage, employs an occlusion-aware training strategy combined with a difficult-example comparative distillation mechanism in the training stage, and combines multi-sensor projection mapping and point cloud fitting to calculate the location of power equipment. The system includes three-dimensional coordinates on the map; an inspection path generation module, based on the calculated set of coordinates of the power equipment, uses an improved path planning algorithm with a composite cost function and dynamic task reordering mechanism to generate the optimal inspection path that satisfies inspection priority, timing, and sequence constraints; a dynamic obstacle avoidance and path replanning module, based on the optimal inspection path, detects dynamic obstacles by combining semantic and geometric information, and uses local trajectory adjustment and path replanning to achieve obstacle avoidance and obtain the planned path; and an autonomous inspection execution and data acquisition module, which drives the inspection robot to move autonomously according to the planned path, collects images and infrared data at the target power equipment, and feeds the results back to the background system to complete the closed-loop inspection.

[0069] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0070] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0071] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0072] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0073] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0074] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for power equipment inspection path planning based on laser and vision, characterized in that, The laser and vision-based power equipment inspection path planning method includes: A multi-sensor system was constructed based on 3D LiDAR, IMU, camera and odometer. The external parameters of each sensor were calibrated and time was synchronized. An improved LIO-SAM algorithm was used for environmental modeling and localization. In this process, prior constraints on power structure and dynamic point cloud removal mechanism were introduced into the factor graph. Power equipment identification is performed using an improved YOLOv8 model. The improved YOLOv8 model embeds a structure-enhanced attention mechanism in the feature extraction stage, adopts an occlusion perception training strategy and combines a difficult example contrast distillation mechanism in the training stage, and calculates the three-dimensional coordinates of the power equipment in the map by combining multi-sensor projection mapping and point cloud fitting. Based on the calculated set of coordinates of the power equipment, an improved path planning algorithm with a composite cost function and a dynamic task reordering mechanism is used to generate the optimal inspection path that satisfies the constraints of inspection priority, timing and sequence. Based on the optimal inspection path, dynamic obstacles are detected by combining semantic and geometric information. Obstacle avoidance is achieved by using local trajectory adjustment and path replanning, and the planned path is obtained. The inspection robot is driven to move autonomously along the planned path, collect images and infrared data at the target power equipment, and feed the results back to the background system to complete the closed-loop inspection.

2. The method for planning inspection paths for power equipment based on laser and vision as described in claim 1, characterized in that: The prior constraint of the power structure is to identify power equipment with regular geometric features in the substation by camera, and obtain geometric primitives by fitting them in the point cloud. The geometric primitives are then injected as static landmarks into the factor map optimization process to enhance the geometric consistency of the map. The dynamic point cloud culling mechanism detects dynamic objects visually and analyzes their movement based on time-series trajectories to remove the corresponding point cloud data, thus retaining only the static environmental point cloud for mapping and localization.

3. The method for planning inspection paths for power equipment based on laser and vision as described in claim 1, characterized in that: The detection head of the improved YOLOv8 model is a structural constraint orientation regression head. In addition to regressing the traditional center and scale, the detection head also regresses the target orientation and includes a direction regularization term in the loss function. The improved YOLOv8 model has a geometrically consistent cross-modal attention module between its backbone and neck. This module generates a depth or normal cue map by projecting sparse point clouds onto the image and calculates attention weights based on point density and normal consistency. This enhances the feature response of geometrically supported pixels during the feature fusion stage and automatically reduces the weights of pure texture or regions without geometric support.

4. The method for planning inspection paths for power equipment based on laser and vision as described in claim 1, characterized in that: The hard example contrastive distillation is a training strategy that includes constructing a feature prototype memory for each type of power equipment, incorporating highly confusing hard negative samples that are manually labeled or rule-mined into the feature prototype memory, applying contrastive loss to the classification branch to encourage sample features to move closer to their positive class prototypes and away from hard negative prototypes, and combining the soft labels of the teacher network for distillation training.

5. The method for planning inspection paths for power equipment based on laser and vision as described in claim 1, characterized in that: For the three-dimensional coordinates of power equipment, the geometric parameters are obtained by projecting the orientation box onto the laser coordinate system according to the camera intrinsic parameters and selecting the point cloud return point closest to the projection ray. Then, a robust fitting of a straight line, plane, or cylinder is performed on the subset of the point cloud to determine the coordinates.

6. The method for power equipment inspection path planning based on laser and vision as described in claim 1, characterized in that, The composite cost function includes a priority penalty term, a due date penalty term, and a sequential default penalty term.

7. The method for planning inspection paths for power equipment based on laser and vision as described in claim 1, characterized in that, The dynamic task rescheduling mechanism updates the task sequence according to a strategy based on the comparison between the estimated future arrival time and the task deadline, and performs only local improved A* replanning on the affected interval to ensure real-time performance.

8. The method for planning inspection paths for power equipment based on laser and vision as described in claim 1, characterized in that: For the detection of dynamic obstacles, Kalman filtering is used to predict the two-dimensional position and velocity of the obstacles. Based on the prediction, a probabilistic safe domain penalty term is added to the local spatiotemporal trajectory optimization cost to achieve prediction-driven obstacle avoidance.

9. The method for planning inspection paths for power equipment based on laser and vision as described in claim 8, characterized in that: For local trajectory adjustment, a spatiotemporal constraint optimization based on time elastic band is adopted. The spatiotemporal distance penalty of predicting dynamic obstacles is added to the objective function, and soft constraints are applied to velocity, angular velocity and curvature to obtain a smooth and executable obstacle avoidance trajectory. For path replanning, the triggering condition is that the main channel is predicted to be blocked for a period of time exceeding a preset threshold, or the task completion time is expected to exceed the allowed time window after local replanning is triggered.

10. A power equipment inspection path planning device based on laser and vision, characterized in that, The laser and vision-based power equipment inspection path planning device includes: The environment modeling and localization module is used to build a multi-sensor system based on 3D LiDAR, IMU, camera and odometer, to perform external parameter calibration and time synchronization of each sensor, and to use the improved LIO-SAM algorithm for environment modeling and localization. In this module, the prior constraints of power structure and dynamic point cloud removal mechanism are introduced into the factor graph. The target device identification and localization module is used to identify power equipment using an improved YOLOv8 model. The improved YOLOv8 model embeds a structure-enhanced attention mechanism in the feature extraction stage, adopts an occlusion perception training strategy and combines a difficult example contrast distillation mechanism in the training stage, and calculates the three-dimensional coordinates of the power equipment in the map by combining multi-sensor projection mapping and point cloud fitting. The inspection path generation module is used to generate the optimal inspection path that satisfies the inspection priority, timing and sequence constraints based on the calculated set of coordinates of the power equipment and an improved path planning algorithm with a composite cost function and dynamic task reordering mechanism. The dynamic obstacle avoidance and path replanning module is used to detect dynamic obstacles based on the optimal inspection path, combined with semantic and geometric information, and to achieve obstacle avoidance by using local trajectory adjustment and path replanning to obtain the planned path. The autonomous inspection execution and data acquisition module is used to drive the inspection robot to move autonomously along the planned path, collect images and infrared data at the target power equipment, and feed the results back to the background system to complete the closed-loop inspection.

Images