A kind of unmanned aerial vehicle electric power inspection device and system based on laser radar

Abstract

This invention discloses a UAV power line inspection device and system based on lidar, belonging to the field of intelligent power equipment inspection technology. It includes an inspection management center, which is communicatively connected to the following modules: a fusion modeling module, used to fuse lidar and visual sensors to acquire high-precision 3D point cloud and texture information of power components in real time, and construct a refined 3D model of the power components. This invention achieves reliable data acquisition under complex weather and lighting conditions such as nighttime, fog, haze, and strong backlighting through the fusion perception of lidar and visual sensors. LiDAR provides accurate 3D point clouds unaffected by lighting conditions, while the visual sensor supplements texture information. After fine registration and complementarity, the two form a fusion perception dataset with deep coupling of geometric structure and surface texture, significantly improving the accuracy and completeness of 3D reconstruction of power components.

Description

Technical Field

[0001] This invention relates to the field of intelligent power equipment inspection technology, specifically to a drone-based power inspection device and system based on lidar. Background Technology

[0002] With the rapid development of power infrastructure, traditional manual inspection methods face many challenges, such as high workload, low efficiency, and safety hazards. As a high-precision ranging technology, lidar can quickly acquire three-dimensional spatial data of power facilities under various weather conditions. Combined with UAV platforms, lidar not only improves the efficiency of inspection operations but also greatly reduces labor costs and safety risks. In addition, lidar can realize real-time monitoring of power equipment, promptly detect potential safety hazards, and ensure the stable operation of the power system.

[0003] For example, in the Chinese patent publication number CN119562036A, a method and system for power line inspection based on drones is proposed. The inspection client and server are separated. The server receives the data collected by the client, processes the data, trains the model, and performs quantization and compression before downloading it to the inspection client. The client receives the processed data. It does not require a lot of calculations and can realize local offline and real-time drone obstacle avoidance reminders and fault detection and processing when it cannot be connected to the server.

[0004] In existing technologies, UAV power line inspection relies on visual sensors and GPS for environmental perception and modeling. However, under complex weather and lighting conditions such as nighttime, fog, haze, and strong backlight, the perception capability is severely reduced and component identification fails, resulting in missed power components and insufficient accuracy in 3D reconstruction. At the same time, there is also a reliance on preset static obstacle avoidance areas for flight planning, making it difficult to perceive and respond to dynamic obstacles and environmental changes that occur during the inspection process in real time. It also lacks the ability to conduct real-time risk assessment and adaptive adjustment of the inspection path, which limits the safety and reliability of UAVs in real complex environments. To address these issues, a UAV power line inspection device and system based on lidar is proposed. Summary of the Invention

[0005] To solve the above-mentioned technical problems, the present invention is implemented through the following technical solution: Firstly, a drone-based power line inspection device based on lidar, comprising an inspection management center, wherein the inspection management center is communicatively connected to the following modules:

[0006] The fusion modeling module is used to fuse LiDAR and visual sensors to acquire high-precision 3D point cloud and texture information of power components in real time, build a refined 3D model of power components, improve perception and reconstruction capabilities in complex environments such as night and fog, and provide a data foundation for subsequent identification and risk assessment.

[0007] The risk mapping module, based on real-time point cloud and environmental data, constructs a dynamic inspection risk map, assesses the safety level of the route, identifies potential obstacles and risk areas, and generates a visualized risk heat map, providing a visualized and quantitative risk assessment basis for flight decisions;

[0008] The adaptive flight control module, combining dynamic inspection risk map and fruit fly algorithm, generates the optimal inspection trajectory for safety and efficiency. Through the combination of real-time control on the end side and global scheduling on the cloud side, it performs dynamic replanning of the UAV flight path and emergency obstacle avoidance, ensuring that the UAV can safely complete the inspection task even in emergency environments.

[0009] The component identification module is used to perform feature learning and classification on the collected 3D point cloud and texture information based on deep learning and pattern recognition algorithms under the optimal inspection trajectory. It identifies power components, including towers, conductors and insulators, and detects whether there are structural anomalies such as damage, deformation and detachment. It realizes stable identification and status assessment of power components in complex environments and improves the automation and accuracy of inspection.

[0010] Preferably, the fusion modeling module includes a laser vision fusion perception unit and a three-dimensional structure reconstruction unit;

[0011] The laser vision fusion perception unit is used to synchronously collect multi-source inspection data of power components using lidar and vision sensors, covering high-precision three-dimensional point cloud and texture information. Combined with calibration and registration algorithms, it performs spatial alignment and information complementarity of multi-source inspection data to enhance the robustness and accuracy of environmental perception.

[0012] The three-dimensional structure reconstruction unit, based on the fused three-dimensional point cloud and texture data, and using point cloud registration and surface reconstruction algorithms, reconstructs a refined three-dimensional model of power components including towers, insulators and conductors, supporting subsequent identification and spatial analysis.

[0013] Preferably, the laser vision fusion sensing unit performs the following steps:

[0014] The laser radar and vision sensor on the target UAV are triggered simultaneously to collect multi-source inspection data of the power components, including high-precision 3D point cloud and texture image. The data is then time-stamp aligned and spatially registered to ensure that the data is synchronized in time and space, providing a consistent information basis for subsequent fusion and avoiding the accumulation of registration errors.

[0015] Based on pre-calibrated intrinsic and extrinsic parameters, a multi-sensor fusion calibration algorithm is used to perform pixel-level spatial alignment between 3D point cloud data and texture image data, and to complement the geometric and texture information of multi-source inspection data, thereby achieving high-precision alignment between geometric structure and surface texture, significantly enhancing the information integrity and realism of the 3D model.

[0016] By using feature matching and optimization algorithms to perform fine registration on the fused multi-source inspection data, a spatially consistent fused perception dataset is generated, providing high-quality input for subsequent 3D reconstruction, improving the registration accuracy of point clouds and images, and forming a high-quality, highly consistent fused perception dataset that supports fine-grained modeling and recognition.

[0017] Preferably, the three-dimensional structure reconstruction unit performs the following steps:

[0018] The system receives relevant data from the fusion sensing dataset, extracts the 3D point cloud and texture features of the power components, and performs noise filtering and outlier removal. Multi-source heterogeneous data are represented in a unified manner, providing high-dimensional and complementary input features for subsequent deep neural networks, which significantly improves the robustness and discriminative ability of the model in complex environments.

[0019] The iterative nearest point algorithm is used to register point clouds from multiple perspectives to construct a complete point cloud model of power components. A meshed surface model is generated based on the triangulation method. This model simultaneously completes accurate classification and three-dimensional positioning, realizing automated and integrated perception of multiple types of power components in the inspection scenario, and greatly improving the efficiency and accuracy of inspection operations.

[0020] The registered texture image is mapped onto the reconstructed mesh surface to generate a refined 3D model of the power component with realism and structural details, supporting subsequent identification and spatial analysis. Comprehensive quantitative analysis realizes a closed loop from component perception to health diagnosis, generating accurate reports that can directly guide operation and maintenance decisions, and improving the intelligence level and practical value of inspection.

[0021] Preferably, the risk mapping module includes a dynamic environment modeling unit and a risk level assessment unit;

[0022] The dynamic environment modeling unit is used to fuse the 3D point cloud of lidar and meteorological sensor data in real time to construct a dynamic inspection risk map that includes factors such as obstacles, wind speed, and visibility. It supports real-time risk assessment, real-time fusion of multi-source data, dynamic updating of the risk map, and improved response capability to environmental changes.

[0023] The risk level assessment unit, based on the dynamic inspection risk map and historical inspection data, calculates the safety level of each segment of the path through a risk quantification algorithm, generates a risk heat map, guides flight path optimization, provides trend analysis and risk warning based on historical data, and realizes intelligent optimization of path planning.

[0024] Preferably, the dynamic environment modeling unit performs the following steps:

[0025] It receives 3D point cloud data collected by lidar and simultaneously acquires meteorological data including wind speed, visibility, temperature and humidity monitored by meteorological sensors. It performs spatiotemporal alignment and data fusion to achieve a unified benchmark for multi-source information and lay the foundation for environmental fusion perception.

[0026] Based on point cloud clustering and obstacle recognition algorithms, static and dynamic obstacle information in the inspection environment is extracted. Combined with meteorological data, a multi-dimensional dynamic inspection risk map containing obstacle distribution and environmental status is constructed to clearly identify risk factors.

[0027] The system updates the dynamic inspection risk map based on real-time data, marks risk areas and safe passages, provides real-time environmental modeling support for risk level assessment and route planning, updates the risk situation in real time, and provides dynamic scenario support for safe flight decisions.

[0028] Preferably, the risk level assessment unit performs the following steps:

[0029] Based on the dynamic inspection risk map output by the dynamic environment modeling unit, the obstacle density, wind speed level, and visibility index of each path segment are extracted as risk quantification inputs, effectively quantifying the core risk factors in the inspection environment and providing a structured and computable data foundation for subsequent safety assessments.

[0030] A multi-factor weighted risk assessment algorithm is adopted to calculate the safety score of each route segment and classify the risk level according to the score results, so as to achieve objective and precise classification of the risk of the inspection route and enhance the scientificity and operability of flight decision-making.

[0031] Based on the risk level, a visual risk heat map is generated, and risk trend analysis is performed in combination with historical inspection data. This provides a quantitative decision-making basis for the path optimization of the adaptive flight control module, intuitively displays the risk distribution and changing trends, and provides the flight control system with a real-time and reliable basis for dynamic path optimization.

[0032] Preferably, the adaptive flight control module performs the following steps:

[0033] The system receives the generated risk heat map and inspection task constraints, performs global path search based on the fruit fly algorithm, generates the optimal inspection trajectory that balances safety and efficiency, effectively avoids known risk areas, plans the globally optimal flight route, and significantly improves the safety foundation of the inspection task.

[0034] During the flight of the drone, environmental changes and sudden obstacles are monitored in real time. By combining edge-side perception data with the dynamic inspection risk map in the cloud, local paths are dynamically replanned. The drone can respond to sudden environmental changes and dynamic threats in real time, and autonomously adjust its flight path to ensure the continuous safety and reliability of the inspection process.

[0035] By executing dynamic adjustment commands through the flight controller, the UAV can perform real-time obstacle avoidance and heading correction, ensuring that it can safely and stably complete inspection tasks in complex environments. This achieves precise closed-loop control of flight attitude and heading, ensuring that the UAV can stably perform inspection tasks in complex environments.

[0036] Preferably, the component identification module performs the following steps:

[0037] Based on the fusion sensing dataset, the three-dimensional geometric and texture features of power components are extracted, and a multi-source feature vector is constructed as the recognition input. By fusing geometric and texture features, the system can effectively resist changes in light and background interference, and enhance feature robustness.

[0038] Deep learning networks are used to train and classify feature vectors to achieve automatic identification and positioning of power components, including towers, insulators and conductors. This enables high-precision automatic identification of component categories and accurate three-dimensional spatial positioning, replacing manual identification.

[0039] The system performs structural integrity analysis on the identified power components, detects any structural abnormalities such as damage, deformation, or detachment, and outputs a component health assessment report. It also automatically detects structural defects and generates quantitative reports to support precise operation and maintenance and preventive maintenance.

[0040] Secondly, a lidar-based UAV power line inspection system includes at least one control processor and a memory for communicating with the at least one control processor. The system has a clear architecture, efficient hardware and software collaboration, and facilitates centralized control and modular upgrades. The memory stores instructions that can be executed by the at least one control processor, ensuring stable and solidified control logic and preventing instruction loss or errors during runtime. The instructions are executed by the at least one control processor, enabling it to execute the lidar-based UAV power line inspection device, achieving fully automated inspection and significantly improving operational safety and detection reliability.

[0041] This invention provides a drone-based power line inspection device and system based on lidar. It offers the following advantages:

[0042] (I) The UAV power inspection device and system based on lidar achieves reliable data acquisition under complex weather and lighting conditions such as night, fog, haze, and strong backlight through the fusion perception of lidar and visual sensors. LiDAR provides accurate three-dimensional point clouds that are not affected by lighting, and visual sensors supplement texture information. After fine registration and complementarity, the two form a fusion perception dataset that deeply couples geometric structure and surface texture, which significantly improves the accuracy and completeness of three-dimensional reconstruction of power components.

[0043] (II) This UAV power line inspection device and system based on lidar constructs a dynamic inspection risk map integrating multi-dimensional information such as geometric obstacles and environmental meteorology, and achieves real-time updates and risk assessments at a frequency of no less than 10Hz. By quantitatively analyzing core factors such as obstacle density, wind speed, and visibility, it can dynamically classify path safety levels and generate a visual heat map. When new obstacles are detected or environmental risks increase sharply, it can quickly trigger local path replanning. Combined with aircraft dynamics constraints, it guides the UAV to safely avoid sudden threats, greatly enhancing the autonomous flight safety and adaptability of the UAV in complex and ever-changing environments.

[0044] (III) This UAV power line inspection device and system based on lidar uses an ant colony algorithm for multi-target path planning based on a dynamic inspection risk map. While ensuring the avoidance of medium and high risk areas, it comprehensively optimizes the path length and flight smoothness. The globally optimal trajectory generated by this mechanism improves the overall safety and flight efficiency of the inspection task from the source. At the same time, the system supports the import of task-level constraints, so that the planned route is not only safe and reliable, but also meets the feasibility requirements of the actual task, thus achieving a unity of safety, efficiency and task feasibility. Attached Figure Description

[0045] Figure 1 This is a schematic diagram illustrating the workflow of a drone-based power line inspection device and system based on lidar according to the present invention.

[0046] Figure 2 This is a data flow diagram of a UAV power line inspection device and system based on lidar according to the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Example 1, please refer to Figure 1 , Figure 2 This invention provides a technical solution: a drone-based power line inspection device based on lidar, comprising an inspection management center, which is communication-connected to the following modules:

[0049] The fusion modeling module is used to fuse LiDAR and visual sensors to acquire high-precision 3D point cloud and texture information of power components in real time, build a refined 3D model of power components, improve perception and reconstruction capabilities in complex environments such as night and haze, provide a data foundation for subsequent identification and risk assessment, effectively overcome the limitations of lighting and weather conditions, and realize accurate 3D modeling of power facilities in all weather and all time. The fusion modeling module includes a laser vision fusion perception unit and a 3D structure reconstruction unit.

[0050] The laser-vision fusion perception unit utilizes lidar and vision sensors to simultaneously acquire multi-source inspection data of power components, encompassing high-precision 3D point clouds and texture information. Combining calibration and registration algorithms, it performs spatial alignment and information complementarity of the multi-source inspection data, enhancing the robustness and accuracy of environmental perception. This achieves deep fusion of geometric structures and surface textures, significantly improving the integrity and reliability of the perception data. Simultaneously, it triggers the lidar and vision sensors mounted on the target UAV to acquire multi-source inspection data of power components, including high-precision 3D point clouds and texture images, and performs timestamp alignment and preliminary spatial registration to ensure data spatiotemporal synchronization, providing consistency for subsequent fusion. Based on information foundations and to avoid the accumulation of registration errors, a multi-sensor fusion calibration algorithm is adopted based on pre-calibrated intrinsic and extrinsic parameters to perform pixel-level spatial alignment of 3D point cloud data and texture image data. This complements the geometric and texture information of multi-source inspection data, achieving high-precision alignment of geometric structure and surface texture, significantly enhancing the information integrity and realism of the 3D model. Feature matching and optimization algorithms are used to perform fine registration on the fused multi-source inspection data, generating a spatially consistent fusion perception dataset. This provides high-quality input for subsequent 3D reconstruction, improves the registration accuracy of point cloud and image, and forms a high-quality, highly consistent fusion perception dataset that supports fine modeling and recognition.

[0051] The specific work content is as follows: During the initial stage of the UAV power line inspection operation, the 32-line LiDAR and 20-megapixel high-definition vision sensor on the UAV are simultaneously triggered to collect multi-source inspection data of key power components such as power poles, insulator strings, and conductors at a frequency of 10Hz. The LiDAR acquires 3D point clouds with an accuracy better than ±3cm within an effective range of 100 meters. At the same time, the vision sensor acquires RGB texture images at a resolution of 1920×1080. The hardware time synchronization module ensures that the timestamp deviation between the two types of data is less than 10ms. Based on the UAV's real-time positioning and attitude information, preliminary spatial registration is performed to form a spatiotemporally aligned raw data stream. Based on the pre-completed sensor joint calibration parameters, a multi-sensor fusion calibration algorithm based on a checkerboard calibration board and feature point matching is adopted to perform pixel-level spatial alignment between the 3D point cloud and image data. Specifically, through the extrinsic parameter matrices (rotation matrix and translation vector) of the LiDAR and camera and the intrinsic parameter matrix of the camera, perspective projection transformation is used to map the 3D point cloud to the image coordinate system to achieve millimeter-level spatial alignment. Based on this, texture information is filled into the missing areas of the point cloud, and point cloud geometry is completed in the occluded areas of the image, forming an enhanced fusion dataset with complementary geometric structure and surface texture. This fusion process adopts an iterative optimization method to ensure that the registration error is controlled within 5 pixels, significantly improving the integrity and reliability of data representation. After completing spatial alignment, a fine registration process based on SIFT feature points and ICP (Iterative Nearest Point) algorithm is further used to optimize the fusion data. SIFT feature points in the image are extracted and coarsely matched with FPFH feature descriptors in the point cloud. Then, the RANSAC algorithm is used to remove mismatched point pairs. On this basis, a weighted ICP algorithm is used for fine registration. The optimization objective function is the weighted sum of point-to-surface distance and feature similarity. The iterative convergence threshold is 0.001m. Finally, a fusion sensing dataset with high spatial consistency is generated. This fusion sensing dataset also contains three-dimensional coordinate information, reflection intensity, and texture color attributes. The point cloud density reaches more than 1000 points per square meter, which meets the data requirements for fine three-dimensional reconstruction of power components.

[0052] The 3D structure reconstruction unit, based on fused 3D point cloud and texture data, employs point cloud registration and surface reconstruction algorithms to reconstruct refined 3D models of power components, including towers, insulators, and conductors. This supports subsequent recognition and spatial analysis, generating realistic 3D models with millimeter-level detail. It provides high-quality data support for component recognition and spatial analysis, receiving relevant data from the fused sensing dataset, extracting 3D point cloud and texture features of power components, and performing noise filtering and outlier removal. Multi-source heterogeneous data achieves unified representation here, providing high-dimensional, complementary input features for subsequent deep neural networks, significantly improving the model's robustness and discriminative ability in complex environments. The system employs an iterative nearest-point algorithm to register multi-view point clouds, constructing a complete point cloud model of power components. Based on the triangulation method, it generates a meshed surface model, which simultaneously performs accurate classification and 3D localization. This enables automated and integrated perception of multiple types of power components in inspection scenarios, significantly improving the efficiency and accuracy of inspection operations. The registered texture image is mapped onto the reconstructed mesh surface to generate a refined 3D model of the power components with realism and structural details, supporting subsequent recognition and spatial analysis. Comprehensive quantitative analysis realizes a closed loop from component perception to health diagnosis, generating accurate reports that can directly guide operation and maintenance decisions, thus improving the intelligence level and practical value of inspection.

[0053] The specific work involves: receiving an enhanced fusion sensing dataset from the fusion sensing unit. This dataset contains 3D coordinates, reflection intensity, and RGB texture information. For data extraction from power components (such as towers, insulators, and conductors), a statistical outlier removal algorithm is used, specifically setting the number of neighboring points to 50 and the standard deviation multiple to 1.5 to effectively filter out discrete points caused by environmental reflection or sensor noise. Simultaneously, a radius filtering method is applied, setting the search radius to 0.05 meters to smooth the point cloud while preserving the main structural features. Texture feature extraction is achieved through image segmentation and color space conversion, combining the mapping relationship between point clouds and pixels to ensure consistency between geometric and texture data. After acquiring the preprocessed multi-view point cloud data, an iterative nearest-point algorithm is used for sequence registration. Specifically, the maximum number of iterations is set to 50, the transformation error threshold is 0.001 meters, and point cloud normal vectors are introduced as matching constraints to improve registration accuracy. For complex... For the electrical components of the structure, feature-based coarse registration is used as the initial alignment for ICP to avoid local optima. After registration, a triangular mesh model is generated using the Poisson surface reconstruction algorithm, with a reconstruction depth of 9 to balance detail preservation and computational efficiency. This process ensures uniform point cloud density and that the mesh vertex spacing is controlled within 0.01 meters, forming a continuous and closed component surface model. The registered high-resolution texture image is then accurately mapped onto the reconstructed mesh surface using UV mapping technology. Texture coordinates are calculated based on camera parameters and mesh vertex coordinates, and pixel jaggedness is eliminated through bilinear interpolation. For overlapping areas of multi-view images, a weighted average fusion algorithm is used, with weights dynamically adjusted according to the viewing angle and image sharpness to eliminate inconsistencies in seams and lighting. The final refined 3D model has the same texture resolution as the original image (1920×1080 corresponding to the mapped area), and millimeter-level structural features (such as gaps between insulator discs and scratches on the conductor surface) can be distinguished. The model is output in a standard 3D file format.

[0054] The risk mapping module, based on real-time point cloud and environmental data, constructs a dynamic inspection risk map, assesses the safety level of the path, identifies potential obstacles and risk areas, and generates a visualized risk heat map. This provides a visualized and quantitative risk assessment basis for flight decisions, enabling the visualization and quantitative assessment of inspection risks and providing dynamic decision support for the safe flight of UAVs.

[0055] The adaptive flight control module, combining dynamic inspection risk map and fruit fly algorithm, generates safe and efficient optimal inspection trajectory. Through the combination of real-time control on the end side and global scheduling on the cloud side, it performs dynamic replanning of the UAV flight path and emergency obstacle avoidance, ensuring that the UAV can safely complete the inspection task in the event of an emergency. It achieves seamless connection between global path planning and local dynamic obstacle avoidance, ensuring the autonomous flight safety of the UAV in complex environments.

[0056] The component identification module is used to perform feature learning and classification on the collected 3D point cloud and texture information based on deep learning and pattern recognition algorithms under the optimal inspection trajectory. It identifies power components, including towers, conductors and insulators, and detects whether there are structural anomalies such as damage, deformation and detachment. It realizes stable identification and status assessment of power components in complex environments, improves the automation and accuracy of inspection, realizes high-precision automatic identification and health status assessment of power components, and improves inspection efficiency and fault detection rate.

[0057] Example 2, as Figure 1 , Figure 2 As shown, based on Embodiment 1, the present invention provides a technical solution: the risk mapping module includes a dynamic environment modeling unit and a risk level assessment unit;

[0058] The dynamic environment modeling unit is used to fuse 3D point cloud data from lidar and meteorological sensor data in real time to construct a dynamic inspection risk map that includes factors such as obstacles, wind speed, and visibility. This supports real-time risk assessment, real-time fusion of multi-source data, and dynamic updates of the risk map, improving the responsiveness to environmental changes. It receives 3D point cloud data collected by lidar and simultaneously acquires meteorological data monitored by meteorological sensors, including wind speed, visibility, temperature, and humidity. It performs spatiotemporal alignment and data fusion to achieve a unified benchmark for multi-source information, laying the foundation for environmental fusion perception. Based on point cloud clustering and obstacle recognition algorithms, it extracts static and dynamic obstacle information in the inspection environment and constructs a multi-dimensional dynamic inspection risk map that includes obstacle distribution and environmental status, clearly identifying risk elements. It updates the dynamic inspection risk map according to real-time data and marks risk areas and safe passages, providing real-time environmental modeling support for risk level assessment and path planning, and updating the risk situation in real time to provide dynamic scenario support for safe flight decisions.

[0059] The specific tasks are as follows: In actual operation, at a fixed frequency of no less than 10Hz, synchronously receive 3D point cloud data generated by a 32-line lidar mounted on the UAV, as well as real-time meteorological data collected by the airborne meteorological sensor module. The lidar provides spatial point cloud data with an accuracy better than ±3cm within its effective measurement range (set to 100 meters). The data format includes the 3D coordinates and reflection intensity value of each point. The synchronously acquired meteorological data includes at least instantaneous wind speed (measurement range 0-30m / s, accuracy ±0.5m / s), horizontal visibility (measurement range 5-2000 meters), and ambient temperature. Temperature and relative humidity are synchronized using a hardware time synchronization mechanism to ensure that the timestamp deviation between point cloud data frames and meteorological data packets is less than 100 milliseconds. Furthermore, the spatiotemporal reference (latitude, longitude, altitude, attitude angle) provided by the UAV's high-precision integrated navigation system is used to spatially align and temporally synchronize all data, forming a fused data stream under a unified spatiotemporal reference system. Based on the fused spatiotemporally aligned data, the lidar point cloud is processed using a point cloud clustering algorithm based on Euclidean distance, with a neighborhood search radius of 0.2 meters and a minimum cluster size of 10 points, to distinguish static backgrounds such as ground, vegetation, buildings, and towers. For each identified obstacle cluster, including birds, other drones, and temporary construction machinery, the system calculates its 3D bounding box, centroid position, point cloud density, and motion state (by analyzing displacement vectors from consecutive frame point clouds). Simultaneously, real-time meteorological data is quantified into influencing factors. Finally, multi-dimensional information (spatial location, type, dynamic attributes of obstacles, and environmental conditions such as wind speed and visibility) is integrated into a digital raster map with the inspection area as its spatial dimension. This constructs a multi-dimensional dynamic inspection risk map containing geometric, semantic, and physical environmental information. The raster resolution of this map is set according to the inspection accuracy requirements. The distance is set at 0.5 meters. The dynamic inspection risk map is updated in real time at the same frequency as the data acquisition. Each update is based on the latest point cloud frame and meteorological data to recalculate the risk attributes of each grid. For the identified obstacle area, whether static or dynamic, it is marked as a no-fly zone or high-risk area in the map, and its risk impact range is estimated. At the same time, considering factors such as areas with reduced perception due to low visibility and areas with difficult operation due to excessive wind speed, environmental risk areas are marked on the map. Conversely, for continuous spaces without obstacles and with good environmental conditions, they are marked as recommended safe passages.

[0060] The risk level assessment unit, based on dynamic inspection risk maps and historical inspection data, calculates the safety level of each segment of the path using a risk quantification algorithm and generates a risk heatmap to guide flight path optimization. It provides trend analysis and risk warnings based on historical data, enabling intelligent optimization of path planning. Based on the dynamic inspection risk map output by the dynamic environment modeling unit, it extracts obstacle density, wind speed, and visibility indicators for each path segment as risk quantification inputs, effectively quantifying core risk factors in the inspection environment. This provides a structured and computable data foundation for subsequent safety assessments. A multi-factor weighted risk assessment algorithm is used to calculate the safety score of each path segment and classify risk levels based on the score results, achieving objective and precise risk classification of the inspection path. This enhances the scientific nature and operability of flight decisions. A visual risk heatmap is generated based on the risk level, and risk trend analysis is performed using historical inspection data. This provides a quantitative decision-making basis for path optimization by the adaptive flight control module, intuitively displaying risk distribution and changing trends, and providing real-time and reliable dynamic path optimization basis for the flight control system.

[0061] The specific work involves: based on a multi-dimensional dynamic inspection risk map, using a pre-set spatial grid as the basic assessment unit, extracting core quantitative parameters from the map as input for risk assessment. Specifically, from each grid and the aggregated area of ​​adjacent grids, extracting the obstacle density (calculated through point cloud statistics after clustering), instantaneous wind speed level (divided into 5 levels based on a range of 0-30 m / s, with each level spaced 6 m / s apart, and the percentage position of the current wind speed value within the range used as a weighting factor), and horizontal visibility index (normalized based on a measurement range of 5-2000 meters, mapped to a visibility coefficient between 0 and 1). Each core quantitative parameter is refreshed in real time at a frequency of no less than 10 Hz to ensure that the input data is synchronized with the actual state of the current inspection environment, forming... A continuous and computable risk attribute vector is used. A multi-factor weighted risk assessment model based on the entropy weight method and the analytic hierarchy process is adopted to quantify the safety status of each path segment. Judgment matrices for three factors, namely obstacle density, wind speed, and visibility, are constructed. The basic weights of each factor are set according to the power inspection safety regulations, and the weights are dynamically fine-tuned by calculating the entropy value of each parameter through historical data to obtain a comprehensive weight vector. Then, the parameters extracted in real time are normalized and weighted with the weight vector to calculate the safety score S of the path segment. The score range is defined between 0 and 100. According to the score results, the risk level is divided into four levels: safe (S≥80), low risk (60≤S<80), medium risk (40≤S<60), and high risk (S<40).

[0062] The formula for calculating the safety score is as follows:

[0063] ;

[0064] ;

[0065] ;

[0066] ;

[0067] In the formula: A safety score is assigned, with higher scores indicating greater safety. For the first The overall weight of each risk factor; For the first Normalized risk values ​​for each risk factor; The preset maximum obstacle density threshold; For obstacle density, , This represents the number of obstacle clusters identified by a point cloud clustering algorithm within the current grid and its adjacent grid aggregation areas. The area of ​​the aggregation region; Wind speed weighting factor , This is the current instantaneous measured wind speed. This is the upper limit of the wind speed measurement range; This is the visibility coefficient; a higher value indicates better visibility. , This is the current measured horizontal visibility value. This is the lower limit of the visibility measurement range. This represents the upper limit of the visibility measurement range.

[0068] Weight Determination: Based on the analytic hierarchy process (AHP), a judgment matrix is ​​established to obtain the initial weights. Calculate the entropy value of each factor based on historical data. Entropy weight is Overall weight ;

[0069] Based on the real-time calculated risk level, a risk heat map is dynamically generated on the visual control interface and overlaid on the 3D environment map. The risk heat map uses gradient color rendering, with green, yellow, orange and red corresponding to the four risk levels. The transparency of the color blocks is positively correlated with the risk value. At the same time, historical risk assessment records of the same area and similar environmental conditions in the historical inspection database are called up, and risk trend is judged through time series analysis. If the risk level of the current path segment continues to rise or high-risk markers appear repeatedly, a risk warning and quantitative analysis results are sent to the adaptive flight control module, including the risk location coordinates, level, dominant factors and changing trends, as the basis for its decision to replan the path in real time or adjust the flight strategy.

[0070] The adaptive flight control module performs the following steps: receiving the generated risk heat map and inspection task constraints, performing global path search based on the fruit fly algorithm, generating the optimal inspection trajectory that balances safety and efficiency, effectively avoiding known risk areas, planning the globally optimal flight route, significantly improving the safety foundation of the inspection task, monitoring environmental changes and sudden obstacles in real time during the UAV's flight, combining edge-side perception data and the dynamic inspection risk map in the cloud, performing dynamic replanning of local paths, responding to environmental changes and dynamic threats in real time, autonomously adjusting the flight path, ensuring the continuous safety and reliability of the inspection process, executing dynamic adjustment commands through the flight controller, performing real-time obstacle avoidance and heading correction for the UAV, ensuring that the inspection task can be completed safely and stably in complex environments, achieving precise closed-loop control of flight attitude and heading, and ensuring that the UAV can stably perform inspection tasks in complex environments;

[0071] The specific work content is as follows: After receiving the task instructions issued by the inspection management center, the adaptive flight control module loads the risk heat map and task constraint parameters generated by the risk level assessment unit. The task constraint parameters include at least the sequence of inspection tower target points, the upper limit of the total task time, the UAV's endurance limit, and the safe flight altitude range. Based on this, a global path search module based on the fruit fly algorithm is adopted, using three-dimensional geographic space as the search domain. The algorithm's initial population size is set to 30 fruit flies, and the maximum number of iterations is 500. The algorithm's objective function is defined as a multi-objective optimization problem, comprehensively considering the total path length, the cumulative safety score of traversing each risk grid, and the smoothness of flight altitude changes. The weights are configured to 0.4, 0.4, and 0.2 based on task priority. During the search process, the optimal solution is iteratively sought in the solution space by simulating the olfactory and visual foraging behavior of a fruit fly swarm. Each iteration updates and filters the swarm position based on the fitness value. Finally, a globally optimal inspection trajectory is output, from the takeoff point to all target towers and back to the takeoff point (or a specified landing point). This trajectory consists of a series of ordered spatial waypoints, each containing precise latitude, longitude, altitude, and preset flight speed. The entire trajectory has initially avoided the medium and high risk areas marked on the risk heat map, forming a baseline route that balances safety and flight efficiency. The UAV follows the global trajectory. During autonomous flight, the adaptive flight control module operates continuously at a frequency of no less than 10Hz, simultaneously receiving two types of real-time data: first, local high-precision point cloud and image data provided by the UAV's edge-side perception system (laser vision fusion perception unit), used to detect sudden obstacles ahead; second, incremental information of the dynamic inspection risk map, updated at the same frequency by the dynamic environment modeling unit and transmitted by the inspection management center via data link. This information reflects the evolution of environmental risks such as sudden changes in wind speed and a sharp drop in visibility. When real-time monitoring data indicates that a new high-risk obstacle has appeared within a predetermined distance ahead of the current flight path (the warning threshold is set at 15 meters), or when the risk map shows that the original low-risk area has been upgraded to medium-risk, the system will respond accordingly. When the above risks are detected, local path dynamic replanning is immediately triggered. Starting from the current position of the UAV and taking the next safe waypoint of the current flight segment as the temporary endpoint, a local path search is performed within a limited time and space window (planning time budget less than 200 milliseconds) using a fast random exploration tree algorithm combined with kinematic constraints. The newly planned local path segment must meet the dynamic constraints of the UAV, such as the minimum turning radius (not less than 2 meters) and the maximum climb / descent angle (not more than 30 degrees). The UAV is guided to detour to the recommended safe passage area marked on the risk heat map, thereby generating a smooth, safe and executable local avoidance trajectory to replace the threatened segment in the original global trajectory.After local replanning is completed, the new path segments are transformed into a series of low-level control commands that can be executed by the UAV flight controller. These low-level control commands include the target waypoint sequence, desired airspeed (adaptively adjusted according to wind speed, within the range of 3-8 m / s), heading angle, and rate of climb. The control module adopts a hierarchical control strategy: the upper layer calculates attitude and position tracking commands based on a proportional-integral-derivative control algorithm, and its control parameters have been pre-calibrated and optimized for the inspection UAV's dynamic model; the lower layer is directly executed by the flight controller, achieving precise attitude control by adjusting motor speed and control surface deflection angle throughout the obstacle avoidance or path tracking process. It sends commands to the flight controller at a frequency of 100Hz via a high-speed data bus and receives status feedback from the inertial measurement unit and the global navigation satellite system receiver in real time, forming a closed-loop control. At the same time, based on the embedded flight envelope protection logic, it continuously monitors the UAV's attitude angles (the absolute values ​​of roll angle and pitch angle are both limited to within 45 degrees), airspeed, and remaining battery power to ensure that all maneuvers are performed within the safety boundaries. Through real-time command generation, issuance, and closed-loop correction, it achieves precise control of the UAV, enabling it to stably and safely complete the inspection mission along the desired trajectory even when facing complex environments and sudden threats.

[0072] The component identification module performs the following steps: Based on the fused perception dataset, it extracts the three-dimensional geometric and texture features of power components, constructs multi-source feature vectors as identification inputs, fuses geometric and texture features to effectively resist changes in light and background interference, enhances feature robustness, and uses a deep learning network to train and classify the feature vectors to achieve automatic identification and positioning of power components including towers, insulators, and conductors. It achieves high-precision automatic identification of component categories and accurate three-dimensional spatial positioning, replacing manual identification. It performs structural integrity analysis on the identified power components, detects whether there are structural abnormalities such as damage, deformation, or detachment, and outputs a component health assessment report. It automatically detects structural defects and generates quantitative reports to support precise operation and maintenance and preventive maintenance.

[0073] The specific work involves: receiving the fused perception dataset from the laser vision fusion unit, preprocessing the data containing high-precision 3D coordinates and RGB texture information; for 3D point cloud data, using voxelization downsampling to unify the point cloud density to no less than 1000 points per cubic meter, and calculating the normal vector, curvature, and other local geometric features of each point; for texture image data, based on the completed spatial registration, extracting the HOG texture features and statistical histogram features in the HSV color space of the surface areas of each power component, and then, through a pre-established projection mapping relationship, mapping the geometric features of each 3D point... The features are aligned and fused with the texture features of the corresponding two-dimensional image pixel blocks to construct a unified 512-dimensional multi-source feature vector. A deep convolutional neural network model containing a feature fusion layer and a multi-task output head is constructed. This model takes the 512-dimensional multi-source feature vector as input, and the backbone network adopts the ResNet-50 architecture for deep feature extraction. The network has been pre-trained and fine-tuned on a power inspection dataset containing more than 100,000 labeled samples (covering towers, insulators, conductors, and other components under different lighting, weather, and viewing angles). The final output of the model contains two branches: a classification branch outputs the category probability of the component (using...). The Softmax function is used for categories including towers, insulator strings, conductors, equipotential rings, and vibration dampers. The positioning regression branch outputs the minimum bounding box parameters (center point coordinates, length, width, height, and attitude angles) of the components in 3D space. The model's recognition accuracy on the test set reaches an average of 99.2%, and the positioning error does not exceed 0.05 meters at real-world scale, meeting the requirements for automated, high-precision component identification and positioning during inspections. After component identification and positioning, a fine-grained structural integrity analysis is performed on each power component instance, comparing the current power component's 3D point cloud model with the corresponding benchmark model in the standard component template library. For non-rigid registration, the average registration error between points on the surface is calculated, with the error threshold set at 3% of the component feature size. Simultaneously, the density distribution of the power component point cloud and the consistency with the normal vector are analyzed to detect surface damage or local missing parts. For insulator strings, the standard deviation of the spacing between adjacent skirts is calculated. If it exceeds the standard value by ±15%, it is judged as structural deformation. Finally, based on the quantitative analysis results of various indicators, a structured health assessment report is automatically generated. The report clearly lists the ID, type, spatial location, detected anomaly type (damage, deformation, detachment) and confidence level of each identified power component, and identifies the power components that are recommended for priority repair.

[0074] Example 3, as Figure 1 , Figure 2As shown in Embodiments 1-2, this invention also provides a UAV power inspection system based on LiDAR, including at least one control processor and a memory for communicating with the at least one control processor. The system architecture is clear, the hardware and software work together efficiently, and it is easy to achieve centralized control and modular upgrades. The memory stores instructions that can be executed by at least one control processor, ensuring that the control logic is stable and solidified, preventing instruction loss or errors during operation. The instructions are executed by at least one control processor, enabling the UAV power inspection device based on LiDAR that can be executed by at least one control processor to achieve fully automated inspection, significantly improving operational safety and detection reliability.

[0075] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0076] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A drone-based power line inspection device based on lidar, comprising an inspection management center, characterized in that, The inspection management center has the following communication modules: The fusion modeling module is used to fuse LiDAR and visual sensors to acquire high-precision 3D point cloud and texture information of power components in real time, and to build a refined 3D model of the power components. The risk mapping module, based on real-time point cloud and environmental data, constructs a dynamic inspection risk map, assesses the safety level of the path, identifies potential obstacles and risk areas, and generates a visualized risk heat map. The adaptive flight control module, combining dynamic inspection risk map and fruit fly algorithm, generates safe and optimal inspection trajectory, and performs dynamic replanning of UAV flight path and emergency obstacle avoidance through a combination of real-time control on the end side and global scheduling on the cloud side. The component identification module is used to identify power components by performing feature learning and classification on the collected 3D point cloud and texture information based on deep learning and pattern recognition algorithms under the optimal inspection trajectory, and to detect whether there are structural anomalies such as damage, deformation, or detachment.

2. The UAV power line inspection device based on lidar according to claim 1, characterized in that: The fusion modeling module includes a laser vision fusion perception unit and a three-dimensional structure reconstruction unit; The laser vision fusion perception unit is used to synchronously collect multi-source inspection data of power components using lidar and vision sensors, covering high-precision three-dimensional point cloud and texture information, and combining calibration and registration algorithms to perform spatial alignment and information complementarity of multi-source inspection data. The three-dimensional structure reconstruction unit, based on the fused three-dimensional point cloud and texture data, and using point cloud registration and surface reconstruction algorithms, reconstructs a refined three-dimensional model of the power components.

3. The UAV power line inspection device based on lidar according to claim 2, characterized in that: The laser vision fusion sensing unit performs the following steps: The laser radar and vision sensor on the target UAV are triggered simultaneously to collect multi-source inspection data of the power components, including high-precision 3D point cloud and texture image, and perform timestamp alignment and preliminary spatial registration. Based on pre-calibrated intrinsic and extrinsic parameters, a multi-sensor fusion calibration algorithm is used to perform pixel-level spatial alignment between 3D point cloud data and texture image data, thereby complementing the geometric and texture information of multi-source inspection data. By using feature matching and optimization algorithms, fine-grained registration is performed on the fused multi-source inspection data to generate a fused sensing dataset with spatial consistency.

4. The UAV power line inspection device based on lidar according to claim 3, characterized in that: The three-dimensional structure reconstruction unit performs the following steps: Receive relevant data from the fusion sensing dataset, extract the 3D point cloud and texture features of the power components, and perform noise filtering and outlier removal; The iterative nearest point algorithm is used to register the multi-view point cloud to construct a complete point cloud model of the power component, and a meshed surface model is generated based on the triangulation method. The registered texture image is mapped onto the reconstructed mesh surface to generate a refined 3D model of the power component with realism and structural detail.

5. The UAV power line inspection device based on lidar according to claim 2, characterized in that: The risk mapping module includes a dynamic environment modeling unit and a risk level assessment unit; The dynamic environment modeling unit is used to fuse the 3D point cloud of lidar and meteorological sensor data in real time to construct a dynamic inspection risk map. The risk level assessment unit calculates the safety level of each segment of the path based on the dynamic inspection risk map and historical inspection data, and generates a risk heat map to guide flight path optimization.

6. The UAV power line inspection device based on lidar according to claim 5, characterized in that: The dynamic environment modeling unit performs the following steps: It receives 3D point cloud data collected by lidar and simultaneously acquires meteorological data, including wind speed, visibility, temperature and humidity, monitored by meteorological sensors, and performs spatiotemporal alignment and data fusion. Based on point cloud clustering and obstacle recognition algorithms, static and dynamic obstacle information in the inspection environment is extracted, and combined with meteorological data, a multi-dimensional dynamic inspection risk map containing obstacle distribution and environmental status is constructed. The dynamic inspection risk map is updated based on real-time data, and risk areas and safe passages are marked.

7. The UAV power line inspection device based on lidar according to claim 5, characterized in that: The risk level assessment unit performs the following steps: Based on the dynamic inspection risk map output by the dynamic environment modeling unit, the obstacle density, wind speed level, and visibility index of each path segment are extracted as risk quantification inputs. A multi-factor weighted risk assessment algorithm is used to calculate the safety score of each path segment and classify the risk level based on the score results; A visual risk heatmap is generated based on the risk level, and risk trend analysis is conducted by combining it with historical inspection data.

8. The UAV power line inspection device based on lidar according to claim 5, characterized in that: The adaptive flight control module performs the following steps: Receive the generated risk heat map and inspection task constraints, perform global path search based on the fruit fly algorithm, and generate the optimal inspection trajectory that balances safety and efficiency. During the flight of the drone, environmental changes and sudden obstacles are monitored in real time. By combining edge-side perception data with the dynamic inspection risk map in the cloud, local paths are dynamically replanned. The flight controller executes dynamic adjustment commands to perform real-time obstacle avoidance and heading correction for the drone.

9. A UAV power line inspection device based on lidar according to claim 8, characterized in that: The component identification module performs the following steps: Based on the fusion sensing dataset, the three-dimensional geometric and texture features of power components are extracted, and multi-source feature vectors are constructed as recognition inputs. Deep learning networks are used to train and classify feature vectors, enabling automatic identification and location of power components, including towers, insulators, and conductors. Structural integrity analysis is performed on the identified power components to detect any structural abnormalities such as damage, deformation, or detachment, and a component health assessment report is generated.

10. A drone-based power line inspection system based on lidar, characterized in that: It includes at least one control processor and a memory for communicatively connecting to the at least one control processor; The memory stores instructions that can be executed by the at least one control processor, which, when executed, enables the at least one control processor to perform the lidar-based unmanned aerial vehicle power inspection device as described in any one of claims 1 to 9.

Images