Network power equipment inspection method, device, equipment and medium

Abstract

The application relates to a distribution network power equipment inspection method and device, equipment and medium, and relates to the technical field of power equipment inspection. The method comprises the following steps: synchronously collecting point cloud data and visible light image data and preprocessing; based on a distribution network digital twin model, a multi-dimensional dynamic clearance threshold system is constructed by combining environmental parameters, obstacle semantic types and voltage levels, clearance safety distance calculation and obstacle risk classification elimination are realized; the three-dimensional model of the distribution network equipment is reversely projected to the image to determine the bounding box, the key components are identified through the multi-modal feature fusion target detection model of YOLOv8; the three-dimensional geometric features and two-dimensional texture features of the key components are extracted, the geometric-texture double guide factors are constructed, and the defect feature precise enhancement is realized by adaptively adjusting the enhancement algorithm parameters. The application can realize the synergy of clearance inspection and defect detection, improve the fusion accuracy of multi-dimensional perception data, and use geometric information to guide image enhancement to improve the recognition accuracy in complex environments.

Description

Technical Field

[0001] This application relates to the field of power equipment inspection technology, specifically to a method, device, equipment, and medium for inspecting power distribution network equipment. Background Technology

[0002] With the deepening of the construction of new power systems and the intelligent upgrading of distribution networks, the coverage of distribution network lines is gradually expanding, the scenarios are becoming more complex, and the number of equipment has increased significantly. This necessitates greater manpower and resources for the inspection of equipment such as poles, cables, and insulators. Manual inspection is constrained by terrain limitations (such as steep mountain roads and river obstructions), efficiency bottlenecks, and safety risks such as close contact with high-voltage equipment. Power drones, with their powerful functional advantages and ability to effectively avoid the safety risks of climbing poles and getting close to high-voltage equipment, play a crucial role in distribution network inspection, significantly improving manual efficiency and becoming a core means of monitoring distribution network equipment.

[0003] The core objective of distribution network inspection is to simultaneously ensure "airspace safety" and "equipment health," both of which have a significant impact on the stability of power grid operation. Insufficient airspace can easily lead to line discharge and short-circuit tripping. From the perspective of equipment health, if defects such as insulator damage, cable sheath cracking, and tower corrosion are not detected or addressed in a timely manner, they may lead to insulation breakdown, wire breakage, or even power outages. Therefore, improvements in airspace monitoring and defect detection are crucial for reducing the failure rate and ensuring power supply reliability.

[0004] However, existing drone inspection technologies still have some problems and need to be improved in terms of automation and precision. First, airspace safety assessment (measuring the safe distance between power lines and trees, houses) and equipment defect detection (identifying insulator damage, cable cracks, etc.) are two separate processes, which not only increases costs but also reduces manual efficiency. Second, the accuracy of single sensor data is insufficient, and airspace analysis is unreliable. LiDAR alone cannot distinguish between "power equipment" and "trees, houses, etc.", requiring manual target screening and prone to missed detections. Visible light images alone cannot accurately calculate the three-dimensional distance between power lines and obstacles, and airspace assessment lacks quantitative basis, failing to meet the requirements of safety inspection. Finally, in complex environments, anti-interference is weak, data quality is poor, and real-world conditions such as changes in outdoor lighting, weather interference, and background obfuscation increase the rate of missed defect detection and reduce the accuracy of airspace calculation. In addition, existing image enhancement technologies (such as traditional Retinex) are mostly global or based on two-dimensional features, unable to use three-dimensional geometric information to distinguish between real defects and background textures. In complex scenes, this can easily lead to excessive background enhancement, which in turn interferes with defect identification.

[0005] In summary, how to achieve synergy between airspace inspection and defect detection, improve the fusion accuracy of multi-dimensional sensing data, and utilize geometric information to guide image enhancement to improve recognition accuracy in complex environments are the technical problems that urgently need to be solved for the current power distribution network drone inspection technology to upgrade towards automation and precision. Summary of the Invention

[0006] In view of this, this application provides a method, device, equipment and medium for inspecting power distribution network equipment. The main purpose is to solve the technical problems of how to achieve the synergy between clearance inspection and defect detection, improve the fusion accuracy of multi-dimensional sensing data, and use geometric information to guide image enhancement to improve the recognition accuracy in complex environments.

[0007] Firstly, this application provides a method for inspecting power distribution network equipment, including:

[0008] Simultaneously collect point cloud data and visible light image data from lidar in the power distribution network area, and preprocess the point cloud data and visible light image data;

[0009] A digital twin model of the distribution network is constructed based on the preprocessed point cloud data and the visible light image data. The digital twin model of the distribution network includes a digital surface model, a digital elevation model, a normalized digital surface model, and a semantic model of the distribution network equipment.

[0010] Based on the digital twin model of the power distribution network, an airspace analysis is performed on the power distribution network area to identify power conductors and obstacles in the power distribution network area. A multi-dimensional dynamic airspace threshold system is constructed by combining environmental perception data, obstacle semantic features and distribution network voltage level to calculate the safe airspace distance between the power conductors and the obstacles.

[0011] Risk classification is performed based on the clearance safety distance and the semantic features of the obstacles. Non-electric obstacles in the power distribution network area are eliminated, power poles in the power distribution network area are extracted, and the topological connection relationship between the power conductors and the power poles is established through spatial topological relationship analysis. A three-dimensional model of the power distribution network equipment containing the spatial location information and topological relationship information of the power equipment is constructed.

[0012] The spatial location information of the power equipment in the three-dimensional model of the power distribution network is reverse-projected onto the visible light image data to determine the image bounding box of the key components. The key component image within the image bounding box is identified by a pre-trained deep learning target detection model that fuses multimodal features. The deep learning target detection model that fuses multimodal features is a YOLOv8 detection model that fuses three-dimensional geometric features and two-dimensional texture features.

[0013] The three-dimensional geometric features and two-dimensional texture features corresponding to the key component image are extracted, and a geometric-texture dual guiding factor is constructed. The geometric-texture dual guiding factor is used to perform defect feature adaptive enhancement processing on the key component image to obtain a defect-enhanced key component image.

[0014] Secondly, this application provides a power distribution network equipment inspection device, comprising:

[0015] The acquisition module is used to simultaneously acquire point cloud data and visible light image data of lidar in the power distribution network area, and to preprocess the point cloud data and the visible light image data.

[0016] The first construction module is used to construct a digital twin model of the distribution network based on the preprocessed point cloud data and the visible light image data. The digital twin model of the distribution network includes a digital surface model, a digital elevation model, a normalized digital surface model, and a semantic model of the distribution network equipment.

[0017] The calculation module is used to perform clearance analysis on the power distribution network area based on the power distribution network digital twin model to identify power conductors and obstacles in the power distribution network area. It combines environmental perception data, obstacle semantic features and distribution network voltage level to construct a multi-dimensional dynamic clearance threshold system to calculate the clearance safety distance between the power conductors and the obstacles.

[0018] The second construction module is used to classify risks based on the clearance safety distance and the semantic features of the obstacles, remove non-power obstacles in the power distribution network area, extract the power poles in the power distribution network area and establish the topological connection relationship between the power conductors and the power poles through spatial topological relationship analysis, and construct a three-dimensional model of the power distribution network equipment containing the spatial location information and topological relationship information of the power equipment.

[0019] The identification module is used to reverse project the spatial location information of the power equipment in the three-dimensional model of the power distribution network equipment onto the visible light image data, determine the image bounding box of the key components, and use a pre-trained deep learning target detection model with multimodal feature fusion to identify the key component images within the image bounding box. The deep learning target detection model with multimodal feature fusion is a YOLOv8 detection model that fuses three-dimensional geometric features and two-dimensional texture features.

[0020] An enhancement module is used to extract the three-dimensional geometric features and two-dimensional texture features corresponding to the key component image, construct a geometric-texture dual guiding factor, and use the geometric-texture dual guiding factor to perform defect feature adaptive enhancement processing on the key component image to obtain a defect-enhanced key component image.

[0021] Thirdly, this application provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the power distribution equipment inspection method described in the first aspect.

[0022] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the power distribution equipment inspection method described in the first aspect.

[0023] By employing the above technical solutions, this application provides a method, device, equipment, and medium for inspecting power distribution network equipment. Compared with existing technologies, this application can simultaneously collect point cloud data and visible light image data from lidar in the power distribution network area, and preprocess the point cloud data and visible light image data. Based on the preprocessed point cloud data and visible light image data, a digital twin model of the power distribution network is constructed. The digital twin model includes a digital surface model, a digital elevation model, a normalized digital surface model, and a semantic model of the power distribution network equipment. Based on the digital twin model, a clearance analysis is performed on the power distribution network area to identify power conductors and obstacles in the power distribution network area. Combining environmental perception data, obstacle semantic features, and power distribution network voltage levels, a multi-dimensional dynamic clearance threshold system is constructed to calculate the clearance threshold between power conductors and obstacles. The clearance safety distance between obstacles is determined; risk classification is performed based on the clearance safety distance and the semantic features of obstacles, non-electric obstacles in the power distribution network area are removed, power poles in the power distribution network area are extracted, and the topological connection relationship between power conductors and power poles is established through spatial topological relationship analysis. A three-dimensional model of power distribution network equipment containing spatial location information and topological relationship information of power equipment is constructed; the spatial location information of power equipment in the three-dimensional model of power distribution network equipment is inversely projected onto visible light image data to determine the image bounding boxes of key components. A pre-trained deep learning object detection model with multimodal feature fusion is used to identify the key component images within the image bounding boxes. The deep learning object detection model with multimodal feature fusion is a YOLOv8 detection model that integrates three-dimensional geometric features and two-dimensional texture features; the three-dimensional geometric features and two-dimensional texture features corresponding to the key component images are extracted, and a geometric-texture dual guiding factor is constructed. The geometric-texture dual guiding factor is used to perform defect feature adaptive enhancement processing on the key component images to obtain the defect-enhanced key component images.

[0024] Using the above technical solution, this application constructs a digital twin model of the distribution network (including a digital surface model, a digital elevation model, a normalized digital surface model, and a semantic model of distribution network equipment) based on synchronously acquired lidar point cloud and visible light image data. This model is used for unified clearance analysis (identifying conductors and obstacles, calculating safe clearance distances) and risk classification. Subsequently, based on the analysis results, power poles are extracted and topological relationships are established to construct a three-dimensional model of the distribution network power equipment. Finally, the spatial location information in the three-dimensional model is inversely projected onto the visible light image to determine the image bounding boxes of key components, enabling direct defect detection. This achieves seamless data flow from clearance analysis to defect detection, integrating two originally independent processes into a unified framework based on digital twins, eliminating the need for manual switching or repetitive data processing.

[0025] This application constructs a digital twin model incorporating the semantic model of power distribution network equipment. By combining environmental perception data and obstacle semantic features, it can accurately distinguish between "power equipment" and non-power obstacles such as "trees and houses," and eliminate the latter. A 3D model is constructed using point cloud data, solving the problem of inaccurate 3D distance calculation using only visible light; semantic features are used to address the difficulty of classification using only LiDAR. In the defect detection stage, a YOLOv8 detection model fusing 3D geometric features and 2D texture features is used, improving the accuracy of target recognition.

[0026] This application extracts the 3D geometric features and 2D texture features corresponding to the images of key components, constructing a "geometry-texture dual-guided factor." This dual-guided factor is then used to adaptively enhance the defect features of the key component images. Unlike traditional global or 2D feature-based enhancements (such as Retinex), this method utilizes 3D geometric information to distinguish between real defects and background texture. In complex scenes (lighting variations, background obfuscation), it avoids excessive background enhancement, focusing on enhancing the defect area, thereby reducing the false negative rate and improving recognition accuracy.

[0027] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0028] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0029] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A flowchart illustrating a method for inspecting power distribution network equipment provided in this application embodiment;

[0031] Figure 2 A flowchart illustrating another method for inspecting power distribution network equipment provided in this application embodiment;

[0032] Figure 3 A schematic diagram of a geometry-texture dual guidance factor enhancement algorithm provided in this application embodiment;

[0033] Figure 4 This is a schematic diagram of the structure of a power distribution network equipment inspection device provided in an embodiment of this application. Detailed Implementation

[0034] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0035] The following description, with reference to the accompanying drawings, describes the method, apparatus, equipment, and medium for inspecting power distribution network equipment according to embodiments of this application.

[0036] This application provides a method, device, equipment and medium for inspecting power distribution network equipment. The main purpose is to solve the technical problems of how to achieve the synergy between clearance inspection and defect detection, improve the fusion accuracy of multi-dimensional sensing data, and use geometric information to guide image enhancement to improve the recognition accuracy in complex environments.

[0037] like Figure 1 As shown, an embodiment of this application provides a method for inspecting power distribution network equipment, including:

[0038] Step 101: Synchronously collect point cloud data and visible light image data of the lidar in the power distribution network area, and preprocess the point cloud data and visible light image data.

[0039] The principle of this application is to deeply fuse the three-dimensional geometric point cloud data acquired by LiDAR with the two-dimensional texture image data acquired by a visible light camera to construct a unified multimodal data base. On this basis, three core steps are executed in sequence: clearance analysis and equipment extraction based on nDSM, accurate component identification, and image enhancement for defect detection, forming an automated and intelligent closed-loop processing flow of data acquisition, data fusion, and data processing.

[0040] Specifically, such as Figure 2 As shown, a heavy-duty, highly stable industrial-grade drone can be used as a flight platform. A rigid bracket can be used to simultaneously mount a lidar module and a visible light camera, minimizing the spatial distance between the centers of the two sensors. Hardware synchronization signals or high-precision timestamps are used to ensure that the lidar scanning point cloud and the camera exposure imaging time are strictly aligned, thereby synchronously acquiring lidar point cloud data and visible light image data of the power distribution network area.

[0041] For embodiments of this disclosure, preprocessing of point cloud data may specifically include:

[0042] First, to filter out noise points caused by dust, raindrops, etc. in the air, the Statistical Outlier Removal (SOR) algorithm can be used to denoise the point cloud data. Specifically, for each point in the point cloud data, the average distance to all K nearest neighbors (e.g., setting K=50) is calculated. Assuming that the distance distribution in the entire point cloud follows a Gaussian distribution, all points whose average distance exceeds the preset Gaussian distribution range are removed. (For example, n=3) noisy points, thus obtaining denoised point cloud data, where, This represents the mean of the average distances between all points in the point cloud data. The standard deviation factor is the coefficient of variation. is the standard deviation of the distance between all points in the point cloud data and their nearest neighbors.

[0043] Secondly, to balance accuracy and computational efficiency, a voxel grid downsampling algorithm can be used to filter and downsample the denoised point cloud data. Specifically, the three-dimensional space can be divided into a uniform cubic grid (voxels). The centroid coordinates of all points within each voxel are used to approximate all points within that voxel, resulting in the processed point cloud data. Given the voxel size v, for each voxel V_i, the coordinates of its representative point P_representative are:

[0044]

[0045] In the formula, P_representative is the coordinate of the representative point, P_j is the j-th point in voxel V_i, and m is the total number of points in the voxel.

[0046] After completing the above denoising, filtering, and downsampling processes, the processed point cloud data also needs to undergo inter-frame registration and fusion. Since point cloud data in power distribution network areas is usually collected in frames by UAVs or multi-view sensors, there is a problem of inconsistent spatial locations. Therefore, it is necessary to use inter-frame registration algorithms (such as the Iterative Closest Point (ICP) algorithm or its variants) to align multiple frames of point cloud data to the same global coordinate system, and then fuse the aligned multi-frame data to eliminate redundant points in overlapping areas and fill gaps. Finally, complete, continuous, and high-precision processed point cloud data is obtained for subsequent use in building a digital twin model of the power distribution network.

[0047] For embodiments of this disclosure, preprocessing of visible light image data may specifically include:

[0048] First, the Brown-Conrady distortion model is used, utilizing pre-calibrated camera intrinsic parameters. Distortion correction is performed on visible light image data using distortion coefficients (radial distortion k1, k2, k3, tangential distortion p1, p2), and the distorted image coordinates are then... Corrected to distortion-free coordinates To eliminate lens distortion, in the camera imaging model, the principal point is the intersection of the optical axis and the imaging plane, corresponding to the origin of the image coordinate system, expressed in pixel coordinates. , ) express, The pixel coordinates of the principal point in the horizontal direction (x-axis) of the image. The pixel coordinates of the principal point in the vertical direction (y-axis) of the image are a core component of the camera's intrinsic parameters, and are related to the focal length. , Together, they form the camera intrinsic parameter matrix, which is used to describe the camera's geometric imaging relationship. The focal length is in the horizontal direction (x-axis). Let be the focal length in the vertical direction (y-axis), k1 be the first-order radial distortion coefficient, k2 be the second-order radial distortion coefficient, k3 be the third-order radial distortion coefficient, p1 be the tangential distortion coefficient along the vertical direction (y-axis), and p2 be the tangential distortion coefficient along the horizontal direction (x-axis). The coordinates of the horizontally distorted image are... The coordinates of the distorted image in the vertical direction. For horizontal coordinates without distortion, For coordinates without distortion in the vertical direction, the process can be represented as:

[0049]

[0050] In the formula, The distance from the distorted image coordinates to the image center. For undistorted coordinates, For the coordinates of the distorted image, The radial distortion coefficient is... The tangential distortion coefficient;

[0051] Secondly, to improve image visibility under complex lighting conditions and suppress noise amplification, a contrast-limited adaptive histogram equalization algorithm can be used to enhance the contrast of the corrected visible light image data. This algorithm divides the image into several non-overlapping small regions (i.e., context-dependent blocks) and performs histogram equalization independently within each region. To avoid amplifying noise due to excessive local contrast, the algorithm sets a contrast limit threshold (Clip Limit) for the histogram of each block, uniformly cropping and redistributing gray levels exceeding this threshold across the entire gray level range. Finally, bilinear interpolation is used to fuse the processing results of each block to eliminate block artifacts. This step significantly improves the overall contrast of the visible light image data, highlighting the detailed features of power equipment and the background, preparing for subsequent target recognition.

[0052] Finally, to address the image blurring issue caused by vibration or rapid movement during drone flight, inter-frame deblurring is performed on the contrast-enhanced visible light image data. This process leverages the temporal correlation between consecutive frames, reconstructing clear image textures by analyzing the motion vectors or point spread functions (PSFs) of adjacent frames. Specifically, a blind deconvolution algorithm based on multi-frame fusion or a deep learning deblurring network can be used to align consecutive frames and fuse high-frequency information to recover edge details lost due to motion blur. This step optimizes the image texture details of the visible light image data, ensuring the texture clarity of key components (such as insulators and wires) and improving the input quality of subsequent defect detection models.

[0053] Step 102: Construct a digital twin model of the distribution network based on the preprocessed point cloud data and visible light image data. The digital twin model of the distribution network includes a digital surface model, a digital elevation model, a normalized digital surface model, and a semantic model of the distribution network equipment.

[0054] In this embodiment of the disclosure, a digital twin model of the distribution network is constructed based on preprocessed point cloud data and visible light image data. The digital twin model includes a digital surface model (DSM), a digital elevation model (DEM), a normalized digital surface model (nDSM), and a semantic model of the distribution network equipment. The specific construction process is as follows:

[0055] First, a spatial mapping relationship is established between the preprocessed point cloud data and the visible light image data. The texture and color information of the visible light image is then fused into the 3D point cloud to generate a color point cloud with true color information. This is the foundation for achieving multimodal data fusion.

[0056] Specifically, such as Figure 2 As shown, the rigid body transformation matrix from the lidar coordinate system {L} to the camera coordinate system {C} can be obtained through joint calibration. , where R is Rotation matrix, t is Translation vector; at the same time, the camera intrinsic parameter matrix K is obtained through camera calibration, and the coordinate mapping relationship between the three-dimensional spatial points in the point cloud data and the two-dimensional pixel points in the visible light image data is established based on the rigid body transformation matrix and the camera intrinsic parameter matrix.

[0057] Secondly, the pixel coordinates of the preprocessed point cloud (i.e., LiDAR points) in the camera pixel coordinate system are calculated based on the coordinate mapping relationship. The specific formula is as follows:

[0058]

[0059] In the formula, These are the pixel coordinates of the LiDAR points on the camera's image plane. To expand the 3D coordinates of the LiDAR points to homogeneous coordinates, K is the rigid body transformation matrix to be applied, and K is the camera intrinsic parameter matrix to be applied.

[0060] The image color information (RGB values) in the visible light image data is mapped to the point cloud in pixel coordinates to generate a color point cloud (i.e. a three-dimensional point cloud with true color information).

[0061] The colored point cloud is rasterized (e.g., with a resolution of 0.1 meters), and the three-dimensional space is projected onto a two-dimensional grid plane. For each grid cell, the maximum elevation value (Z value) of all point cloud data within it is selected as the grid cell value, generating a digital surface model that reflects the elevation of the earth's surface and all objects on the surface (such as vegetation, buildings, and power equipment).

[0062] Filtering algorithms (such as progressive densification triangular mesh filtering algorithm or slope-based filtering algorithm) can be used to remove non-ground points (such as buildings and vegetation) from the color point cloud, and interpolation processing (such as Kriging interpolation or inverse distance weighted interpolation) can be performed on the ground points in the removed color point cloud to generate a digital elevation model that only reflects the elevation of the bare terrain.

[0063] The digital surface model is subtracted from the digital elevation model (DEM), i.e., nDSM = DSM - DEM, to generate a normalized digital surface model. The normalized digital surface model effectively removes the influence of terrain undulations and intuitively highlights the absolute height of all objects on the ground (such as trees, houses, and power equipment).

[0064] Finally, a semantic model of distribution network equipment can be constructed based on colored point clouds, normalized digital surface models, and pre-trained multimodal feature fusion deep learning models.

[0065] Specifically, the height features of a normalized digital surface model can be used to initially screen potential non-ground object regions. The geometric features (such as linearity and flatness) and texture features (RGB color and gradient) of the color point cloud are then input into a deep learning segmentation network to classify and label each point or voxel in the point cloud. The identified categories include at least: power poles, power lines, insulators, trees, buildings, and other obstacles.

[0066] The categorized and labeled point cloud data is transformed into a collection of three-dimensional objects with semantic attributes, forming a semantic model of power distribution network equipment. In this model, each power equipment object not only contains its precise three-dimensional spatial coordinates and geometric shape, but also includes its semantic category label (such as "110kV conductor", "cement tower", etc.). This semantic model serves as the core data foundation for subsequent obstacle semantic feature extraction, non-power obstacle removal, and the establishment of power equipment topological connection relationships.

[0067] Step 103: Based on the digital twin model of the power distribution network, perform clearance analysis on the power distribution network area to identify power conductors and obstacles in the power distribution network area. Combine environmental perception data, obstacle semantic features and distribution network voltage level to construct a multi-dimensional dynamic clearance threshold system to calculate the safe clearance distance between power conductors and obstacles.

[0068] For embodiments of this disclosure, such as Figure 2 As shown, a clearance analysis is performed on the power distribution network area based on the digital twin model of the distribution network to identify power conductors and obstacles in the power distribution network area. A multi-dimensional dynamic clearance threshold system is constructed by combining environmental perception data, obstacle semantic features and distribution network voltage level to calculate the safe clearance distance between power conductors and obstacles. Specifically, it may include the following steps:

[0069] First, based on the elevation feature values ​​of the normalized digital surface model and the prior knowledge of the semantic model of the distribution network equipment, a joint algorithm of sliding window and semantic segmentation is used to extract candidate regions of conductors in the power distribution network area.

[0070] Specifically, since power conductors are typically suspended within a certain height range (e.g., 10kV lines are usually 5-15m, and 35kV lines are even higher), a sliding window (e.g., 3m×3m) is set and traversed on the normalized digital surface model to filter out continuous areas whose elevation values ​​meet the preset voltage level suspension height range and whose aspect ratios exhibit linear characteristics as initial candidate areas. Subsequently, the probability map of the "conductor" category from the semantic model of the distribution network equipment is introduced to perform a secondary filtering on the initial candidate areas, eliminating false detection areas with similar elevations but semantic categories such as "tree branches" or "building edges," resulting in high-precision conductor candidate areas.

[0071] Next, the Random Sample Consensus (RANSAC) algorithm with introduced conductor curvature constraints is used to fit and obtain the three-dimensional coordinates of the power conductors in the candidate conductor region. Traditional RANSAC algorithms typically fit straight lines, but actual conductors are catenary-shaped due to gravity. Therefore, this embodiment introduces a curvature constraint term in the interior point determination stage of RANSAC to construct a mathematical model that includes catenary parameters or piecewise quadratic curve parameters.

[0072] The equation of the straight line of the conductor is as follows:

[0073]

[0074] In the formula, Let A, B, and C be the coordinates of a known point on the traverse, A, B, and C be the direction vectors of the traverse, and X, Y, and Z be the coordinates of any point in space.

[0075] Based on the digital twin model of the power distribution network, the three-dimensional geometric and semantic features of obstacles are integrated to identify and classify them. Specifically, areas with elevation feature values ​​greater than 0 (i.e., above ground level) that are not marked as "conductors" or "towers" (i.e., non-conductor areas) are selected from the normalized digital surface model as potential obstacle areas. For example, all non-conductor objects with a height exceeding 0.5 meters in the normalized digital surface model are identified as potential obstacle areas.

[0076] The system extracts the three-dimensional geometric features of the obstacle region (such as point cloud density, voxelized occupancy grid, normal vector distribution, and convex hull volume) and texture features mapped from the visible light image. These features are then input into a pre-trained deep learning semantic segmentation network for semantic classification, and the system outputs a semantic category label for each point or region.

[0077] Based on semantic category labels, obstacles are classified into different categories (i.e., semantic classification results), which include at least one of the following categories: trees, buildings, and construction machinery. For example, areas with irregular surfaces and seasonal variations are labeled as "trees"; areas with regular geometric shapes and high-reflectivity roofs are labeled as "buildings"; and objects that are temporary, have variable shapes, and are located in construction areas are labeled as "construction machinery." This step solves the problem of traditional methods failing to distinguish obstacle types, leading to a "one-size-fits-all" threshold assessment.

[0078] Traditional airspace safety thresholds are static (based solely on voltage levels), while this application dynamically adjusts the airspace safety threshold based on distribution network voltage levels, obstacle semantic classification results, and real-time collected environmental sensing data. The environmental sensing data includes wind speed, temperature, humidity, and icing thickness. The airspace safety threshold is increased by 20% when wind speed is ≥5 m / s, and by 30% when icing thickness is ≥5 mm.

[0079] Finally, based on the fitted three-dimensional coordinates of the conductor, the surface point coordinates of the obstacle, and the current environmental perception data, the minimum three-dimensional Euclidean distance between the power conductor and the obstacle can be calculated and compared with the dynamic clearance safety threshold.

[0080] The dynamic clearance safety threshold is determined based on the basic safety distance, which depends on the distribution network voltage level and the type of obstacle. For example, according to the "Distribution Network Operation Regulations," the basic distance between a 10kV conductor and a tree is 1.5m, and to a building is 2.5m; for a 35kV conductor, the basic distance between a tree and a building is 3.0m, and to a building is 4.0m. Additional safety margins can be automatically added for high-risk obstacles such as construction machinery.

[0081] In the specific calculation, every point in the point cloud on the obstacle surface can be traversed. Using the three-dimensional spatial distance formula, the minimum three-dimensional Euclidean distance between the power line and the obstacle can be calculated based on the three-dimensional coordinates of the conductor and the surface point coordinates of the obstacle. The formula is as follows:

[0082]

[0083] In the formula, The minimum distance in three dimensions is given by P, where P is any point on the power line, Q is any point on the surface of the obstacle, and V is the direction vector of the power line.

[0084] Considering the impact of wind deflection in environmental perception data, the actual assessment distance needs to be calculated by subtracting the predicted wind deflection displacement from the geometric distance, or by directly comparing the minimum three-dimensional Euclidean distance with the safety threshold. If the minimum three-dimensional Euclidean distance is greater than or equal to the safety threshold, it is considered safe.

[0085] If the minimum three-dimensional Euclidean distance is less than the safety clearance threshold, a safety clearance hazard is identified. Record the specific location of the hazard, the type of obstacle involved (e.g., "trees under a 35kV line"), the current measured distance, the threshold difference, and the main influencing factors (e.g., "insufficient wind deflection due to strong winds"). The final determined safety clearance distance is the assessment result after environmental correction.

[0086] Step 104: Based on the safety distance and semantic features of obstacles, risk classification is carried out, non-electric obstacles in the power distribution network area are eliminated, power poles in the power distribution network area are extracted, and the topological connection relationship between power conductors and power poles is established through spatial topological relationship analysis. A three-dimensional model of power distribution network equipment containing spatial location information and topological relationship information of power equipment is constructed.

[0087] In this embodiment of the disclosure, risk classification is performed based on the safety clearance distance and semantic features of obstacles. Non-power obstacles in the power distribution network area are eliminated, power poles in the power distribution network area are extracted, and the topological connection relationship between power conductors and power poles is established through spatial topological relationship analysis. A three-dimensional model of the power distribution network equipment containing spatial location information and topological relationship information of the power equipment is constructed, which may specifically include:

[0088] First, a classifier that integrates geometric and semantic features is used to classify surface objects in the normalized digital surface model (nDSM) and the corresponding color point cloud data, so as to accurately distinguish between power equipment objects and non-power obstacle objects.

[0089] Specifically, clustering algorithms (such as DBSCAN or Euclidean clustering) are first used to segment the non-ground point cloud on the normalized digital surface model, dividing spatially adjacent points into independent object instances (such as a single tree, a single tower, or a single building). Subsequently, a multi-dimensional feature vector is extracted for each object instance, which may include geometric and semantic features.

[0090] Among them, geometric features can be nDSM elevation values, point cloud density, aspect ratio, flatness, elevation standard deviation, object surface area, verticality, etc.

[0091] Semantic features can be texture histograms extracted from visible light images, color mean, and class probability distributions output by a pre-trained deep learning network.

[0092] Based on the above features, a classifier (such as random forest, support vector machine or deep neural network) is constructed to classify and label each object instance, classifying it into categories such as "power pole", "power line", "tree", "building", "construction machinery" etc., thereby clearly distinguishing between power equipment objects and non-power obstacle objects.

[0093] Based on the above classification results, for the areas marked as candidates for "power poles," a joint algorithm combining morphological dilation, contour detection, and semantic verification is used to extract the tower contours of the power poles, combining the unique geometric features of power poles (such as nDSM elevation values ​​typically ranging from 10-30m, bottom diameters from 3-5m, and top diameters from 1-2m, exhibiting obvious vertical linearity) and semantic features. The 3D bounding box coordinates of the power poles are then calculated. The specific steps are as follows:

[0094] First, morphological dilation operations are performed on the binarized masks classified as towers (e.g., using 5×5 or 7×7 rectangular / circular structural elements) to fill the voids in the tower body caused by sparse or occluded point clouds and connect broken edges.

[0095] Apply contour detection algorithms (such as Canny edge detection or contour finding algorithms) to the enhanced mask to extract the precise closed contour of the tower on the horizontal projection plane.

[0096] The extracted contours are mapped back to the original point cloud to verify whether the verticality, height distribution, and texture features of the internal points conform to the physical properties of the pole, and to eliminate falsely detected non-pole objects.

[0097] For the verified tower profile, its minimum bounding box coordinates are calculated by combining the elevation range (minimum to maximum value of the Z-axis) of its corresponding point cloud.

[0098] By analyzing spatial topology, the power conductors and power poles obtained in the previous steps are spatially associated and semantically bound to establish a topological connection between the conductors and poles.

[0099] Specifically, the three-dimensional coordinate sequence of each power conductor segment is traversed, and the spatial distances between its two endpoints and the tops of the three-dimensional bounding boxes of all extracted power poles are calculated. If the distance between a conductor endpoint and the top of a pole's bounding box is less than a preset threshold (e.g., 1.0 meter), and the conductor's direction is geometrically compatible with the direction of the pole's crossarm, then the conductor is determined to be suspended on that pole. This constructs a graph-structured topology model. , where nodes Represents power poles, side This represents a power conductor. Each edge records the starting tower ID, ending tower ID, conductor type, and geometric parameters of its connection. This topological connection not only describes the spatial location of equipment but also reconstructs the electrical connection logic of the power distribution network.

[0100] Based on the calculated safety clearance distance and semantic features of non-power obstacles, risk classification is performed on non-power obstacles. Low-priority non-power obstacles in the power distribution network area are then removed according to their risk level to simplify the model and focus on core risks. The risk classification strategy is as follows:

[0101] If the semantic type of the obstacle is "tree" or "construction machinery," and its measured distance from the guide wire is less than the dynamic clearance safety threshold, or within the safety margin range of the threshold (e.g., distance < 1.2 times the threshold), such objects are retained in the model and marked as "emergency hazards," i.e., the risk level is high risk.

[0102] If the obstacle's semantic type is "building" and it is relatively close but within the limit, or its semantic type is "tree" but it is relatively far away, such objects will be retained in the model as "objects of concern," i.e., their risk level is medium risk.

[0103] If the obstacle's semantic type is "shrub", "grass", or other objects far from the guide wire, and the measured distance is much greater than the dynamic clearance safety threshold, then the risk level is low.

[0104] Non-power obstacle objects that are judged as "low risk" can be removed from the dataset and no longer participate in subsequent modeling, thereby reducing data redundancy and highlighting elements that have a real impact on the safe operation of the power grid.

[0105] Finally, based on the power equipment objects retained in the power distribution network area (including screened high / medium risk obstacles as environmental background, as well as all poles and conductors), 3D bounding box coordinates, and topological connections, a final 3D model of the power distribution network equipment containing the spatial location information and topological relationship information of poles, conductors, and auxiliary components is generated.

[0106] Step 105: Inversely project the spatial location information of the power equipment in the 3D model of the power distribution network onto the visible light image data to determine the image bounding boxes of key components. Use a pre-trained deep learning target detection model that fuses multimodal features to identify the key component images within the image bounding boxes. The deep learning target detection model that fuses multimodal features is a YOLOv8 detection model that fuses 3D geometric features and 2D texture features.

[0107] First, the coordinate mapping relationship (i.e., camera intrinsic parameter matrix and rigid body transformation matrix) between point cloud data and visible light image data established in the aforementioned steps can be used to reverse project the spatial location information of power equipment in the three-dimensional model of power distribution network equipment onto the two-dimensional visible light image plane.

[0108] Specifically, the 3D bounding box vertex coordinates of the key components to be detected (such as insulator strings, conductor clamps, surge arresters, etc.) in the 3D model are extracted, and their projected coordinates in the image pixel coordinate system are calculated. Based on the projected coordinates of all vertices, the minimum bounding rectangle is calculated, thereby determining the region of interest (ROI) of the key component in the visible light image, i.e., the image bounding box. This step utilizes 3D prior knowledge to greatly narrow the search range and eliminate background interference.

[0109] This embodiment uses an improved YOLOv8 architecture as the basic detection model to construct a multimodal feature fusion deep learning object detection model that integrates 3D geometric features and 2D texture features. The specific construction process of this model includes the following three core improvement modules:

[0110] (1) Add a point cloud feature projection fusion module to the backbone network.

[0111] In the YOLOv8 backbone network, a point cloud feature projection fusion module is added. This module is configured to project the three-dimensional geometric features extracted from the three-dimensional model of the power distribution network equipment (including but not limited to the spatial relative position, size ratio, and normal vector distribution of tower components) onto the two-dimensional image feature layer.

[0112] In practice, the 3D point cloud can be voxelized or converted into a depth map / height map. Geometric feature maps are then extracted through a lightweight convolutional network. Simultaneously, the visible light image is processed by a backbone CNN to extract 2D texture feature maps. Using the aforementioned projection relationship, the geometric feature maps are aligned and resampled to the same resolution as the 2D texture feature maps. Then, through channel concatenation or element-wise addition, the 3D geometric information and 2D texture information are deeply fused in the shallow or middle layers of the backbone network to generate multimodal fusion features. This allows the model to not only "see" color textures but also "perceive" the spatial structure of objects.

[0113] (2) Neck network introduces attention mechanism fusion layer

[0114] In the Neck network (feature pyramid part) of the YOLOv8 architecture, an attention mechanism fusion layer (such as CBAM, SE-Block, or self-attention mechanism) is introduced. This layer is configured to perform attention-weighted processing on the aforementioned multimodal fusion features. Since the background in power distribution network scenarios is complex (e.g., interference from trees and buildings), and key components (e.g., insulators, vibration dampers) are typically small in size, the attention mechanism can adaptively learn the importance weights of feature channels and spatial locations. Through this layer, the model can significantly enhance the feature representation weights of key components (e.g., insulators, cable joints, etc.), suppress background noise and the response of non-critical areas, thereby improving the model's feature extraction capabilities for small and occluded targets.

[0115] (3) Add geometric constraint loss term to the loss function

[0116] Based on the original loss function of the YOLOv8 architecture (which can include classification loss, bounding box regression loss, and confidence loss), a geometric constraint loss term is innovatively added. .

[0117] Taking YOLO as an example, its loss function L is approximately:

[0118]

[0119] In the formula, L is the loss function. For coordinate loss weights, Weight the object confidence loss. Weights for the no-object confidence loss. For class loss weights, Loss is calculated for the bounding box location coordinates. For object confidence loss, For objectless confidence loss, This is the category loss.

[0120] The geometric constraint loss term is configured to construct a constraint term based on the deviation of three-dimensional geometric features and participate in the backpropagation calculation to optimize the model's regression ability on the target spatial location and its class discrimination ability. Its calculation formula is as follows:

[0121]

[0122] In the formula, For geometric constraint loss terms, These are the weighting coefficients. These are the coordinates of the components in the 3D model. These are the projected coordinates in the two-dimensional image.

[0123] The image patches cropped from the defined Regions of Interest (ROIs) are input into the pre-trained deep learning object detection model that integrates multimodal feature fusion. After forward propagation, the model outputs precise bounding boxes, class labels, and confidence scores for key components such as insulators, cables, tower hardware, and grading rings.

[0124] By incorporating three-dimensional geometric priors and geometric constraint losses, this model can maintain extremely high detection robustness and positioning accuracy even in complex backgrounds (such as tree branches obscuring insulators and changes in light and shadow affecting metal reflection).

[0125] Step 106: Extract the three-dimensional geometric features and two-dimensional texture features corresponding to the key component images, construct a geometric-texture dual guiding factor, and use the geometric-texture dual guiding factor to perform defect feature adaptive enhancement processing on the key component images to obtain the defect-enhanced key component images.

[0126] For embodiments of this disclosure, such as Figure 3 As shown, the three-dimensional geometric features and two-dimensional texture features corresponding to the key component images are extracted, and a geometric-texture dual-guided factor is constructed. This geometric-texture dual-guided factor is then used to adaptively enhance the defect features of the key component images, resulting in defect-enhanced key component images. Specifically, this may include:

[0127] Extract images of key components (such as insulators, conductor clamps, surge arresters, etc.) and their corresponding 3D point cloud clusters, and extract high-dimensional 3D geometric features and 2D texture features respectively. The 3D geometric features may include surface normal map N, local curvature map K, and relative height map H.

[0128] Among them, the surface normal map N: using principal component analysis (PCA) or the neighborhood fitting plane method, the normal vector of each point is calculated to characterize the local orientation change of the component surface;

[0129] Local curvature map K: Based on normal information, the Gaussian curvature or average curvature of each point is calculated. Positive values ​​indicate convexity, negative values ​​indicate concavity, and the absolute value indicates the degree of convexity or concavity. It is a sensitive indicator for identifying geometric deformation defects such as cracks and damage.

[0130] Relative height map H: Directly uses the normalized digital surface model data of this area to characterize the absolute height of each point of the component relative to the ground, and is used to distinguish the suspended equipment from the ground background;

[0131] Geometric roughness map: Calculates the standard deviation of the point cloud elevation in the local neighborhood or the root mean square distance from the point to the fitted plane, characterizing the micro-roughness of the surface, and is used to identify surface degradation defects such as corrosion and weathering.

[0132] Two-dimensional texture features may include texture gradient maps, edge detection maps, gray-level co-occurrence matrix feature maps, and defect texture sensitivity maps.

[0133] Among them, the texture gradient map is used to calculate the gradient magnitude and direction of the image using the Sobel or Scharr operator, which represents the dramatic changes in edges and details.

[0134] Edge detection map: Use Canny or Laplacian operators to extract clear edge contours.

[0135] Gray-level co-occurrence matrix feature map: calculates the contrast, energy, entropy and second moment of local regions, and quantifies the roughness, regularity and directionality of the texture.

[0136] Defect texture sensitivity map: Based on a pre-trained texture anomaly detection network or specific frequency domain filtering (such as Gabor filter group), it highlights areas that are significantly different from the texture patterns of normal parts (such as dirt patches, discharge traces).

[0137] Based on the relative height map and geometric roughness map, a parameter-adaptive sigmoid function is used to calculate the geometric guidance factor, aiming to accurately distinguish the power equipment body area from the background area and mark geometrically abnormal areas. The specific calculation process is as follows:

[0138] First, the highly significant weights are calculated using the Sigmoid function. This makes the weight of the power equipment body area (high nDSM value) approach 1, and the weight of the ground background area approach 0, as shown in the following formula:

[0139]

[0140] In the formula, The highly significant weights at image pixel coordinates (u,v) are used. This represents the nDSM height value of the relative height map at the image pixel coordinates (u,v). The preset height threshold is used to distinguish power equipment from the ground background, and α is the slope parameter that controls the steepness of the Sigmoid function curve.

[0141] Defect sensitivity weights are calculated based on surface normal maps and local curvature maps. Combined with the variance of normal variation and absolute value of curvature The formula for identifying regions with geometric anomalies is shown below:

[0142]

[0143] In the formula, Let (u, v) be the defect-sensitive weight at the image pixel coordinates (u, v). and These are harmonic coefficients, used to adjust the weights of normal variation and curvature in defect sensitivity calculations. Indicated in pixels Within a local window centered on the surface, the variance of the surface normal direction is used to characterize the degree of surface irregularity. It represents the absolute value of the local curvature at that point, used to characterize the degree of unevenness of the surface.

[0144] Secondly, geometric anomaly weights are constructed by combining the geometric roughness map. The formula is shown below:

[0145]

[0146] In the formula, Let be the geometric anomaly weight at the image pixel coordinates (u,v). For normalization operations, This is the geometric roughness map at the image pixel coordinates (u,v).

[0147] The geometric guiding factor is obtained by performing a weighted product of the high significance weight and the geometric anomaly weight. The formula is shown below:

[0148]

[0149] In the formula, As a geometric guiding factor, The highly significant weights at image pixel coordinates (u,v) are used. This represents the geometric anomaly weight at the image pixel coordinates (u,v).

[0150] Based on the texture gradient map and the defect texture sensitivity map, a Gaussian weighted algorithm is used to calculate the texture guiding factor, which aims to distinguish defective texture regions from normal texture regions and mark texture anomalous regions. The formula is shown below:

[0151]

[0152] In the formula, As a texture guiding factor, This is a texture-sensitive map of defects at image pixel coordinates (u,v). This is the texture gradient map at pixel coordinates (u,v) in the image. For normalization operations, The mean of the defect texture sensitivity map. denoted as the standard deviation of the defect texture sensitivity map.

[0153] The geometry guiding factor and the texture guiding factor are weighted and fused to obtain the final geometry-texture dual guiding factor. :

[0154]

[0155] In the formula, It is a geometry-texture dual guiding factor. The fusion weights for the geometric guiding factors, As a geometric guiding factor, The fusion weights for texture guiding factors. This is a texture guiding factor.

[0156] Among them, the fusion weight of the geometric guiding factor fusion weights with texture guiding factors The configuration is set to dynamically adjust based on the lighting conditions or background complexity of the inspection scenario.

[0157] Traditional Retinex algorithms use fixed parameters, which can easily lead to over-enhancing of the background or loss of detail. This embodiment utilizes a geometry-texture dual-guided factor to adaptively adjust the key parameters of the Retinex image enhancement algorithm, performing enhancement processing on images of key components. The formula is shown below:

[0158]

[0159] In the formula, Let be the filter kernel size of the Retinex image enhancement algorithm after adaptive adjustment at the image pixel coordinates (u,v). This is the maximum value of the preset filter kernel size. This is the minimum value of the preset filter kernel size. It is a geometry-texture dual guiding factor.

[0160] In defective regions (large G-values), a small filter kernel is used to preserve details and achieve high-intensity local enhancement; in background / flat regions (small G-values), a large filter kernel is used to smooth illumination, suppress noise, and prevent over-enhancement. The illumination component L is estimated using the adjusted kernel function, and the reflection component R = I / L is calculated, where I is the original input image. A geometric guidance factor can be used to adaptively stretch the contrast of the reflection component R, ultimately obtaining the enhanced image of the key components. This process requires no manual intervention; its core lies in utilizing the normal and curvature information in the 3D point cloud to precisely enhance geometrically anomalous regions on a 2D image while suppressing flat backgrounds.

[0161] In summary, the power distribution network equipment inspection method provided in this application, compared with the existing technology, can simultaneously collect point cloud data and visible light image data from lidar in the power distribution network area, and preprocess the point cloud data and visible light image data; based on the preprocessed point cloud data and visible light image data, a digital twin model of the power distribution network is constructed, including a digital surface model, a digital elevation model, a normalized digital surface model, and a semantic model of the power distribution network equipment; based on the digital twin model of the power distribution network, a clearance analysis is performed on the power distribution network area to identify power conductors and obstacles in the power distribution network area; and a multi-dimensional dynamic clearance threshold system is constructed by combining environmental perception data, obstacle semantic features, and power distribution network voltage levels to calculate the clearance between power conductors and obstacles. The system employs several methods: First, it establishes a safety distance for the power distribution network. Based on the safety distance and semantic features of obstacles, risk classification is performed. Non-electrical obstacles in the power distribution network area are removed. Power poles in the power distribution network area are extracted, and the topological connection between power conductors and power poles is established through spatial topological relationship analysis. A three-dimensional model of the power distribution network equipment, containing spatial location information and topological relationship information, is constructed. The spatial location information of the power equipment in the three-dimensional model is then projected onto visible light image data to determine the image bounding boxes of key components. A pre-trained deep learning target detection model integrating multimodal feature fusion is used to identify the key component images within the image bounding boxes. This multimodal feature fusion deep learning target detection model is a YOLOv8 detection model that integrates three-dimensional geometric features and two-dimensional texture features. The three-dimensional geometric features and two-dimensional texture features corresponding to the key component images are extracted, and a geometric-texture dual-guiding factor is constructed. This geometric-texture dual-guiding factor is used to adaptively enhance the defect features of the key component images, resulting in defect-enhanced images of the key components.

[0162] Using the above technical solution, this application constructs a digital twin model of the distribution network (including a digital surface model, a digital elevation model, a normalized digital surface model, and a semantic model of distribution network equipment) based on synchronously acquired lidar point cloud and visible light image data. This model is used for unified clearance analysis (identifying conductors and obstacles, calculating safe clearance distances) and risk classification. Subsequently, based on the analysis results, power poles are extracted and topological relationships are established to construct a three-dimensional model of the distribution network power equipment. Finally, the spatial location information in the three-dimensional model is inversely projected onto the visible light image to determine the image bounding boxes of key components, enabling direct defect detection. This achieves seamless data flow from clearance analysis to defect detection, integrating two originally independent processes into a unified framework based on digital twins, eliminating the need for manual switching or repetitive data processing.

[0163] This application constructs a digital twin model incorporating the semantic model of power distribution network equipment. By combining environmental perception data and obstacle semantic features, it can accurately distinguish between "power equipment" and non-power obstacles such as "trees and houses," and eliminate the latter. A 3D model is constructed using point cloud data, solving the problem of inaccurate 3D distance calculation using only visible light; semantic features are used to address the difficulty of classification using only LiDAR. In the defect detection stage, a YOLOv8 detection model fusing 3D geometric features and 2D texture features is used, improving the accuracy of target recognition.

[0164] This application extracts the 3D geometric features and 2D texture features corresponding to the images of key components, constructing a "geometry-texture dual-guided factor." This dual-guided factor is then used to adaptively enhance the defect features of the key component images. Unlike traditional global or 2D feature-based enhancements (such as Retinex), this method utilizes 3D geometric information to distinguish between real defects and background texture. In complex scenes (lighting variations, background obfuscation), it avoids excessive background enhancement, focusing on enhancing the defect area, thereby reducing the false negative rate and improving recognition accuracy.

[0165] Based on the above Figure 1 The specific implementation of the method shown in this embodiment provides a power distribution network equipment inspection device, such as... Figure 4 As shown, the device includes: a data acquisition module 31, a first construction module 32, a calculation module 33, a second construction module 34, a recognition module 35, and an enhancement module 36;

[0166] The acquisition module 31 is used to simultaneously acquire point cloud data and visible light image data of the lidar in the power distribution network area, and to preprocess the point cloud data and the visible light image data.

[0167] The first construction module 32 is used to construct a digital twin model of the distribution network based on the preprocessed point cloud data and the visible light image data. The digital twin model of the distribution network includes a digital surface model, a digital elevation model, a normalized digital surface model, and a semantic model of the distribution network equipment.

[0168] The calculation module 33 is used to perform clearance analysis on the power distribution network area based on the power distribution network digital twin model, so as to identify the power conductors and obstacles in the power distribution network area, and construct a multi-dimensional dynamic clearance threshold system by combining environmental perception data, obstacle semantic features and distribution network voltage level, so as to calculate the clearance safety distance between the power conductors and the obstacles.

[0169] The second construction module 34 is used to perform risk classification based on the clearance safety distance and the semantic features of the obstacle, remove non-power obstacles in the power distribution network area, extract the power poles in the power distribution network area and establish the topological connection relationship between the power conductor and the power pole through spatial topological relationship analysis, and construct a three-dimensional model of the power distribution network equipment containing the spatial location information and topological relationship information of the power equipment.

[0170] The identification module 35 is used to reverse project the spatial location information of the power equipment in the three-dimensional model of the power distribution network equipment onto the visible light image data, determine the image bounding box of the key components, and use a pre-trained deep learning target detection model with multimodal feature fusion to identify the key component image within the image bounding box. The deep learning target detection model with multimodal feature fusion is a YOLOv8 detection model that fuses three-dimensional geometric features and two-dimensional texture features.

[0171] The enhancement module 36 is used to extract the three-dimensional geometric features and two-dimensional texture features corresponding to the key component image, construct a geometric-texture dual guiding factor, and use the geometric-texture dual guiding factor to perform defect feature adaptive enhancement processing on the key component image to obtain a defect-enhanced key component image.

[0172] In a specific application scenario, the acquisition module 31 is specifically used to employ a statistical outlier removal algorithm to calculate the average distance from each point in the point cloud data to its nearest neighbors, and remove noise points in the point cloud data whose average distance exceeds the preset Gaussian distribution range; to employ a voxel grid downsampling algorithm to filter and downsample the denoised point cloud data, and to perform inter-frame registration and fusion processing on the filtered and downsampled point cloud data to obtain the processed point cloud data.

[0173] In specific application scenarios, the acquisition module 31 is specifically used to perform distortion correction processing on the visible light image data using the Brown-Conrady distortion model and pre-calibrated camera intrinsic parameters and distortion coefficients; to perform contrast enhancement processing on the corrected visible light image data using a limited contrast adaptive histogram equalization algorithm to improve the overall contrast of the visible light image data; and to perform inter-frame deblurring processing on the contrast-enhanced visible light image data to optimize the image texture details of the visible light image data.

[0174] In specific application scenarios, the calculation module 33 is specifically used to extract candidate conductor regions in the power distribution network area based on the elevation feature values ​​of the normalized digital surface model and the semantic model of the distribution network equipment, using a sliding window and semantic segmentation joint algorithm. It then uses a random sampling consensus algorithm incorporating conductor curvature constraints to fit and obtain the three-dimensional coordinates of the power conductors in the candidate conductor regions. Based on the distribution network digital twin model, it fuses the three-dimensional geometric and semantic features of the obstacles, identifies regions in the power distribution network area with elevation feature values ​​greater than 0 that are not conductors as obstacles, and performs semantic classification on these obstacles. Obtain the semantic classification result of the obstacle, wherein the semantic classification result includes at least one of the following classifications: tree, building, and construction machinery; dynamically adjust the airspace safety threshold according to the distribution network voltage level, the obstacle semantic classification result, and the environmental perception data, wherein the environmental perception data includes wind speed, temperature, humidity, and icing thickness; calculate the minimum three-dimensional Euclidean distance between the power conductor and the obstacle based on the three-dimensional coordinates of the conductor, the surface point coordinates of the obstacle, and the environmental perception data, and compare the minimum three-dimensional Euclidean distance with the airspace safety threshold to determine the final airspace safety distance.

[0175] In a specific application scenario, the second construction module 34 is specifically used to classify surface objects in the normalized digital surface model using a classifier that integrates geometric and semantic features to distinguish between power equipment objects and non-power obstacle objects. Based on the classification results, combined with the geometric and semantic features of the power poles, a joint algorithm of morphological dilation, contour detection, and semantic verification is used to extract the pole contour and calculate the three-dimensional bounding box coordinates of the power poles. Through spatial topology analysis, the power conductors and power poles are spatially associated and semantically bound to establish a conductor-tower topological connection relationship. Based on the clearance safety distance and the semantic features of the non-power obstacle objects, the non-power obstacle objects are risk-classified, and low-priority non-power obstacle objects in the power distribution network area are removed according to the risk level. Based on the power equipment objects retained in the power distribution network area, the three-dimensional bounding box coordinates, and the topological connection relationship, a three-dimensional model of the power distribution network equipment is generated. The three-dimensional model of the power distribution network equipment includes the spatial location information and topological relationship information of the poles, conductors, and auxiliary components.

[0176] In specific application scenarios, the recognition module 35 is specifically used to add a point cloud feature projection fusion module to the backbone network of the YOLOv8 architecture. The point cloud feature projection fusion module is configured to project the three-dimensional geometric features of the three-dimensional model of the power distribution network equipment onto a two-dimensional image feature layer to achieve multimodal fusion of the three-dimensional geometric features and the two-dimensional texture features. An attention mechanism fusion layer is introduced into the Neck network of the YOLOv8 architecture. This attention mechanism fusion layer is configured to perform attention-weighted processing on the multimodal fused features to enhance the feature representation weights of the key component images. A geometric constraint loss term is added to the loss function of the YOLOv8 architecture. This geometric constraint loss term is configured to construct constraint terms based on the deviation of the three-dimensional geometric features and participate in backpropagation calculation to optimize the model's regression ability and category discrimination ability for the target spatial location.

[0177] In specific application scenarios, enhancement module 36 is specifically used to extract the three-dimensional geometric features of the three-dimensional point cloud clusters corresponding to the key component image, and to extract the two-dimensional texture features of the key component image. The three-dimensional geometric features include a surface normal map, a local curvature map, a relative height map, and a geometric roughness map. The two-dimensional texture features include a texture gradient map, an edge detection map, a gray-level co-occurrence matrix feature map, and a defect texture sensitivity map. Based on the relative height map and the geometric roughness map, a geometric guidance factor is calculated using a parameter-adaptive Sigmoid function to distinguish between the power equipment body area and the background area, and to mark geometrically abnormal areas. Based on the texture gradient map and the defect texture sensitivity map, a Gaussian weighted algorithm is used to calculate a texture guidance factor to distinguish between defective texture areas and normal texture areas, and to mark texturely abnormal areas. The geometric guidance factor and the texture guidance factor are weighted and fused to obtain the geometric-texture dual guidance factor, wherein the weights of the weighted fusion are dynamically adjusted according to the lighting conditions or background complexity of the inspection scene. Based on the geometric-texture dual guidance factor, the Retinex is adaptively adjusted. The image enhancement algorithm uses a filter kernel size, enhancement intensity, and texture preservation coefficient to perform processing on the key component image, including enhancing the contrast of the defective texture region, preserving the normal texture region, and suppressing background interference, to obtain the enhanced key component image.

[0178] It should be noted that other corresponding descriptions of the functional units involved in the power distribution network equipment inspection device provided in this embodiment can be found in [reference needed]. Figure 1 The corresponding descriptions in [the document] will not be repeated here.

[0179] Based on the above, Figure 1Accordingly, this embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. Figure 1 The method shown.

[0180] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0181] Based on the above, Figure 1 The method shown, and Figure 4 To achieve the above objectives, the present application also provides an electronic device, comprising a storage medium and a processor; the storage medium for storing a computer program; and the processor for executing the computer program to implement the above-described virtual device embodiments. Figure 1 The method shown.

[0182] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0183] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0184] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical devices, supporting the operation of power distribution equipment inspection programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software within the power distribution equipment inspection physical devices.

[0185] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms, or it can be implemented by hardware. By applying the solution of this embodiment, compared with the existing technology, this application can simultaneously collect point cloud data and visible light image data of lidar in the power distribution network area, and preprocess the point cloud data and visible light image data; construct a distribution network digital twin model based on the preprocessed point cloud data and visible light image data, the distribution network digital twin model includes a digital surface model, a digital elevation model, a normalized digital surface model, and a distribution network equipment semantic model; perform clearance analysis on the power distribution network area based on the distribution network digital twin model to identify power conductors and obstacles in the power distribution network area, and construct a multi-dimensional dynamic clearance threshold system by combining environmental perception data, obstacle semantic features, and distribution network voltage level to calculate the clearance safety distance between power conductors and obstacles; Risk classification is performed based on clearance safety distance and obstacle semantic features. Non-electrical obstacles in the power distribution network area are eliminated, and power poles in the power distribution network area are extracted. The topological connection relationship between power conductors and power poles is established through spatial topological relationship analysis. A three-dimensional model of power distribution network equipment containing spatial location information and topological relationship information of power equipment is constructed. The spatial location information of power equipment in the three-dimensional model of power distribution network equipment is inversely projected onto visible light image data to determine the image bounding boxes of key components. A pre-trained deep learning object detection model with multimodal feature fusion is used to identify the key component images within the image bounding boxes. The deep learning object detection model with multimodal feature fusion is a YOLOv8 detection model that integrates three-dimensional geometric features and two-dimensional texture features. The three-dimensional geometric features and two-dimensional texture features corresponding to the key component images are extracted, and a geometric-texture dual guiding factor is constructed. The geometric-texture dual guiding factor is used to perform defect feature adaptive enhancement processing on the key component images to obtain the defect-enhanced key component images.

[0186] Using the above technical solution, this application constructs a digital twin model of the distribution network (including a digital surface model, a digital elevation model, a normalized digital surface model, and a semantic model of distribution network equipment) based on synchronously acquired lidar point cloud and visible light image data. This model is used for unified clearance analysis (identifying conductors and obstacles, calculating safe clearance distances) and risk classification. Subsequently, based on the analysis results, power poles are extracted and topological relationships are established to construct a three-dimensional model of the distribution network power equipment. Finally, the spatial location information in the three-dimensional model is inversely projected onto the visible light image to determine the image bounding boxes of key components, enabling direct defect detection. This achieves seamless data flow from clearance analysis to defect detection, integrating two originally independent processes into a unified framework based on digital twins, eliminating the need for manual switching or repetitive data processing.

[0187] This application constructs a digital twin model incorporating the semantic model of power distribution network equipment. By combining environmental perception data and obstacle semantic features, it can accurately distinguish between "power equipment" and non-power obstacles such as "trees and houses," and eliminate the latter. A 3D model is constructed using point cloud data, solving the problem of inaccurate 3D distance calculation using only visible light; semantic features are used to address the difficulty of classification using only LiDAR. In the defect detection stage, a YOLOv8 detection model fusing 3D geometric features and 2D texture features is used, improving the accuracy of target recognition.

[0188] This application extracts the 3D geometric features and 2D texture features corresponding to the images of key components, constructing a "geometry-texture dual-guided factor." This dual-guided factor is then used to adaptively enhance the defect features of the key component images. Unlike traditional global or 2D feature-based enhancements (such as Retinex), this method utilizes 3D geometric information to distinguish between real defects and background texture. In complex scenes (lighting variations, background obfuscation), it avoids excessive background enhancement, focusing on enhancing the defect area, thereby reducing the false negative rate and improving recognition accuracy.

[0189] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0190] The above are merely specific embodiments of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for inspecting power distribution network equipment, characterized in that, include: Simultaneously collect point cloud data and visible light image data from lidar in the power distribution network area, and preprocess the point cloud data and visible light image data; A digital twin model of the distribution network is constructed based on the preprocessed point cloud data and the visible light image data. The digital twin model of the distribution network includes a digital surface model, a digital elevation model, a normalized digital surface model, and a semantic model of the distribution network equipment. Based on the digital twin model of the power distribution network, an airspace analysis is performed on the power distribution network area to identify power conductors and obstacles in the power distribution network area. A multi-dimensional dynamic airspace threshold system is constructed by combining environmental perception data, obstacle semantic features and distribution network voltage level to calculate the safe airspace distance between the power conductors and the obstacles. Risk classification is performed based on the clearance safety distance and the semantic features of the obstacles. Non-electric obstacles in the power distribution network area are eliminated, power poles in the power distribution network area are extracted, and the topological connection relationship between the power conductors and the power poles is established through spatial topological relationship analysis. A three-dimensional model of the power distribution network equipment containing the spatial location information and topological relationship information of the power equipment is constructed. The spatial location information of the power equipment in the three-dimensional model of the power distribution network is reverse-projected onto the visible light image data to determine the image bounding box of the key components. The key component image within the image bounding box is identified by a pre-trained deep learning target detection model that fuses multimodal features. The deep learning target detection model that fuses multimodal features is a YOLOv8 detection model that fuses three-dimensional geometric features and two-dimensional texture features. The three-dimensional geometric features and two-dimensional texture features corresponding to the key component image are extracted, and a geometric-texture dual guiding factor is constructed. The geometric-texture dual guiding factor is used to perform defect feature adaptive enhancement processing on the key component image to obtain a defect-enhanced key component image.

2. The method according to claim 1, characterized in that, The point cloud data is preprocessed, specifically including: A statistical outlier removal algorithm is used to calculate the average distance from each point in the point cloud data to its nearest neighbors, and to remove noise points in the point cloud data whose average distance exceeds the preset Gaussian distribution range. The denoised point cloud data is filtered and downsampled using a voxel grid downsampling algorithm, and then inter-frame registration and fusion are performed on the filtered and downsampled point cloud data to obtain the processed point cloud data.

3. The method according to claim 1, characterized in that, Preprocessing the visible light image data specifically includes: The Brown-Conrady distortion model is used to perform distortion correction processing on the visible light image data using pre-calibrated camera intrinsic parameters and distortion coefficients; A contrast-limited adaptive histogram equalization algorithm is used to perform contrast enhancement processing on the corrected visible light image data in order to improve the overall contrast of the visible light image data. Inter-frame deblurring is performed on the visible light image data after contrast enhancement processing to optimize the image texture details of the visible light image data.

4. The method according to claim 1, characterized in that, The method involves performing clearance analysis on the power distribution network area based on the digital twin model of the distribution network to identify power conductors and obstacles in the power distribution network area. A multi-dimensional dynamic clearance threshold system is constructed by combining environmental perception data, obstacle semantic features, and distribution network voltage levels to calculate the safe clearance distance between the power conductors and the obstacles. Specifically, this includes: Based on the elevation feature values ​​of the normalized digital surface model and the semantic model of the power distribution equipment, a sliding window and semantic segmentation joint algorithm is used to extract the conductor candidate region in the power distribution network area. Then, a random sampling consistency algorithm with conductor curvature constraints is used to fit and obtain the three-dimensional coordinates of the power conductors in the conductor candidate region. Based on the digital twin model of the power distribution network, the three-dimensional geometric features and semantic features of the obstacles are integrated to identify areas in the power distribution network area with elevation feature values ​​greater than 0 and which are not conductors as obstacles. The obstacles are then semantically classified to obtain the semantic classification results of the obstacles. The semantic classification results include at least one of the following classifications: trees, buildings, and construction machinery. The airspace safety threshold is dynamically adjusted based on the distribution network voltage level, the obstacle semantic classification results, and the environmental perception data, wherein the environmental perception data includes wind speed, temperature, humidity, and icing thickness. Based on the three-dimensional coordinates of the conductor, the surface point coordinates of the obstacle, and the environmental perception data, the minimum three-dimensional Euclidean distance between the power conductor and the obstacle is calculated, and the minimum three-dimensional Euclidean distance is compared with the clearance safety threshold to determine the final clearance safety distance.

5. The method according to claim 1, characterized in that, The process involves risk classification based on the safety clearance distance and the semantic features of the obstacles, removal of non-power obstacles in the power distribution network area, extraction of power poles in the power distribution network area, establishment of topological connections between power conductors and power poles through spatial topology analysis, and construction of a three-dimensional model of the power distribution network equipment containing spatial location information and topological relationship information. Specifically, this includes: A classifier that integrates geometric and semantic features is used to classify surface objects in the normalized digital surface model to distinguish between power equipment objects and non-power obstacle objects. Based on the classification results, and combined with the geometric and semantic features of the power pole, a joint algorithm of morphological dilation operation, contour detection and semantic verification is used to extract the pole contour and calculate the three-dimensional bounding box coordinates of the power pole. Through spatial topology analysis, the power conductors and power poles are spatially associated and semantically bound to establish a topological connection between the conductors and the poles. Based on the airspace safety distance and the semantic features of the non-power obstacles, the non-power obstacles are classified into risk levels, and low-priority non-power obstacles in the power distribution network area are removed according to the risk level. Based on the power equipment objects retained in the power distribution network area, the three-dimensional bounding box coordinates, and the topological connection relationship, a three-dimensional model of the power distribution network equipment is generated. The three-dimensional model of the power distribution network equipment includes the spatial location information and topological relationship information of towers, conductors, and auxiliary components.

6. The method according to claim 1, characterized in that, The construction of the deep learning object detection model based on multimodal feature fusion specifically includes: A point cloud feature projection fusion module is added to the backbone network of the YOLOv8 architecture. The point cloud feature projection fusion module is configured to project the three-dimensional geometric features of the three-dimensional model of the power distribution network equipment onto the two-dimensional image feature layer to achieve multimodal fusion of the three-dimensional geometric features and the two-dimensional texture features. An attention mechanism fusion layer is introduced into the Neck network of the YOLOv8 architecture, wherein the attention mechanism fusion layer is configured to perform attention weighting processing on multimodal fusion features to enhance the feature representation weight of the key component image; A geometric constraint loss term is added to the loss function of the YOLOv8 architecture. The geometric constraint loss term is configured to construct a constraint term based on the deviation of the three-dimensional geometric features and participate in the backpropagation calculation to optimize the model's regression ability and class discrimination ability for the target spatial location.

7. The method according to claim 1, characterized in that, The process of extracting the three-dimensional geometric features and two-dimensional texture features corresponding to the key component image, constructing a geometric-texture dual-guided factor, and using the geometric-texture dual-guided factor to perform adaptive enhancement processing on the key component image to obtain a defect-enhanced key component image specifically includes: The three-dimensional geometric features of the three-dimensional point cloud clusters corresponding to the key component image are extracted, and the two-dimensional texture features of the key component image are extracted. The three-dimensional geometric features include surface normal map, local curvature map, relative height map and geometric roughness map, and the two-dimensional texture features include texture gradient map, edge detection map, gray-level co-occurrence matrix feature map and defect texture sensitivity map. Based on the relative height map and the geometric roughness map, a geometric guidance factor is calculated using a parameter-adaptive Sigmoid function to distinguish the power equipment body area from the background area and to mark geometrically abnormal areas. Based on the texture gradient map and the defect texture sensitivity map, a Gaussian weighted algorithm is used to calculate the texture guiding factor to distinguish defect texture regions from normal texture regions and to mark texture abnormal regions. The geometric guidance factor and the texture guidance factor are weighted and fused to obtain the geometric-texture dual guidance factor. The weight of the weighted fusion is dynamically adjusted according to the lighting conditions or background complexity of the inspection scene. Based on the geometry-texture dual guiding factor, the filter kernel size, enhancement intensity, and texture preservation coefficient of the Retinex image enhancement algorithm are adaptively adjusted. The key component image is then processed to enhance the contrast of the defective texture region, preserve the normal texture region, and suppress the interference of the background region, resulting in the enhanced image of the key component.

8. A power distribution network equipment inspection device, characterized in that, include: The acquisition module is used to simultaneously acquire point cloud data and visible light image data of lidar in the power distribution network area, and to preprocess the point cloud data and the visible light image data. The first construction module is used to construct a digital twin model of the distribution network based on the preprocessed point cloud data and the visible light image data. The digital twin model of the distribution network includes a digital surface model, a digital elevation model, a normalized digital surface model, and a semantic model of the distribution network equipment. The calculation module is used to perform clearance analysis on the power distribution network area based on the power distribution network digital twin model to identify power conductors and obstacles in the power distribution network area. It combines environmental perception data, obstacle semantic features and distribution network voltage level to construct a multi-dimensional dynamic clearance threshold system to calculate the clearance safety distance between the power conductors and the obstacles. The second construction module is used to classify risks based on the clearance safety distance and the semantic features of the obstacles, remove non-power obstacles in the power distribution network area, extract the power poles in the power distribution network area and establish the topological connection relationship between the power conductors and the power poles through spatial topological relationship analysis, and construct a three-dimensional model of the power distribution network equipment containing the spatial location information and topological relationship information of the power equipment. The identification module is used to reverse project the spatial location information of the power equipment in the three-dimensional model of the power distribution network equipment onto the visible light image data, determine the image bounding box of the key components, and use a pre-trained deep learning target detection model with multimodal feature fusion to identify the key component images within the image bounding box. The deep learning target detection model with multimodal feature fusion is a YOLOv8 detection model that fuses three-dimensional geometric features and two-dimensional texture features. An enhancement module is used to extract the three-dimensional geometric features and two-dimensional texture features corresponding to the key component image, construct a geometric-texture dual guiding factor, and use the geometric-texture dual guiding factor to perform defect feature adaptive enhancement processing on the key component image to obtain a defect-enhanced key component image.

9. An electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images