An artificial intelligence-based power distribution network line tree barrier identification method and system

Abstract

This invention relates to an artificial intelligence-based method and system for identifying tree obstacles in power distribution networks. The method improves the quality and applicability of the preprocessed dataset through spatiotemporal registration and generative adversarial networks (GANs) data augmentation. It utilizes a dual-branch tree obstacle identification model built based on deep learning algorithms. On one hand, semantic segmentation accurately locates tree areas and power line areas; on the other hand, 3D feature extraction deeply mines the spatial features of both. This dual-branch synergy significantly improves the accuracy of target area identification and reduces the possibility of false positives or false negatives. Finally, a warning is generated by calculating the minimum clearance distance between tree areas and power line areas, quantitatively assessing tree obstacle risk and making the identification results more objective and scientific, further ensuring the accuracy of tree obstacle identification. Compared with existing technologies, this invention, by integrating multi-model deep learning algorithms, can improve the accuracy of tree obstacle identification and the precision of risk warnings.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of power distribution network line operation and maintenance, and in particular to a power distribution network line tree barrier identification method and system based on artificial intelligence. BACKGROUND

[0002] Tree barriers refer to trees in the transmission line corridor whose height, branch extension range, and other factors cause the distance to the line facilities to be less than the safety threshold, which may cause line short circuit, discharge, trip, and other faults, and are one of the important factors threatening the safe and stable operation of the power distribution network. With the expansion of the coverage of the power distribution network and the changes in the ecological environment, the growth trend of trees in the line corridor is complex, and the dynamic monitoring and accurate identification of tree barrier risks have become a key requirement for power grid operation and maintenance.

[0003] Existing tree barrier identification methods mainly include manual inspection and automated identification. Manual inspection relies on field observation by operation and maintenance personnel or measurement with simple tools, which is not only inefficient and costly, but also limited by environmental factors such as terrain, weather, and vegetation obstruction, making it difficult to achieve large-scale and high-frequency monitoring, and prone to missed or incorrect judgments. To improve efficiency, automated identification methods have been gradually applied, but they still have significant shortcomings in accuracy, especially in technical solutions that rely on a single data or a single model:

[0004] On the one hand, a single data type cannot balance semantic information and spatial features. Some methods only rely on two-dimensional data such as visible light images and infrared images, which can preliminarily identify trees and lines through texture, color, and other semantic features. However, the image details are easily blurred due to factors such as light, weather, and season, resulting in low semantic segmentation accuracy of trees and line components, and making it difficult to distinguish the subtle differences between tree branches and line conductors. Another part of the method only uses three-dimensional point cloud data, which can reflect the spatial shape and positional relationship of the target. However, point cloud data lacks semantic labels, making it difficult to accurately distinguish trees from other objects, and data loss may occur during point cloud collection due to obstruction and insufficient device accuracy, resulting in blurred boundaries between tree and line areas.

[0005] On the other hand, a single model cannot realize the synergistic fusion of multi-modal features. Existing automated methods mostly use a single deep learning model, which is difficult to simultaneously utilize the complementary advantages of different data. For example, a CNN-based model can extract image semantic features, but it cannot capture the three-dimensional spatial distance relationship between trees and lines, and it is easy to misjudge the trees and lines that overlap on the plane but are actually far apart as tree barriers; a PointNet-based model can process the spatial features of point clouds, but it lacks the guidance of image semantic information, making it difficult to accurately locate the key parts of trees and sensitive points of lines, leading to regional division deviation, and thus affecting the accuracy of tree barrier early warning. For example, Chinese patent CN120218632B discloses a tree barrier intelligent detection method and system based on multi-modal perception, which uses a UAV to carry multi-modal sensors to collect data, uses multi-spectral data fusion and deep learning algorithms to construct a three-dimensional tree model, and calculates the minimum safety clearance distance between power lines and trees in real time, and performs risk assessment, but its accuracy is not high, the lines and trees are processed independently, and the original accuracy of the point cloud is relied on, which is easily affected by occlusion, and it is easy to misjudge the trees or lines that overlap on the plane but are actually far apart as risks. SUMMARY

[0006] The purpose of the present application is to overcome the defects of the prior art and provide a power distribution network line tree barrier identification method and system based on artificial intelligence, which solves the technical problem of insufficient accuracy and easy misjudgment or omission caused by relying on single data or single model in the prior art.

[0007] The purpose of the present application can be achieved by the following technical solutions:

[0008] A power distribution network line tree barrier identification method based on artificial intelligence, the method comprising:

[0009] Collecting multi-modal data of the power transmission line corridor by a UAV, including three-dimensional point cloud data and visible light images;

[0010] Performing spatio-temporal registration on the collected multi-modal data, and enhancing the registered multi-modal data by a generative adversarial network to obtain a preprocessed data set;

[0011] Using a double-branch tree barrier identification model to perform semantic segmentation and three-dimensional feature extraction on the preprocessed data set to obtain a tree region and a line region; the double-branch tree barrier identification model is pre-constructed based on a deep learning algorithm;

[0012] Calculating the minimum clearance distance of the tree region and the line region, and generating a tree barrier risk warning when the minimum clearance distance is less than zero.

[0013] Further, the process of spatio-temporal registration on the collected multi-modal data comprises:

[0014] Timestamp synchronization is performed on three-dimensional point cloud data in the multi-modal data, and an interpolation compensation algorithm is used to eliminate the acquisition delay of the sensor;

[0015] Two-dimensional feature points of the visible light image and geometric feature points of the three-dimensional point cloud are extracted;

[0016] The two-dimensional feature points and the geometric feature points are matched based on a spatial constraint condition, and a feature point pair whose projection error exceeds a preset distance threshold is removed, to obtain registered multi-modal data; wherein the registered multi-modal data includes a registered image, a registered point cloud, and a mapping matrix of the registered image and the registered point cloud;

[0017] The registered multi-modal data is converted to a unified geographic coordinate system.

[0018] Further, the generative adversarial network includes a Pix2PixHD network and a PointGAN network, and the process of enhancing the registered multi-modal data by the generative adversarial network includes:

[0019] The registered image is processed by the Pix2PixHD network to obtain enhanced image data;

[0020] The registered point cloud is processed by the PointGAN network to complete the missing area of the registered point cloud, to obtain enhanced point cloud data;

[0021] The preprocessed data set includes the enhanced image data and the enhanced point cloud data.

[0022] Further, the dual-branch tree barrier identification model includes a first branch, a second branch, a cross-modal guidance module, a feature fusion module, and a tree barrier region generation module; wherein,

[0023] The first branch is used for feature extraction on the enhanced visible light image data, and outputs two-dimensional semantic masks of trees and lines and their confidence; the lines include but are not limited to wires, insulators, and towers;

[0024] The cross-modal guidance module is used for mapping the two-dimensional semantic masks output by the first branch to the three-dimensional point cloud space according to the mapping matrix, to obtain three-dimensional attention masks and three-dimensional attention weights of the trees and the lines, and to guide the second branch to focus on three-dimensional feature extraction of the trees and the lines by using the three-dimensional attention masks;

[0025] The second branch is used for processing the enhanced point cloud data according to the three-dimensional attention masks, and capturing the spatial form and position relationship of the trees and the lines by using the three-dimensional attention weights, to output three-dimensional geometric features of the trees and the lines;

[0026] The feature fusion module is configured to perform dimensional splicing and feature reorganization on the high-level semantic features of the first branch and the three-dimensional geometric features of the second branch to generate cross-modal fusion features with fused semantic information and spatial information; the high-level semantic features of the first branch are features obtained after mapping matrix conversion of the last feature map before output of a two-dimensional semantic mask to a three-dimensional point cloud space;

[0027] The tree barrier region generation module is configured to perform point cloud clustering and category determination based on the cross-modal fusion features, divide three-dimensional space regions of trees and lines respectively, and remove noise points in the three-dimensional space regions through boundary optimization to output tree region and line region.

[0028] Further, the process of the cross-modal guidance module obtaining three-dimensional attention masks and three-dimensional attention weights of trees and lines includes:

[0029] obtaining a mapping matrix M;

[0030] a two-dimensional semantic mask output from the first branch extracting a target pixel coordinate set from the two-dimensional semantic mask satisfying:

[0031] ;

[0032] wherein, a pixel with a coordinate of (x, y) in the two-dimensional semantic mask belongs to a category c, and c=0 represents a tree category and c=1 represents a line category;

[0033] converting each point in the target pixel coordinate set to a three-dimensional point cloud coordinate through the mapping matrix M; the three-dimensional point cloud coordinate satisfies:

[0034] ;

[0035] based on the three-dimensional point cloud coordinates, marking points belonging to in the registration point cloud as 1 and otherwise as 0 to generate a three-dimensional attention mask ;

[0036] calculating a three-dimensional attention weight based on a confidence of the two-dimensional semantic mask and a three-dimensional Euclidean distance; the three-dimensional Euclidean distance is calculated based on three-dimensional point coordinates.

[0037] Further, the process of the second branch outputting three-dimensional geometric features of trees and lines includes:

[0038] ​Downsample and normalize the enhanced point cloud data to obtain a simplified point cloud feature

[0039] Through a three-dimensional attention mask Filter a target point set from the simplified point cloud feature ;

[0040] Point-by-point multiply the target point set and the three-dimensional attention weight to obtain a weighted point cloud ; the weighted point cloud includes the spatial form and positional relationship of trees and lines;

[0041] Extract features of the weighted point cloud using a pre-built PointNet++ network to obtain three-dimensional geometric features of trees and lines , including:

[0042] Output local features through a Set Abstraction layer in the pre-built PointNet++ network;

[0043] Maximum pool the local features to obtain global features;

[0044] Concatenate the local features with the broadcasted global features to obtain three-dimensional geometric features of trees and lines .

[0045] Further, the process of the feature fusion module generating cross-modal fusion features that fuse semantic information and spatial information includes:

[0046] Convert the high-level semantic features of the first branch to a three-dimensional point cloud space through a mapping matrix M to obtain three-dimensional semantic features aligned with the simplified point cloud features ;

[0047] Align the point number and feature dimension of the three-dimensional geometric features and the three-dimensional semantic features through interpolation or 1x1 convolution;

[0048] Concatenate the aligned features along the feature dimension to obtain concatenated features ;

[0049] Nonlinearly transform the concatenated features through a multi-layer perceptron (MLP) layer with a ReLU activation function to output cross-modal fusion features .

[0050] Further, the process of calculating the minimum clearance distance of the tree area and the line area includes:

[0051] reserve a tree area within a preset distance range of a line three-dimensional model, and obtain a tree barrier area; the line three-dimensional model is constructed based on outputs of a first branch and a second branch;

[0052] perform hierarchical clustering on trees in the tree barrier area, and obtain a plurality of tree key parts;

[0053] extract a sag lowest point and an insulator installation point from the line three-dimensional model as a reference point set;

[0054] calculate three-dimensional Euclidean distances between the tree key parts and the reference point set by using a K-nearest neighbor search algorithm, and obtain a static minimum distance ;

[0055] calculate a maximum swing amount of a conductor based on historical meteorological data;

[0056] calculate a flexibility coefficient based on a tree branch diameter, and multiply the flexibility coefficient by an average maximum wind speed v in a preset period to obtain a maximum swing amount of a branch; the tree branch diameter is obtained by extracting geometric features of a three-dimensional point cloud of the tree area;

[0057] calculate a sum of the maximum swing amount of the conductor and the maximum swing amount of the branch to obtain a dynamic maximum distance;

[0058] subtract the dynamic maximum distance from the static minimum distance to obtain a minimum clearance distance.

[0059] Further, the construction process of the line three-dimensional model comprises:

[0060] bind a position label of a line in a two-dimensional semantic mask output by the first branch to the line area through a mapping matrix to obtain a semantic enhanced line point cloud;

[0061] perform surface fitting on the semantic enhanced line point cloud by using a Poisson surface reconstruction algorithm to obtain an initial reconstruction model;

[0062] match stable feature points of a tower and a pole by using an iterative closest point algorithm based on multi-frame time sequence data of a drone, and calculate a transformation matrix of adjacent frame line point clouds; the stable feature points include a tower and pole fixed point, a tower and pole connection node, and an insulator installation point;

[0063] perform time sequence alignment on the initial reconstruction model based on the transformation matrix to obtain a line model after time sequence correction;

[0064] if a conductor sag, a tower spacing and a design value of the line in the line model after time sequence correction deviate from a preset threshold value, adjust a point cloud weight of the line model after time sequence correction by using a Bayesian optimization algorithm to obtain a final line three-dimensional model; the design value of the line is obtained from a power distribution network line design drawing.

[0065] An artificial intelligence-based power distribution network line tree barrier identification system, comprising a data acquisition module, a data processing module and a risk warning module; wherein,

[0066] The data acquisition module is used for collecting multi-modal data of a power transmission line corridor by a drone, including three-dimensional point cloud data and visible light images;

[0067] The data processing module is used for spatio-temporal registration of the collected multi-modal data, and enhancing the registered multi-modal data by a generative adversarial network to obtain a preprocessed data set;

[0068] A dual-branch tree barrier identification model based on a deep learning algorithm is used for semantic segmentation and three-dimensional feature extraction of the preprocessed data set to obtain a tree region and a line region;

[0069] The risk warning module is used for calculating the minimum clearance distance of the tree region and the line region, and generating a tree barrier risk warning when the minimum clearance distance is less than zero.

[0070] Compared with the prior art, the beneficial effects of the present application include:

[0071] 1. The present application provides an artificial intelligence-based power distribution network line tree barrier identification method and system, which can improve the accuracy of tree barrier identification and the accuracy of risk warning by fusing multi-model deep learning algorithms; the quality and applicability of the preprocessed data set are improved through spatio-temporal registration and data enhancement of the generative adversarial network; the dual-branch tree barrier identification model based on the deep learning algorithm is used to accurately locate the tree region and the line region through semantic segmentation, and to deeply mine the spatial features of the two regions through three-dimensional feature extraction; the dual-branch collaborative action greatly improves the accuracy of target region identification and reduces the possibility of misjudgment or omission; finally, the minimum clearance distance between the tree region and the line region is calculated to generate a warning, and the tree barrier risk is judged in a quantitative manner, making the identification result more objective and scientific, and further ensuring the accuracy of tree barrier identification.

[0072] 2. In the data preprocessing stage, the spatio-temporal registration through timestamp synchronization and interpolation compensation can eliminate sensor delay, and then the consistency of multi-modal data in the spatio-temporal dimension can be ensured through feature point matching and error elimination.

[0073] 3. In the data enhancement stage, the Pix2PixHD network is used to enhance the image data quality, and the PointGAN network is used to complete the missing area of the point cloud, which can make up for the shortcomings in the data acquisition process and improve the integrity and reliability of the preprocessed data set.

[0074] 4. In the tree barrier identification stage, the double-branch tree barrier identification model is used to map the two-dimensional semantic mask to the three-dimensional space through the cross-modal guidance module to generate attention masks and weights, guide the second branch to focus on the three-dimensional feature extraction of trees and lines, and simultaneously fuse high-level semantic features and three-dimensional geometric features to generate cross-modal fusion features, which are accurately divided into target regions through clustering and boundary optimization, thereby reducing the misjudgment and omission risk of trees and lines.

[0075] 5. In the minimum clearance distance calculation, the effective tree barrier area is reserved, the key parts of trees are hierarchically clustered, the reference points of lines are extracted, the K nearest neighbor algorithm is used to calculate the static distance, and the dynamic swing amount of the conductor and branches is introduced, thereby ensuring the accuracy of tree barrier identification and realizing dynamic quantitative evaluation of tree barrier risk, making the risk judgment more in line with the actual working condition, and providing more reliable protection for the safe operation of the distribution network line. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 A flowchart of a power distribution network line tree barrier identification method based on artificial intelligence provided by the embodiment of the present application is shown.

[0077] Figure 2 A system architecture diagram of a power distribution network line tree barrier identification system based on artificial intelligence provided by the embodiment of the present application is shown.

[0078] Figure 3 A flowchart of spatiotemporal registration of collected multi-modal data provided by the embodiment of the present application is shown.

[0079] Figure 4 A flowchart of semantic segmentation and three-dimensional feature extraction of preprocessed data set using a double-branch tree barrier identification model to obtain tree and line regions provided by the embodiment of the present application is shown.

[0080] Figure 5 A structural diagram of a power distribution network line tree barrier identification device based on artificial intelligence provided by the embodiment of the present application is shown.

[0081] Figure 6 A hardware structure diagram of a power distribution network line tree barrier identification device based on artificial intelligence provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0082] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0083] Embodiment 1

[0084] The embodiment discloses an artificial intelligence-based power distribution network line tree barrier identification method, and the method specifically comprises the steps of Figure 1 as shown in the figure, comprising:

[0085] In step S1, the unmanned aerial vehicle collects multi-modal data of the power transmission line corridor, including three-dimensional point cloud data and visible light images, performs time-space registration on the collected multi-modal data, and enhances the multi-modal data after registration through a generative adversarial network to obtain a preprocessed data set.

[0086] The power transmission line corridor refers to a strip-shaped area passed by the power transmission line, covering line facilities such as conductors, towers, and insulators, and environmental elements such as surrounding trees and terrain. The multi-modal data set can include visible light images, infrared images, three-dimensional point clouds, etc., which can reflect the characteristics of the line and trees from different dimensions such as optics, thermotics, and spatial structure.

[0087] In some implementations, the time-space registration method of multi-modal data includes two types of time synchronization and space alignment. Specifically, time synchronization can associate different modal data to the same time node by synchronizing the time stamps of each sensor; space alignment can use a feature point matching algorithm (such as Scale-Invariant Feature Transform, SIFT) to extract key feature points in images and point clouds and establish a corresponding relationship, and map the data to the same spatial coordinate system.

[0088] In step S1, the core of time-space registration is to eliminate the deviation of multi-modal data in time and space, ensure the accurate association of different types of data, including time stamp synchronization, feature point extraction and matching, coordinate unification, etc., to ensure that different types of data can be associated and complementary information in subsequent processing.

[0089] For the visible light image and three-dimensional point cloud collected by the unmanned aerial vehicle simultaneously, the image and the point cloud at the same flight time are first bound through the time stamp, and then the edge feature points in the image and the geometric feature points in the point cloud are extracted using the SIFT algorithm. The image pixel position and the point cloud spatial coordinates are associated by matching these feature points to complete the time-space registration.

[0090] In some implementations, when performing time stamp synchronization on three-dimensional point cloud data, the interpolation compensation algorithm can use linear interpolation or cubic spline interpolation, and the specific selection is based on the data sampling frequency (such as using linear interpolation when the sampling frequency is 10Hz); the determination of the pre-set distance threshold needs to be combined with the sensor accuracy. For example, when the visible light camera carried by the unmanned aerial vehicle has an accuracy of 0.05 meters and the laser radar has an accuracy of 0.1 meters, the threshold can be set to 1.5 times the sum of the accuracies, i.e. 0.225 meters.

[0091] The feature point matching needs to be based on spatial constraints (such as collinearity, coplanar constraint) to improve the matching accuracy; the mapping matrix after registration is used to establish the spatial correspondence between the registered image and the registered point cloud, laying a foundation for subsequent cross-modal processing.

[0092] For example, the timestamp of the three-dimensional point cloud data in the multi-modal data has a 0.5-second delay from the visible light image, and the delay is eliminated by a linear interpolation compensation algorithm; the Scale-Invariant Feature Transform (SIFT) two-dimensional feature points of the visible light image and the geometric feature points (such as vertex, edge point) of the three-dimensional point cloud are extracted, after matching based on spatial constraints, the feature point pairs with a projection error exceeding 0.2 meters (preset threshold) are removed, and finally the registered data is converted to the WGS84 geographic coordinate system.

[0093] The specific steps of the above step S1 of performing spatio-temporal registration on the collected multi-modal data are as shown in Figure 3 The specific steps of the above step S1 of performing spatio-temporal registration on the collected multi-modal data are as shown in

[0094] S101.1, performing timestamp synchronization on the three-dimensional point cloud data in the multi-modal data, and using an interpolation compensation algorithm to eliminate the acquisition delay of the sensor.

[0095] The timestamp synchronization is to align the acquisition time of the three-dimensional point cloud data with other modal data (such as visible light image), and the interpolation compensation algorithm is used to correct the time deviation caused by the difference in response speed of the sensor.

[0096] In some implementations, the timestamp synchronization can be performed by obtaining the original timestamps of the three-dimensional point cloud sensor and the image sensor, converting the timestamps of all data to the same time scale based on a high-precision clock (such as GPS time); the interpolation compensation algorithm can use linear interpolation, and when the time interval of two adjacent sampling points is less than a preset threshold (such as 0.1 seconds), the data at the intermediate time is supplemented by linear fitting to ensure the continuity of the time sequence.

[0097] The sensor acquisition delay is mainly caused by the difference in triggering mechanism of different devices, and the interpolation compensation needs to be based on the time correlation of the data to avoid introducing additional errors.

[0098] For example, the sampling time of the three-dimensional point cloud data is t1 and t3, and the acquisition time of the visible light image is t2 (t1 < t2 < t3), the point cloud data at time t2 is calculated by linear interpolation to make the point cloud and the image correspond accurately in time, and the 0.05-second acquisition delay is eliminated.

[0099] S101.2, extracting two-dimensional feature points of the visible light image and geometric feature points of the three-dimensional point cloud.

[0100] Two-dimensional feature points are pixel points with recognition in visible light images (such as edge intersection points and texture mutation points), and three-dimensional geometric feature points are points reflecting spatial structure in a three-dimensional point cloud (such as vertex points and curvature extreme points), which provide a basis for subsequent spatial matching.

[0101] In some implementations, two-dimensional feature points can be extracted by a Scale-Invariant Feature Transform (SIFT) algorithm, which can maintain feature stability under scale and rotation changes, and three-dimensional geometric feature points can be detected by an improved ISS (Intrinsic Shape Signatures) feature point detection algorithm, which filters out representative geometric feature points by calculating the curvature and neighborhood distribution of the point cloud.

[0102] The extraction quality of feature points directly affects the registration accuracy, and the number of feature points and recognition need to be balanced to avoid excessive redundant points or missing key features.

[0103] S101.3, based on the spatial constraint condition, matching the two-dimensional feature points and the geometric feature points, and eliminating the feature point pairs whose projection error exceeds the preset distance threshold after matching, to obtain the registered multi-modal data; wherein the registered multi-modal data includes a registered image, a registered point cloud, and a mapping matrix of the registered image and the registered point cloud.

[0104] The spatial constraint condition includes collinear constraint and coplanar constraint, which is used to ensure that the two-dimensional and three-dimensional feature points have a true corresponding relationship in the physical space; the projection error refers to the pixel distance between the three-dimensional feature points projected to the image plane and the corresponding two-dimensional feature points, and the preset distance threshold is used to screen effective matching pairs.

[0105] In some implementations, the spatial constraint condition can construct a projection relationship through camera intrinsic and extrinsic parameters, project the three-dimensional geometric feature points to the image plane, and preliminarily match them with the two-dimensional feature points; the determination of the preset distance threshold needs to be combined with the image resolution, for example, when the image resolution is 1920x1080 pixels (pixel size 0.01 meters / pixel), the threshold can be set to the actual distance (0.03 meters) corresponding to 3 pixels.

[0106] The matching of feature point pairs needs to eliminate outliers through a Random Sample Consensus (RANSAC) algorithm to ensure the robustness of the matching result.

[0107] Exemplarily, based on the camera parameters, the three-dimensional geometric feature points are projected to the image, the Euclidean distance calculation is performed on the two-dimensional feature points, the matching pairs with a distance less than 0.02 meters (2 pixels) are reserved, the RANSAC algorithm is further used to remove 30 abnormal point pairs with a projection error greater than 0.03 meters, and finally 200 effective feature point pairs are obtained to form a registration image, a registration point cloud and a mapping matrix of the two.

[0108] S101.4, converting the registered multi-modal data to a unified geographic coordinate system.

[0109] The unified geographic coordinate system provides a global spatial reference for the data, facilitating subsequent cross-regional tree barrier analysis and comparison. Common coordinate systems include the WGS84 coordinate system and the national 2000 geodetic coordinate system.

[0110] In some implementations, the conversion process needs to use the GPS positioning information and attitude data (such as heading angle, pitch angle) of the unmanned aerial vehicle to convert the registration data from the local coordinate system (such as the unmanned aerial vehicle body coordinate system) to the target geographic coordinate system through coordinate conversion formula (such as the Bursa model); the conversion parameters can be calibrated by field control points to ensure the conversion accuracy (such as the plane position error less than 0.5 meters).

[0111] It should be noted that the unified coordinate system is a prerequisite for multi-segment route data splicing and long-term monitoring data comparison, and the conversion accuracy needs to meet the distance calculation requirements of tree barrier identification.

[0112] Exemplarily, the registered multi-modal data is converted from the local coordinate system of the unmanned aerial vehicle to the WGS84 coordinate system, and the conversion parameters are calibrated by three known GPS control points. The deviation of the converted point cloud data from the actual position is less than 0.3 meters, meeting the accuracy requirements of subsequent clearance distance calculation.

[0113] Based on the above technical solutions, the time and space registration eliminates the time deviation of the data through time synchronization, establishes the spatial correlation of two-dimensional and three-dimensional data through feature point extraction and matching, and realizes the accurate fusion of multi-modal data by combining coordinate unification, providing a high-quality data basis for subsequent data enhancement and tree barrier identification.

[0114] The specific steps of enhancing the registered multi-modal data by the generative adversarial network in the above step S1 include:

[0115] Step S102.1, processing the registration image by using the Pix2PixHD network to obtain enhanced image data;

[0116] Step S102.2, processing the registration point cloud by using the PointGAN network to complete the missing area of the registration point cloud, and obtaining enhanced point cloud data.

[0117] Step S102.3, the generative adversarial network includes a Pix2PixHD network and a PointGAN network, which are respectively used for processing the registered image and the registered point cloud.

[0118] The Pix2PixHD network is a high-resolution image generation model based on a conditional generative adversarial network, which is good at processing fine generation and enhancement of complex scene images, and can improve the clarity of local details while preserving the overall structure of the image. The PointGAN network is a generative adversarial network for three-dimensional point cloud data, which can learn the spatial distribution characteristics of the point cloud, realize point cloud completion and optimization, and is suitable for irregular and unstructured three-dimensional data processing.

[0119] In some implementations, the pre-training process and the application process of the Pix2PixHD network can include the following steps:

[0120] During training, registered image data of power line corridors in multiple scenes is collected, covering different light, weather and seasonal conditions, and is preprocessed to a uniform size; then paired samples of "low-quality images-high-quality reference images" are constructed (without manual labeling of missing details), the generator learns the mapping from low-quality to high-quality, and the discriminator distinguishes between generated images and real high-quality images, and the two are trained in an adversarial manner until the generator can output enhanced images close to the real ones.

[0121] Then, during model application, the registered line corridor image is input into the trained network, the model repairs the fuzzy and light defect areas, enhances the texture details of the conductor and tree branches, and outputs enhanced images with improved quality.

[0122] In some implementations, the pre-training process and the application process of the PointGAN network can include the following steps:

[0123] During training, complete three-dimensional point clouds of power line corridors are collected, and after preprocessing, the missing areas are marked by labeling to construct paired samples of "incomplete point clouds-complete point clouds"; the generator generates missing area point clouds from incomplete point clouds, and the discriminator judges whether the generated results conform to the real distribution, and after adversarial training, the generator can complete the point clouds that conform to the physical law.

[0124] During application, the registered three-dimensional point cloud is input into the model, the network automatically identifies sparse or missing areas, and based on the learned distribution characteristics, the data is completed, so that the point cloud structure of the tower, trees and other targets is more complete.

[0125] The role of data enhancement using the Pix2PixHD and PointGAN models is:

[0126] The former solves the quality problem of the registered image caused by the limitation of the acquisition condition by improving the image detail definition, and provides more distinct target features for subsequent semantic segmentation; the latter makes up for the information loss caused by occlusion and device limitation in the point cloud acquisition process by completing the missing area of the point cloud, and provides more comprehensive spatial feature support for three-dimensional feature extraction. The two work together to improve the quality and integrity of the preprocessed data set, and lay a solid foundation for the accuracy of subsequent tree barrier identification.

[0127] It should be noted that image enhancement can improve texture detail definition, and point cloud completion can make up for information loss caused by occlusion and insufficient device precision, and both can improve data quality.

[0128] Based on the above technical solutions, step S1 eliminates the space-time deviation of multi-modal data through fine space-time registration, and combines the generative adversarial network to enhance the image and point cloud data, effectively improving the consistency, integrity and quality of the preprocessed data set, and providing high-quality data support for subsequent tree barrier identification.

[0129] In step S2, a dual-branch tree barrier identification model is used to perform semantic segmentation and three-dimensional feature extraction on the preprocessed data set to obtain the tree region and the line region.

[0130] The dual-branch tree barrier identification model is based on a deep learning algorithm, one branch is used to process image data (such as visible light images and infrared images), and the other branch is used to process three-dimensional point cloud data. The first branch identifies the two-dimensional contours of trees and lines through semantic segmentation, and the second branch obtains the spatial position and shape information of trees and lines through three-dimensional feature extraction. The basic architecture can include a convolutional neural network (CNN) for feature extraction of the image branch, and a point cloud neural network (such as PointNet) for spatial feature learning of the point cloud branch. The two branches combine the output results through a feature fusion layer. Finally, the output tree region and line region are both three-dimensional space regions, i.e. the spatial range occupied by trees and lines respectively in a three-dimensional coordinate system.

[0131] In some implementations, the training of the dual-branch model can use multi-modal sample data labeled with tree and line positions, and the model parameters are optimized through a backpropagation algorithm, so that the model gradually learns the feature differences between trees and lines, and improves the accuracy of segmentation and extraction.

[0132] The advantage of the dual-branch structure is that it simultaneously utilizes the texture semantic information of images and the spatial structure information of point clouds, avoiding the limitations of single modal data, thereby more accurately dividing the target region.

[0133] Exemplarily, the image branch performs convolution, pooling and other operations on the enhanced visible light image through a CNN to output two-dimensional semantic masks of trees and lines; the point cloud branch encodes features of the enhanced three-dimensional point cloud through a PointNet to extract spatial coordinates and morphological features; after the features of the two are fused, the model can output boundary coordinates of the trees and lines in the three-dimensional space to form corresponding regions.

[0134] Specifically, the dual-branch tree barrier identification model includes a first branch, a second branch, a cross-modal guidance module, a feature fusion module and a tree barrier region generation module.

[0135] In the above step S2, the dual-branch tree barrier identification model is used to perform semantic segmentation and three-dimensional feature extraction on the preprocessed data set to obtain specific steps of tree region and line region as Figure 4 as shown, comprising:

[0136] S201, processing the enhanced image data using the first branch to output two-dimensional semantic masks of trees and lines.

[0137] The first branch can be constructed based on a convolutional neural network (CNN) and is used to extract semantic features from the enhanced image data to locate two-dimensional regions of trees and lines through semantic segmentation; the output two-dimensional semantic mask is a pixel-level classification result, each pixel is labeled as a tree (class 0) or a line (class 1, including conductor, insulator, tower, etc.), which provides prior semantic information for subsequent three-dimensional feature extraction.

[0138] In some implementations, the first branch can use a U-Net or DeepLab series network, the input of which is the enhanced image, and the output two-dimensional semantic mask has the same resolution as the input image; during network training, a cross-entropy loss function is used to optimize parameters, and the label is the manually labeled tree and line pixel region.

[0139] The core role of the first branch is to quickly locate the target region using the texture, color and other semantic information of the image to provide a "region of interest" guide for three-dimensional processing and reduce the interference of irrelevant regions.

[0140] Exemplarily, the enhanced visible light image contains towers, conductors and trees, the first branch outputs a two-dimensional semantic mask through multi-layer convolution and deconvolution operations: the tower and conductor regions in the mask are labeled as 1, the tree region is labeled as 0, and the background region is labeled as 2, and the background region does not participate in subsequent processing.

[0141] S202, mapping the two-dimensional semantic mask to the three-dimensional point cloud space through the cross-modal guidance module to generate a three-dimensional attention mask and a three-dimensional attention weight.

[0142] The cross-modal guidance module is used to realize the association of two-dimensional and three-dimensional features based on the mapping matrix M (obtained by the spatio-temporal registration in S1) of the registered image and the registered point cloud; the three-dimensional attention mask is used to screen the target points belonging to trees and lines in the point cloud, and the three-dimensional attention weight is used to quantify the importance of the target points in the point cloud.

[0143] In some implementations, the mapping matrix M is calculated by the camera intrinsic parameters (focal length, principal point coordinates) and extrinsic parameters (rotation matrix, translation vector), which can be obtained by a camera calibration tool (such as Zhang Zhengyou calibration method).

[0144] The process of the cross-modal guidance module obtaining the three-dimensional attention mask and the three-dimensional attention weight of trees and lines is specifically as follows:

[0145] Obtaining the mapping matrix M;

[0146] The two-dimensional semantic mask output from the first branch extracting a target pixel coordinate set ; the target pixel coordinates are combined satisfy:

[0147] ;

[0148] wherein, indicates that the pixel with the coordinate in the two-dimensional semantic mask belongs to the class c, and c=0 indicates the tree class and c=1 indicates the line class;

[0149] Each point in the target pixel coordinate set is converted into a three-dimensional point cloud coordinate by the mapping matrix M; the three-dimensional point cloud coordinate satisfies:

[0150] ;

[0151] According to the three-dimensional point cloud coordinates, the points belonging to in the registered point cloud are marked as 1, otherwise as 0, to generate a three-dimensional attention mask ;

[0152] The three-dimensional attention weight is calculated based on the confidence of the two-dimensional semantic mask and the three-dimensional Euclidean distance ; the calculation formula is:

[0153]

[0154] wherein, indicates the classification confidence of the two-dimensional pixel in the two-dimensional semantic mask, indicates the three-dimensional Euclidean distance, which is calculated based on the three-dimensional point coordinates, denotes a distance decay coefficient, denotes coordinates of the jth point in the registration point cloud, denotes an attention weight of the jth point in the registration point cloud.

[0155] The three-dimensional Euclidean distance is calculated based on three-dimensional point coordinates.

[0156] Distance decay coefficient in three-dimensional attention weight calculation , which can be determined according to the average density of the point cloud, such as when the average distance of the point cloud is 0.1 meters, is set to 0.3 meters, ensuring that the weight decays reasonably with distance.

[0157] This module is the key to connecting the two-dimensional and three-dimensional branches. By “transferring” image semantic information to the point cloud space, it solves the problem of missing semantic information of point cloud data and guides the three-dimensional feature extraction to focus on the target area.

[0158] S203, using the second branch to process the enhanced point cloud data according to the three-dimensional attention mask and weight, and outputting the three-dimensional geometric features of trees and lines.

[0159] The second branch can be constructed based on a point cloud neural network (such as PointNet++), which is used to process the enhanced point cloud data according to the three-dimensional attention mask, and capture the spatial form and positional relationship of trees and lines through the three-dimensional attention weight, and output the three-dimensional geometric features of trees and lines; the three-dimensional geometric features can include the curvature, neighborhood distribution and other spatial information of the point cloud, to reflect the three-dimensional structure of trees and lines.

[0160] The process of the second branch outputting the three-dimensional geometric features of trees and lines specifically includes:

[0161] Downsampling and normalizing the enhanced point cloud data to obtain a simplified point cloud feature;

[0162] Through the three-dimensional attention mask Filtering the target point set from the simplified point cloud feature ;

[0163] Multiplying the target point set with the three-dimensional attention weight point by point to obtain a weighted point cloud ; the weighted point cloud includes the spatial form and positional relationship of trees and lines;

[0164] Using a pre-constructed PointNet++ network to extract the features of the weighted point cloud to obtain the three-dimensional geometric features of trees and lines , including:

[0165] The local features are output by a Set Abstraction layer in a pre-constructed PointNet++ network;

[0166] The local features are maximum-pooled to obtain global features;

[0167] The local features are spliced with the broadcasted global features to obtain three-dimensional geometric features of the trees and the line .

[0168] The second branch focuses on the point cloud area of the trees and the line through an attention mechanism, avoids interference of background point clouds, and simultaneously strengthens feature expression of key areas (such as conductor sag points and tree tops) through a weighting operation, so that the three-dimensional geometric features of the trees and the line can be more accurately extracted, high-quality spatial feature input is provided for subsequent cross-modal feature fusion, and the accuracy of tree area and line area division is improved.

[0169] In S204, the high-level semantic features and the three-dimensional geometric features of the first branch are fused by a feature fusion module to generate cross-modal fusion features.

[0170] The feature fusion module is configured to perform dimension splicing and feature reorganization on the high-level semantic features of the first branch and the three-dimensional geometric features of the second branch to generate cross-modal fusion features that fuse semantic information and spatial information, so as to compensate for the limitations of single-modal features; the cross-modal fusion features contain texture semantics and spatial structure information, and provide more comprehensive feature support for subsequent area division; the high-level semantic features of the first branch are features obtained after the last feature map before outputting a two-dimensional semantic mask is converted to a three-dimensional point cloud space through a mapping matrix;

[0171] The process of generating the cross-modal fusion features that fuse semantic information and spatial information by the feature fusion module specifically includes:

[0172] The high-level semantic features of the first branch are converted to a three-dimensional point cloud space through a mapping matrix M to obtain three-dimensional semantic features aligned with the simplified point cloud features ;

[0173] The point number and feature dimension of the three-dimensional geometric features are aligned with those of the three-dimensional semantic features through interpolation or 1x1 convolution;

[0174] The aligned features are spliced along the feature dimension to obtain spliced features ;

[0175] The spliced features are subjected to nonlinear transformation through a multilayer perceptron (MLP) layer with a ReLU activation function to output cross-modal fusion features .​​​​

[0176]

[0177] wherein W is a weight matrix and b is a bias.

[0178] The core of feature fusion is to complement semantic features and spatial features through dimension alignment and nonlinear transformation, so as to improve the adaptability of the model to complex scenes.

[0179] In S205, the tree barrier region generation module divides the tree region and the line region based on the cross-modal fusion feature.

[0180] The tree barrier region generation module is used to separate the three-dimensional space region of trees and lines from the point cloud by point cloud clustering and category determination, and to remove noise points in the three-dimensional space region by boundary optimization, so as to enhance the boundary definition and region integrity of the tree region and the line region.

[0181] In some implementations, the point cloud clustering can use the DBSCAN algorithm, the neighborhood radius of which is set to 0.3 meters (determined according to 1.5 times the average distance of the point cloud), and the minimum point number MinPts is set to 5 to ensure the stability of clustering; the category determination is realized by a softmax classifier, and the clustered point cloud is divided into a tree category and a line category based on the cross-modal fusion feature; the boundary optimization can use statistical filtering to remove points whose distance from the cluster center is more than 3 times the standard deviation, and the standard deviation is calculated based on the spatial distribution of the clustered point cloud. The core of the tree barrier region generation module is to divide the three-dimensional region of trees and lines based on the cross-modal fusion feature, wherein the DBSCAN algorithm is mainly used for point cloud clustering and belongs to the post-processing step of the model inference stage. In a deep learning model, such an unsupervised clustering algorithm can be embedded as a part of feature processing, and its role is to perform spatial clustering on the cross-modal fusion feature output by the model, divide point clouds with similar features into the same region, and thus separate the three-dimensional space region of trees and lines.

[0182] In the training stage, the training focus of the model is on the parameter optimization of the first branch, the second branch and the feature fusion module, so that the output cross-modal fusion feature can accurately reflect the semantic and spatial differences between trees and lines by learning the feature rules of trees and lines in the sample. The DBSCAN algorithm itself does not participate in the learning of model parameters, and only clusters and divides features in the inference process after the training is completed, so it will not interfere with the gradient calculation and parameter update in the training stage, ensuring the normal progress of the training process.

[0183]

[0184] ​Exemplarily, the cross-modal fusion features are clustered by DBSCAN to obtain 8 point cloud sets, and it is determined by a softmax classifier that 3 of them are tree classes and 5 are line classes; boundary optimization is performed on each point cloud set, and noise points are removed, and finally the tree region and line region are output.

[0185] Based on the above technical solution, S2 cooperates through the collaborative work of the double-branch tree barrier identification model, first uses the first branch to extract image semantic information to locate the target, then associates the three-dimensional point cloud through the cross-modal guidance module, extracts the spatial features combined with the second branch, and finally fuses the multi-modal features to accurately divide the three-dimensional region of trees and lines. The whole process fully gives play to the complementary advantages of image semantic information and point cloud spatial information, greatly improves the accuracy of target region recognition, and lays a reliable spatial positioning foundation for tree barrier risk assessment.

[0186] Step S3, calculate the minimum clearance distance of the tree region and the line region, and generate a tree barrier risk warning when the minimum clearance distance is less than zero.

[0187] The minimum clearance distance represents the shortest straight-line distance between the tree region and the line region, and is a core indicator for measuring the safety distance between the two. If the distance is less than zero, it means that the tree has invaded the safety range of the line.

[0188] In some implementations, the minimum clearance distance is usually determined by calculating the Euclidean distance between the boundary point sets of the two regions, that is, selecting a number of points from the boundary points of the tree region and the line region, calculating the distance between all pairs of points, and taking the minimum value as the result.

[0189] The quantitative calculation of the minimum clearance distance changes the tree barrier risk judgment from qualitative to quantitative, avoiding the subjectivity of manual judgment and improving the accuracy and reliability of the warning.

[0190] Exemplarily, 100 feature points are selected from the boundary points of the tree region, and 100 feature points are selected from the boundary points of the line region, the Euclidean distance between the 10000 pairs of points is calculated, and the minimum distance is the minimum clearance distance. If the value is -0.5 meters (less than zero), the system automatically generates a tree barrier risk warning message.

[0191] In the above step S3, the process of calculating the minimum clearance distance of the tree region and the line region specifically includes:

[0192] S301, construct a line three-dimensional model, and retain the tree region within a preset distance range of the line three-dimensional model as a tree barrier region.

[0193] The line three-dimensional model is a refined line space model constructed based on the two-dimensional semantic mask output by the first branch and the three-dimensional geometric features output by the second branch, and contains the three-dimensional structures of components such as conductors, towers and insulators; the tree barrier region is the part of the tree region that may pose a threat to the safety of the line, and by limiting the preset distance from the line three-dimensional model, the interference of irrelevant tree regions is reduced.

[0194] The preset distance range is determined according to the voltage level of the line. For example:

[0195] In some implementations, the construction process of the line three-dimensional model includes:

[0196] The line two-dimensional semantic mask of the first branch is bound to the line region point cloud through a mapping matrix to obtain a semantic enhanced line point cloud;

[0197] A Poisson surface reconstruction algorithm is used for surface fitting, a local linear constraint is introduced for the conductor, and a voxel grid is encrypted for the tower to complete; wherein the local linear constraint means that in the local neighborhood of the conductor point cloud (such as within a preset radius centered on a certain point), the geometric features such as normal vector and curvature of the point cloud are calculated to determine the linear trend that should be met in this region; during the solution of the Poisson equation, a constraint is imposed on the surface reconstruction result of this region to tend to a straight line or a plane, limiting the excessive bending of the surface, and ensuring that the reconstruction result of each small section of the conductor is consistent with the actual linear form;

[0198] The multi-frame time series data of the unmanned aerial vehicle is matched with the stable feature points of the tower through the ICP algorithm, the transformation matrix is calculated and time series alignment is performed; wherein the stable feature points of the tower include but are not limited to the top point of the tower, the connection node of the tower and the cross arm, and the insulator mounting point;

[0199] If the conductor sag and tower spacing deviate from the design value by more than a preset threshold (such as a deviation threshold of 0.5 meters, determined according to the line design specification), a corrected model is obtained by adjusting the point cloud weight through a Bayesian optimization algorithm.

[0200] The accuracy of the line three-dimensional model directly affects the accuracy of the tree barrier region determination, and its construction process through semantic enhancement, surface reconstruction and time series alignment can ensure high consistency with the actual line structure.

[0201] For example, based on the first branch semantic mask and the second branch point cloud features of a 10kV line, a line three-dimensional model is constructed, the conductor sag deviates from the design value by 0.3 meters, which is less than the 0.5 meter threshold, and no further correction is needed; the tree region within a 5-meter range around the model is marked as a tree barrier region, and then the tree point cloud 10 meters away from the line is removed, reducing the amount of invalid data by 60%.

[0202] S302, hierarchical clustering is performed on the trees in the tree barrier region to obtain a plurality of tree key parts; the sag lowest points and the insulator mounting points in the line three-dimensional model are extracted as a reference point set; a K nearest neighbor search algorithm is used to calculate the three-dimensional Euclidean distance between the tree key parts and the reference point set to obtain a static minimum distance .

[0203] The tree key parts refer to parts of the trees that pose a greater threat to the line safety, such as the top of the trunk, the tips of the main branches, etc.; the line reference point set refers to feature points in the line that are most likely to come into contact with the trees, such as the conductor sag lowest points and the insulator mounting points; and the static minimum distance refers to the shortest spatial distance between the two in the absence of external interference (such as wind).

[0204] In some implementations, the hierarchical clustering can use the AGNES algorithm (Agglomerative Nesting), and the number of clustering layers is determined according to the height of the tree, such as 3 layers for a tree height of 5-10 meters and 5 layers for a tree height of 10-20 meters, and the distance threshold for each layer of clustering is 0.5 meters, or can be set based on the average diameter of the tree branches, to finally obtain key parts such as the trunk, the first-order main branches, and the second-order main branches.

[0205] When the line reference point set is extracted, the conductor sag lowest points are obtained by fitting the extreme points of the conductor curve in the line three-dimensional model, and the insulator mounting points can be obtained from the tower point cloud in combination with the structural features of the line three-dimensional model in a preset range of the nodes connecting the tower and the cross arm (the range is determined according to the insulator mounting size in the line design drawing), by searching for a point set having “cylindrical” or “sheet” geometric features (corresponding to the umbrella skirt or connecting hardware structure of the insulator) and having a connection transition with the cross arm point cloud and the tower point cloud.

[0206] Then, the KD tree nearest neighbor search algorithm is used to calculate the static minimum distance.

[0207] It should be noted that the hierarchical clustering focuses on the key parts of the trees that are prone to contact, avoiding redundancy caused by calculating the full tree point cloud; and the selection of the line reference point set is based on the structural features of the line, giving priority to the force-sensitive points and the discharge-prone areas to ensure that the static distance can reflect the true safety distance.

[0208] For example, for a tree barrier region with a tree height of 15 meters, 5-layer clustering is performed to obtain key parts such as the trunk (1st layer), 3 first-order main branches (2nd-4th layers), and 2 second-order main branch tips (5th layer); 3 conductor sag lowest points and 2 insulator mounting points are extracted from the line three-dimensional model as the reference point set; and the three-dimensional Euclidean distance between the key parts and the reference point set is calculated by the KD tree to obtain a static minimum distance = 2.3 meters.

[0209] S303: Calculate the maximum sway of the conductor based on historical meteorological data; calculate the flexibility coefficient based on the diameter of tree branches, and multiply the flexibility coefficient by the average maximum wind speed v within a preset period to obtain the maximum sway of the branches; calculate the sum of the maximum sway of the conductor and the maximum sway of the branches to obtain the dynamic maximum distance; the diameter of tree branches is obtained by extracting the geometric features of the three-dimensional point cloud of the tree area.

[0210] The maximum dynamic sway refers to the maximum displacement that a conductor and tree branches may undergo under the influence of external factors such as wind; the maximum dynamic distance is the sum of the sway of the two, reflecting the distance change under extreme working conditions.

[0211] In some implementations, the maximum swing of the conductor The formula for calculation is:

[0212]

[0213] Where k is the conductor type coefficient, such as k=0.002 for LGJ-120 conductor, which is fitted by experimental data based on conductor material and cross-sectional area. v is the average maximum wind speed within the preset period, which can be obtained by taking the meteorological data of the most recent month. The preset period is set to a time period shorter than one month, such as three days. The average wind speed for each three days is calculated and the maximum value is taken, in m / s.

[0214] Then, the flexibility coefficient is calculated based on the diameter d of the tree branches. :

[0215]

[0216] Where a is the basic flexibility coefficient, with a default value of 0.8, determined through experiments on branches of similar trees; b is the diameter influence coefficient, with a default value of 0.1, used to reflect the change of branch rigidity with diameter; and d is the branch diameter, obtained through point cloud cylindrical fitting.

[0217] The maximum sway of the branch is obtained based on the flexibility coefficient and the average maximum wind speed v. :

[0218]

[0219] The calculation of the dynamic maximum distance takes into account the impact of environmental factors on the safety distance, which can avoid the risk misjudgment caused by relying solely on the static distance, such as the possibility of contact caused by branches or wires swinging significantly in strong winds.

[0220] For example, if the average maximum wind speed within a preset period is v=10m / s and the k=0.002 for the LGJ-120 conductor, then the maximum sway of the conductor is... m; the diameter of a branch in the tree barrier area is d = 0.1 m, the flexibility coefficient a = 0.8-0.1*0.1 = 0.79, and the maximum swing of the branch = 0.79*10 = 7.9 m; the maximum dynamic distance = 0.2 + 7.9 = 8.1 m.

[0221] S304, subtracting the maximum dynamic distance from the minimum static distance to calculate the minimum clearance distance, and generating a tree barrier risk warning when the distance is less than zero.

[0222] The minimum clearance distance is the minimum static distance minus the maximum dynamic distance, which is used to quantitatively evaluate the tree barrier risk; if the value is less than zero, it means that even with a certain distance in the static state, the tree and the line may still contact in the extreme working condition, and a warning needs to be issued.

[0223] In some implementations, the calculation formula of the minimum clearance distance is:

[0224]

[0225] The warning information can include a risk signal and a specific location.

[0226] The minimum clearance distance realizes dynamic quantitative evaluation of the tree barrier risk by combining static and dynamic factors, and is more in line with the actual working condition than pure static distance judgment, which can provide accurate decision-making basis for line operation and maintenance.

[0227] For example, in combination with the results of S302 and S303, the minimum static distance = 2.3 m, the maximum dynamic distance = 8.1 m, and the minimum clearance distance = 2.3-8.1 =-5.8 m < 0, the system generates a high-level risk warning and sends a warning signal to the terminal platform.

[0228] Based on the above technical solution, S3 filters the tree barrier area by constructing an accurate line three-dimensional model, calculates the static distance in combination with hierarchical clustering and reference point extraction, introduces the dynamic swing to evaluate the influence of extreme working conditions, and finally obtains the quantified minimum clearance distance and generates a warning. The whole process takes into account both static structure and dynamic environmental factors, realizes scientific evaluation and accurate warning of tree barrier risk, and provides reliable technical support for safe operation and maintenance of distribution network lines.

[0229] Based on the above steps S1-S3, the tree barrier identification method for distribution network lines based on artificial intelligence provided by the present application integrates multi-modal data acquisition, artificial intelligence processing and quantitative risk evaluation, overcomes the inefficiency of traditional manual inspection, solves the limitations of single data recognition, and can more accurately and efficiently find the tree barrier risk in the distribution network line, providing strong technical support for line operation and maintenance.

[0230] The above describes the scheme of the embodiments of the present application mainly from the perspective of device implementation. It can be understood that each device, for example, an artificial intelligence-based power distribution network line tree barrier identification device, includes at least one of a corresponding hardware structure and a software module for executing each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0231] Embodiment 2

[0232] This embodiment is based on the above-mentioned embodiment 1, and discloses an artificial intelligence-based power distribution network line tree barrier identification system. This embodiment can divide the functional modules of an artificial intelligence-based power distribution network line tree barrier identification system according to the method examples in the above-mentioned embodiment 1. For example, each functional module can be divided according to each function, or two or more functions can be integrated in one processing module, such as Figure 2 As shown in the figure, the system in this embodiment includes a data acquisition module, a data processing module, and a risk early warning module; wherein,

[0233] The data acquisition module is configured to acquire multi-modal data of a power transmission line corridor by using a UAV, including three-dimensional point cloud data and visible light images.

[0234] The data processing module is configured to perform spatio-temporal registration on the acquired multi-modal data, and enhance the registered multi-modal data by using a generative adversarial network to obtain a preprocessed data set.

[0235] The preprocessed data set is subjected to semantic segmentation and three-dimensional feature extraction by using a double-branch tree barrier identification model to obtain a tree region and a line region; the double-branch tree barrier identification model is pre-constructed based on a deep learning algorithm.

[0236] The risk early warning module is configured to calculate the minimum clearance distance of the tree region and the line region, and generate a tree barrier risk early warning when the minimum clearance distance is less than zero.

[0237] Embodiment 3

[0238] The embodiment is based on the above-mentioned embodiment 1-2, and discloses an artificial intelligence-based power distribution network line tree barrier identification device. According to the method example of the above-mentioned embodiment 1, the functional units of the artificial intelligence-based power distribution network line tree barrier identification device are divided, for example, each functional unit can be divided according to each function, or two or more functions can be integrated in one processing unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit. It should be noted that the division of units in this embodiment is illustrative, and is only a logical function division. In actual implementation, there can be another division method.

[0239] In the case of using integrated units, Figure 5 A possible structure diagram of an artificial intelligence-based power distribution network line tree barrier identification device (denoted as an artificial intelligence-based power distribution network line tree barrier identification device 50) involved in the above-mentioned embodiment is shown, which includes a processing unit 501 and a communication unit 502, and can also include a storage unit 503. Figure 5 The structure diagram shown can be used to illustrate the structure of an artificial intelligence-based power distribution network line tree barrier identification device involved in the above-mentioned embodiment.

[0240] When Figure 5 When the structure diagram shown is used to illustrate the structure of an artificial intelligence-based power distribution network line tree barrier identification device involved in the above-mentioned embodiment, the processing unit 501 is used to control and manage the actions of an artificial intelligence-based power distribution network line tree barrier identification device, the communication unit 502 is used for communication between an artificial intelligence-based power distribution network line tree barrier identification device and other devices, and the storage unit 503 is used to store the program code and data of an artificial intelligence-based power distribution network line tree barrier identification device.

[0241] For example, the communication unit 502 is used to receive the power line corridor multi-modal data collected by the unmanned aerial vehicle, and send the generated tree barrier risk warning information to the related terminal device.

[0242] The processing unit 501 is used to perform spatio-temporal registration on the received multi-modal data, enhance the registered multi-modal data through a generative adversarial network to obtain a preprocessed data set, perform semantic segmentation and three-dimensional feature extraction on the preprocessed data set by using a double-branch tree barrier identification model to obtain a tree region and a line region, calculate the minimum clearance distance of the tree region and the line region, and generate a tree barrier risk warning when the distance is less than zero.

[0243] In a possible implementation, the processing unit 501 is further configured to construct a line three-dimensional model, perform hierarchical clustering on the tree barrier region to obtain key parts of the trees, extract a set of reference points from the line three-dimensional model, and calculate a dynamic maximum swing amount of the conductor and the tree branches.

[0244] In a possible implementation, the communication unit 502 is further configured to receive line design values and other related information in a power grid line design drawing; and the processing unit 501 is further configured to perform optimization adjustment on the line model after timing correction according to the received line design values, to obtain a corrected line three-dimensional model.

[0245] The processing unit 501 can be a processor or a controller, and the communication unit 502 can be a communication interface, a transceiver, a transceiver, a transceiver circuit, a transceiver device, or the like. The communication interface is a general term, and can include one or more interfaces. The storage unit 503 can be a memory. When the power grid line tree barrier identification device 50 based on artificial intelligence is a chip, the processing unit 501 can be a processor or a controller, and the communication unit 502 can be an input interface and / or an output interface, a pin, or a circuit, etc. The storage unit 503 can be a storage unit (for example, a register, a cache, etc.) in the chip, or can be a storage unit (for example, a read-only memory (ROM), a random access memory (RAM), etc.) located outside the chip.

[0246] The communication unit can also be referred to as a transceiving unit. The antenna and control circuit with transceiving function in the power grid line tree barrier identification device 50 based on artificial intelligence can be regarded as a communication unit 502 of the power grid line tree barrier identification device 50 based on artificial intelligence, and the processor with processing function can be regarded as a processing unit 501 of the power grid line tree barrier identification device 50 based on artificial intelligence. Optionally, the device for realizing the receiving function in the communication unit 502 can be regarded as a communication unit, and the communication unit is configured to perform the receiving steps in the embodiments of the present application, and the communication unit can be a receiver, a receiver, a receiving circuit, etc. The device for realizing the sending function in the communication unit 502 can be regarded as a sending unit, and the sending unit is configured to perform the sending steps in the embodiments of the present application, and the sending unit can be a transmitter, a transmitter, a sending circuit, etc.

[0247] Figure 5The integrated units in the above-mentioned embodiments, if realized in the form of software function modules and sold or used as independent products, can be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the present application essentially or in other words the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in the embodiments of the present application. The storage medium storing the computer software product includes a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0248] Embodiment 4

[0249] This embodiment is based on the above-mentioned embodiments 1-3, and discloses an artificial intelligence-based power distribution network line tree barrier identification device (denoted as an artificial intelligence-based power distribution network line tree barrier identification device 60). The hardware structure diagram is shown in Figure 6 The artificial intelligence-based power distribution network line tree barrier identification device 60 includes a processor 601, a memory 602 connected with the processor 601, and a transceiver 603.

[0250] The processor 601, the memory 602, and the transceiver 603 are connected through a bus. The transceiver 603 is used for communication with other devices or communication networks. The transceiver 603 can include a transmitter and a receiver. The device for realizing the receiving function in the transceiver 603 can be regarded as a receiver, and the receiver is used for executing the receiving steps in the embodiments of the present application. The device for realizing the sending function in the transceiver 603 can be regarded as a transmitter, and the transmitter is used for executing the sending steps in the embodiments of the present application.

[0251] Figure 6 The structure diagram shown can be used to illustrate the structure of the above-mentioned artificial intelligence-based power distribution network line tree barrier identification device, and can also illustrate a system chip in an artificial intelligence-based power distribution network line tree barrier identification device. In this case, the actions performed by the above-mentioned artificial intelligence-based power distribution network line tree barrier identification device can be realized by the system chip, and the specific actions performed can be referred to in the above, which will not be described here again.

[0252] In the implementation process, each step in the method provided by embodiment 1 can be completed by the integrated logic circuit of hardware in the processor or the instructions in the form of software. The steps of the method disclosed in combination with embodiment 1 can be directly embodied as hardware processor execution completion, or executed by hardware and software modules in the processor.

[0253] The processor in the embodiment can include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, and the like, each of which is a computing device running software and can include one or more cores for executing software instructions to perform operations or processing. The processor can be a separate semiconductor chip, or can be integrated with other circuits as a semiconductor chip, for example, can be integrated with other circuits (such as coding and decoding circuits, hardware acceleration circuits, or various bus and interface circuits) to form a SoC (system on chip), or can be integrated as a built-in processor in an ASIC. The ASIC integrated with the processor can be packaged separately or packaged together with other circuits. In addition to including cores for executing software instructions to perform operations or processing, the processor can further include necessary hardware accelerators, such as field programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits implementing special logic operations.

[0254] The memory in the embodiment can include at least one of the following types: read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, and electrically erasable programmable read-only memory (EEPROM). In some scenarios, the memory can also be a compact disc read-only memory (CD-ROM) or other optical disk storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited thereto.

[0255] Embodiment 5

[0256] The embodiment provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method in Embodiment 1.

[0257] Embodiment 6

[0258] The embodiment provides a computer program product containing instructions, which, when executed on a computer, cause the computer to perform the method in the above embodiment 1.

[0259] Embodiment 7

[0260] The embodiment provides a chip, which comprises a processor and an interface circuit, the interface circuit is coupled with the processor, the processor is used for running a computer program or instructions to implement the method in the above embodiment 1, and the interface circuit is used for communicating with other modules outside the chip.

[0261] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by using a software program, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product contains one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (for example, coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as one or more servers, data centers, etc. integrated with one or more media. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, solid state disk (solid state disk, SSD)) and the like.

[0262] Although the present application is described herein in conjunction with various embodiments, other variations of the disclosed embodiments can be understood and implemented by those skilled in the art with reference to the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "one" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. Some measures are described in mutually different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0263] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements shall be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An artificial intelligence-based power distribution network line tree barrier identification method, characterized in that, The method comprises: Collecting multi-modal data of a power transmission line corridor by a UAV, including three-dimensional point cloud data and visible light images; Performing spatio-temporal registration on the collected multi-modal data, and enhancing the registered multi-modal data by a generative adversarial network to obtain a preprocessed data set; the generative adversarial network comprises a Pix2PixHD network and a PointGAN network; Performing semantic segmentation and three-dimensional feature extraction on the preprocessed data set by a dual-branch tree barrier identification model to obtain a tree region and a line region; the dual-branch tree barrier identification model is pre-constructed based on a deep learning algorithm; Calculating the minimum clearance distance of the tree region and the line region, and generating a tree barrier risk warning when the minimum clearance distance is less than zero; The dual-branch tree barrier identification model comprises a first branch, a second branch, a cross-modal guidance module, a feature fusion module and a tree barrier region generation module; wherein, The first branch is used for feature extraction on the enhanced visible light image data, and outputs two-dimensional semantic masks of trees and lines and their confidence levels; the line comprises a conductor, an insulator and a tower; The cross-modal guidance module is used for mapping the two-dimensional semantic masks output by the first branch to a three-dimensional point cloud space according to a mapping matrix, to obtain three-dimensional attention masks and three-dimensional attention weights of trees and lines; The second branch is used for processing the enhanced point cloud data according to the three-dimensional attention masks, and capturing the spatial form and positional relationship of trees and lines through the three-dimensional attention weights, to output three-dimensional geometric features of trees and lines; The feature fusion module is used for dimension splicing and feature reorganization of high-level semantic features of the first branch and three-dimensional geometric features of the second branch, to generate cross-modal fusion features with fused semantic information and spatial information; the high-level semantic features of the first branch are features obtained by converting the last feature map before outputting the two-dimensional semantic masks to the three-dimensional point cloud space through the mapping matrix; The tree barrier region generation module is used for point cloud clustering and category determination based on the cross-modal fusion features, to respectively divide three-dimensional spatial regions of trees and lines, and to output tree regions and line regions by removing noise points in the three-dimensional spatial regions through boundary optimization.

2. The power distribution network line tree barrier identification method based on artificial intelligence according to claim 1, characterized in that, The process of spatio-temporal registration on the collected multi-modal data comprises: Performing timestamp synchronization on the three-dimensional point cloud data in the multi-modal data, and eliminating the collection delay of the sensor by using an interpolation compensation algorithm; Extracting two-dimensional feature points of the visible light images and geometric feature points of the three-dimensional point cloud; Matching the two-dimensional feature points and the geometric feature points based on a spatial constraint condition, and removing feature point pairs whose projection errors exceed a preset distance threshold after matching, to obtain registered multi-modal data; wherein, the registered multi-modal data comprises a registered image, a registered point cloud and a mapping matrix of the registered image and the registered point cloud; Converting the registered multi-modal data to a unified geographic coordinate system.

3. The method of claim 2, wherein the method further comprises: The process of enhancing the registered multi-modal data by the generative adversarial network comprises: Processing the registered image by the Pix2PixHD network to obtain enhanced image data; The PointGAN network is used to process the registration point cloud, complete the missing area of the registration point cloud, and obtain enhanced point cloud data. The pre-processing data set includes enhanced image data and enhanced point cloud data.

4. The power distribution network line tree barrier identification method based on artificial intelligence according to claim 1, characterized in that, The process of obtaining the three-dimensional attention mask and the three-dimensional attention weight of the trees and the line by the cross-modal guidance module is as follows: Obtain a mapping matrix M. Two-dimensional semantic mask output from the first branch Target pixel coordinate set extracted from the middle Target pixel coordinates combined Satisfies: ； in, The coordinates in the two-dimensional semantic mask are The pixels belong to category c, where c=0 represents the tree category and c=1 represents the line category; each point in the set of target pixel coordinates is converted to a three-dimensional point cloud coordinate by the mapping matrix M; the three-dimensional point cloud coordinate satisfies: ； According to the three-dimensional point cloud coordinates, mark the points belonging to the registered point cloud as 1, and otherwise mark as 0, to generate a three-dimensional attention mask ;​ The three-dimensional attention weight is calculated based on a two-dimensional semantic mask confidence and a three-dimensional Euclidean distance The three-dimensional Euclidean distance is calculated based on three-dimensional point coordinates.

5. The power distribution network line tree barrier identification method based on artificial intelligence according to claim 1, characterized in that, The process of outputting the three-dimensional geometric features of the trees and the line by the second branch includes: Downsample and normalize the enhanced point cloud data to obtain a simplified point cloud feature; Through three-dimensional attention mask Filtering a target point set from the reduced point cloud features ; The target point set is obtained The three-dimensional attention weight is obtained The weighted point cloud is obtained by point-by-point multiplication The weighted point cloud includes spatial forms and position relationships of trees and lines The pre-constructed PointNet++ network is used to extract features of the weighted point cloud, and three-dimensional geometric features of the trees and the line are obtained , comprising: Output local features by a pre-constructed Set Abstraction layer in the PointNet++ network; Perform maximum pooling on the local features to obtain global features; The local features are spliced with global features after broadcasting to obtain three-dimensional geometric features of the trees and the line .

6. The power distribution network line tree barrier identification method based on artificial intelligence according to claim 1, characterized in that, The process of generating the cross-modal fusion features that fuse semantic information and spatial information by the feature fusion module includes: High-level semantic features of the first branch By transforming the point cloud into a 3D point cloud space through a mapping matrix M, 3D semantic features aligned with the simplified point cloud features are obtained. ; aligning the point number and feature dimension of the three-dimensional geometric features with the three-dimensional semantic features by interpolation or 1x1 convolution; The aligned features are spliced along the feature dimension to obtain spliced features ; performing a nonlinear transformation on the concatenated features to output a cross-modal fusion feature .

7. The power distribution network line tree barrier identification method based on artificial intelligence according to claim 1, characterized in that, The process of calculating the minimum clearance distance of the tree area and the line area includes: Retain the tree area within a preset distance range of the line three-dimensional model to obtain a tree barrier area; the line three-dimensional model is constructed based on the outputs of the first branch and the second branch; Perform hierarchical clustering on the trees in the tree barrier area to obtain a plurality of key parts of the trees; Extract the sag lowest point and the insulator installation point from the line three-dimensional model as a reference point set; The K-Nearest Neighbor search algorithm is used to calculate the three-dimensional Euclidean distance between the key parts of the tree and the reference point set to obtain the static minimum distance ; Calculate the maximum swing amount of the conductor based on historical meteorological data; Calculate the flexibility coefficient according to the diameter of the tree branch, and multiply the flexibility coefficient by the average maximum wind speed v in a preset period to obtain the maximum swing amount of the branch; the diameter of the tree branch is obtained by extracting the geometric features of the three-dimensional point cloud of the tree area; Calculate the sum of the maximum swing amount of the conductor and the maximum swing amount of the branch to obtain a dynamic maximum distance; Subtract the dynamic maximum distance from the static minimum distance to obtain the minimum clearance distance.

8. The power distribution network line tree barrier identification method based on artificial intelligence according to claim 7, characterized in that, The construction process of the line three-dimensional model includes: Bind the position label of the line in the two-dimensional semantic mask output by the first branch to the line area through the mapping matrix to obtain a semantic enhanced line point cloud; Perform surface fitting on the semantic enhanced line point cloud by using a Poisson surface reconstruction algorithm to obtain an initial reconstruction model; Match the stable feature points of the tower by using the iterative closest point algorithm based on the multi-frame time series data of the unmanned aerial vehicle, and calculate the transformation matrix of the adjacent frame line point cloud; the stable feature points include the tower fixed point, the connection node of the tower and the cross arm, and the insulator installation point; Perform time series alignment on the initial reconstruction model according to the transformation matrix to obtain a time series corrected line model; If the conductor sag, the tower spacing and the design value of the line in the time series corrected line model deviate from a preset threshold value, adjust the point cloud weight of the time series corrected line model by using a Bayesian optimization algorithm to obtain a final line three-dimensional model; the design value of the line is obtained from the power distribution line design drawing.

9. An artificial intelligence-based power distribution network line tree barrier identification system, characterized by, It includes: The data acquisition module, the data processing module and the risk early warning module; wherein, The data acquisition module is configured to acquire multi-modal data of the power transmission line corridor by using an unmanned aerial vehicle, including three-dimensional point cloud data and visible light images. The data processing module is configured to perform spatio-temporal registration on the collected multi-modal data, and enhance the registered multi-modal data by using a generative adversarial network to obtain a pre-processed data set; the generative adversarial network comprises a Pix2PixHD network and a PointGAN network; The pre-processed data set is subjected to semantic segmentation and three-dimensional feature extraction by using a double-branch tree barrier identification model to obtain a tree region and a line region; the double-branch tree barrier identification model is pre-constructed based on a deep learning algorithm; The risk warning module is configured to calculate the minimum clearance distance of the tree region and the line region, and generate a tree-barrier risk warning when the minimum clearance distance is less than zero; The double-branch tree barrier identification model comprises a first branch, a second branch, a cross-modal guidance module, a feature fusion module and a tree-barrier region generation module; wherein, The first branch is configured to extract features from the enhanced visible light image data, and output two-dimensional semantic masks and their confidence levels of trees and lines; the line comprises a conductor, an insulator and a tower; The cross-modal guidance module is configured to map the two-dimensional semantic masks output by the first branch to a three-dimensional point cloud space according to a mapping matrix, and obtain three-dimensional attention masks and three-dimensional attention weights of trees and lines; The second branch is configured to process the enhanced point cloud data according to the three-dimensional attention masks, and capture the spatial form and positional relationship of trees and lines by using the three-dimensional attention weights, and output three-dimensional geometric features of trees and lines; The feature fusion module is configured to perform dimension concatenation and feature reorganization on the high-level semantic features of the first branch and the three-dimensional geometric features of the second branch to generate cross-modal fusion features that fuse semantic information and spatial information; the high-level semantic features of the first branch are features obtained by converting the last feature map before outputting the two-dimensional semantic masks to the three-dimensional point cloud space by using the mapping matrix; The tree-barrier region generation module is configured to perform point cloud clustering and category determination based on the cross-modal fusion features, divide the three-dimensional space region of trees and lines respectively, and remove noise points in the three-dimensional space region by boundary optimization, and output the tree region and the line region.

Images