Power distribution network line tree obstacle identification method and system based on artificial intelligence

Abstract

The invention relates to a power distribution network line tree obstacle identification method and system based on artificial intelligence, and the method improves the quality and applicability of a preprocessing data set through the time-space registration and the data enhancement of a generative adversarial network; a double-branch tree obstacle recognition model constructed based on a deep learning algorithm is utilized, on one hand, semantic segmentation is performed to accurately position a tree region and a line region, on the other hand, spatial features of the tree region and the line region can be deeply excavated through three-dimensional feature extraction, and the recognition accuracy of a target region is greatly improved through the double-branch synergistic effect; the possibility of misjudgment or missed judgment is reduced; and finally, early warning is generated by calculating the minimum clearance distance of the tree area and the line area, and the tree obstacle risk is judged in a quantitative mode, so that the identification result is more objective and scientific, and the accuracy of tree obstacle identification is further guaranteed. Compared with the prior art, the multi-model deep learning algorithm is fused, so that the accuracy of tree obstacle identification and the accuracy of risk early warning can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of power distribution network line operation and maintenance, and in particular to a power distribution network line tree barrier identification method and system based on artificial intelligence. BACKGROUND

[0002] Tree barriers refer to trees in the transmission line corridor whose height, branch extension range, and other factors cause the distance to the line facilities to be less than the safety threshold, which may cause line short circuit, discharge, trip, and other faults, and is one of the important factors threatening the safe and stable operation of the power distribution network. With the expansion of the coverage of the power distribution network and the changes in the ecological environment, the growth trend of trees in the line corridor is complex, and dynamic monitoring and accurate identification of tree barrier risks have become a key requirement for power grid operation and maintenance.

[0003] Existing tree barrier identification methods mainly include manual inspection and automated identification. Manual inspection relies on field observation by operation and maintenance personnel or measurement with simple tools, which is not only inefficient and costly, but also limited by environmental factors such as terrain, weather, and vegetation obstruction, making it difficult to achieve large-scale and high-frequency monitoring, and prone to missed or incorrect judgments. To improve efficiency, automated identification methods have been gradually applied, but they still have significant shortcomings in accuracy, especially in technical solutions that rely on a single data or a single model: On the one hand, a single data type cannot balance semantic information and spatial features. Some methods only rely on two-dimensional data such as visible light images and infrared images, which can initially identify trees and lines through texture, color, and other semantic features, but are affected by factors such as light, weather, and season, and image details are easily blurred, resulting in low semantic segmentation accuracy of trees and line components, and difficulty in distinguishing subtle differences between tree branches and line conductors. Another part of the method only uses three-dimensional point cloud data, which can reflect the spatial shape and positional relationship of the target, but point cloud data lacks semantic labels, making it difficult to accurately distinguish trees from other objects, and data loss may occur during point cloud collection due to obstruction and insufficient device accuracy, resulting in blurred boundaries between tree and line areas.

[0004] On the other hand, a single model cannot realize the synergistic fusion of multi-modal features. Existing automated methods mostly use a single deep learning model, which is difficult to simultaneously utilize the complementary advantages of different data. For example, a CNN-based model can extract image semantic features, but it cannot capture the three-dimensional spatial distance relationship between trees and lines, and it is easy to misjudge the trees and lines that overlap on the plane but are actually far apart as tree barriers; a PointNet-based model can process the spatial features of point clouds, but it lacks the guidance of image semantic information, making it difficult to accurately locate the key parts of trees and sensitive points of lines, leading to regional division deviation, and thus affecting the accuracy of tree barrier early warning. For example, Chinese patent CN120218632B discloses a tree barrier intelligent detection method and system based on multi-modal perception, which uses a UAV to carry multi-modal sensors to collect data, uses multi-spectral data fusion and deep learning algorithms to construct a three-dimensional tree model, and calculates the minimum safety clearance distance between power lines and trees in real time, and performs risk assessment, but its accuracy is not high, the lines and trees are processed independently, and the original accuracy of the point cloud is relied on, which is easily affected by occlusion, and it is easy to misjudge the trees or lines that overlap on the plane but are actually far apart as risks. SUMMARY

[0005] The purpose of the present application is to overcome the defects of the prior art and provide a power distribution network line tree barrier identification method and system based on artificial intelligence, which solves the technical problem of insufficient accuracy and easy misjudgment or omission caused by relying on single data or single model in the prior art.

[0006] The purpose of the present application can be achieved by the following technical solutions: A power distribution network line tree barrier identification method based on artificial intelligence, the method comprising: collecting multi-modal data of a power transmission line corridor by a UAV, including three-dimensional point cloud data and visible light images; spatially and temporally registering the collected multi-modal data, and enhancing the registered multi-modal data by a generative adversarial network to obtain a preprocessed data set; using a double-branch tree barrier identification model to perform semantic segmentation and three-dimensional feature extraction on the preprocessed data set to obtain a tree region and a line region; the double-branch tree barrier identification model is pre-constructed based on a deep learning algorithm; calculating the minimum clearance distance of the tree region and the line region, and generating a tree barrier risk warning when the minimum clearance distance is less than zero.

[0007] Further, the process of spatially and temporally registering the collected multi-modal data comprises: performing timestamp synchronization on the three-dimensional point cloud data in the multi-modal data, and using an interpolation compensation algorithm to eliminate the collection delay of the sensor; extracting two-dimensional feature points of the visible light image and geometric feature points of the three-dimensional point cloud; matching the two-dimensional feature points and the geometric feature points based on a spatial constraint condition, and eliminating feature point pairs whose projection errors exceed a preset distance threshold after matching, to obtain registered multi-modal data; wherein the registered multi-modal data includes a registered image, a registered point cloud, and a mapping matrix of the registered image and the registered point cloud; converting the registered multi-modal data to a unified geographic coordinate system.

[0008] Further, the generative adversarial network includes a Pix2PixHD network and a PointGAN network, and the process of enhancing the registered multi-modal data by the generative adversarial network includes: processing the registered image by the Pix2PixHD network to obtain enhanced image data; processing the registered point cloud by the PointGAN network to complete the missing area of the registered point cloud, to obtain enhanced point cloud data; the preprocessed data set includes the enhanced image data and the enhanced point cloud data.

[0009] Further, the dual-branch tree barrier identification model includes a first branch, a second branch, a cross-modal guidance module, a feature fusion module, and a tree barrier region generation module; wherein, the first branch is configured to extract features of the enhanced visible light image data, and output two-dimensional semantic masks of trees and lines and their confidence levels; the lines include but are not limited to conductors, insulators, and towers; the cross-modal guidance module is configured to map the two-dimensional semantic masks output by the first branch to a three-dimensional point cloud space according to the mapping matrix, obtain three-dimensional attention masks and three-dimensional attention weights of the trees and the lines, and guide the second branch to focus on three-dimensional feature extraction of the trees and the lines by using the three-dimensional attention masks; the second branch is configured to process the enhanced point cloud data according to the three-dimensional attention masks, and capture spatial forms and positional relationships of the trees and the lines by using the three-dimensional attention weights, and output three-dimensional geometric features of the trees and the lines; the feature fusion module is configured to concatenate and reorganize high-level semantic features of the first branch and three-dimensional geometric features of the second branch in dimensions, to generate cross-modal fusion features that fuse semantic information and spatial information; the high-level semantic features of the first branch are features obtained after converting the last feature map before outputting the two-dimensional semantic masks to the three-dimensional point cloud space by using the mapping matrix; The tree barrier region generation module is used to perform point cloud clustering and category determination based on the cross-modal fusion features, to divide the three-dimensional spatial regions of trees and lines respectively, and to remove noise points in the three-dimensional spatial regions through boundary optimization, and output tree regions and line regions.

[0010] Furthermore, the process by which the cross-modal guidance module obtains the 3D attention mask and 3D attention weights for trees and lines is as follows: Obtain the mapping matrix M; Two-dimensional semantic mask output from the first branch Extract the target pixel coordinate set The target pixel coordinates are combined satisfy: ; in, The coordinates in the two-dimensional semantic mask are The pixels belong to category c, where c=0 represents the tree category and c=1 represents the line category; The target pixel coordinate set is obtained through the mapping matrix M. Each point in the graph is converted into 3D point cloud coordinates. The three-dimensional point cloud coordinates satisfy: ; Based on the three-dimensional point cloud coordinates, the points in the registration point cloud belonging to... Points that meet the criteria are marked as 1, otherwise they are marked as 0, generating a 3D attention mask. ; The three-dimensional attention weights are calculated based on the confidence level of the two-dimensional semantic mask and the three-dimensional Euclidean distance. The three-dimensional Euclidean distance is calculated based on the three-dimensional point coordinates.

[0011] Furthermore, the process of the second branch outputting the three-dimensional geometric features of the trees and lines includes: The enhanced point cloud data is downsampled and normalized to obtain simplified point cloud features; Using 3D attention mask Target point set is selected from the simplified point cloud features. ; The target point set With the three-dimensional attention weights Perform point-by-point multiplication to obtain the weighted point cloud. The weighted point cloud includes the spatial morphology and positional relationships of trees and lines. The pre-built PointNet++ network is used to extract features from the weighted point cloud to obtain the 3D geometric features of trees and lines. , comprising: outputting local features by a Set Abstraction layer in a pre-constructed PointNet++ network; max-pooling the local features to obtain global features; concatenating the local features with the broadcasted global features to obtain three-dimensional geometric features of the trees and the line .

[0012] Further, the process of the feature fusion module generating the cross-modal fusion features fusing semantic information and spatial information includes: transforming the high-level semantic features of the first branch to a three-dimensional point cloud space through a mapping matrix M to obtain three-dimensional semantic features aligned with the simplified point cloud features ; aligning the point number and feature dimension of the three-dimensional geometric features and the three-dimensional semantic features by interpolation or 1x1 convolution; concatenating the aligned features along the feature dimension to obtain concatenated features ; performing nonlinear transformation on the concatenated features by a multilayer perception machine (MLP) layer with a ReLU activation function to output cross-modal fusion features .

[0013] Further, the process of calculating the minimum clearance distance of the tree area and the line area includes: retaining the tree area within a preset distance range of the line three-dimensional model to obtain a tree barrier area; the line three-dimensional model is constructed based on the outputs of the first branch and the second branch; performing hierarchical clustering on the trees in the tree barrier area to obtain a plurality of tree key parts; extracting the sag lowest point and the insulator installation point from the line three-dimensional model as a reference point set; calculating the three-dimensional Euclidean distance between the tree key parts and the reference point set using a K-nearest neighbor search algorithm to obtain a static minimum distance ; calculating the maximum swing amount of the conductor based on historical meteorological data; calculating the flexibility coefficient based on the diameter of the tree branch, and multiplying the flexibility coefficient by the average maximum wind speed v in a preset period to obtain the maximum swing amount of the branch; the diameter of the tree branch is obtained by extracting the geometric features of the three-dimensional point cloud of the tree area; calculating the sum of the maximum swing amount of the conductor and the maximum swing amount of the branch to obtain a dynamic maximum distance; ​Subtract the dynamic maximum distance from the static minimum distance to obtain the minimum clearance distance.

[0014] Further, the construction process of the line three-dimensional model comprises: Bind the position label of the line in the two-dimensional semantic mask output by the first branch to the line region through the mapping matrix to obtain a semantic enhanced line point cloud; Perform surface fitting on the semantic enhanced line point cloud by adopting a Poisson surface reconstruction algorithm to obtain an initial reconstruction model; By using multi-frame time sequence data of the unmanned aerial vehicle, the stable feature points of the tower are matched through an iterative closest point algorithm, and the transformation matrix of the adjacent frame line point cloud is calculated; the stable feature points include tower fixed points, tower and cross arm connection nodes and insulator installation points; According to the transformation matrix, the initial reconstruction model is time sequence aligned to obtain a line model after time sequence correction; If the deviation of the conductor sag and the tower spacing in the line model after time sequence correction from the design value of the line exceeds a preset threshold, the point cloud weight of the line model after time sequence correction is adjusted through a Bayesian optimization algorithm to obtain a final line three-dimensional model; the design value of the line is obtained through a power distribution network line design drawing.

[0015] An artificial intelligence-based power distribution network line tree barrier identification system comprises a data acquisition module, a data processing module and a risk early warning module; wherein, The data acquisition module is configured to acquire multi-modal data of a power transmission line corridor through an unmanned aerial vehicle, including three-dimensional point cloud data and visible light images; The data processing module is configured to perform time-space registration on the acquired multi-modal data, and enhance the registered multi-modal data through a generative adversarial network to obtain a preprocessed data set; A double-branch tree barrier identification model is used to perform semantic segmentation and three-dimensional feature extraction on the preprocessed data set to obtain a tree region and a line region; the double-branch tree barrier identification model is pre-constructed based on a deep learning algorithm; The risk early warning module is configured to calculate the minimum clearance distance of the tree region and the line region, and generate a tree barrier risk early warning when the minimum clearance distance is less than zero.

[0016] Compared with the prior art, the beneficial effects of the present application include: 1. The present application provides a power distribution network line tree barrier identification method and system based on artificial intelligence, which can improve the accuracy of tree barrier identification and the accuracy of risk warning by fusing multi-model deep learning algorithm; through space-time registration and data enhancement of generative adversarial network, the quality and applicability of the preprocessed data set are improved; a double-branch tree barrier identification model based on deep learning algorithm is used, on the one hand, semantic segmentation can accurately locate the tree area and line area, on the other hand, through three-dimensional feature extraction, the spatial features of the two can be deeply mined, the double-branch collaborative action greatly improves the accuracy of target area identification, reduces the possibility of misjudgment or omission, and finally generates a warning by calculating the minimum clearance distance between the tree area and the line area, so as to quantitatively judge the tree barrier risk, making the identification result more objective and scientific, and further ensuring the accuracy of tree barrier identification.

[0017] 2. In the data preprocessing stage, through space-time registration of time stamp synchronization and interpolation compensation, the sensor delay can be eliminated, and then combined with feature point matching and error elimination, the consistency of multi-modal data in space-time dimension can be ensured.

[0018] 3. In the data enhancement stage, the Pix2PixHD network is used to enhance the image data quality, and the PointGAN network is used to complete the missing area of the point cloud, which can make up for the deficiency in the data acquisition process and improve the integrity and reliability of the preprocessed data set.

[0019] 4. In the tree barrier identification stage, the double-branch tree barrier identification model uses the cross-modal guidance module to map the two-dimensional semantic mask to the three-dimensional space to generate attention mask and weight, guide the second branch to focus on three-dimensional feature extraction of trees and lines, and at the same time, fuse high-level semantic features and three-dimensional geometric features to generate cross-modal fusion features, which are accurately divided into target areas through clustering and boundary optimization, which can reduce the misjudgment and omission risk of trees and lines.

[0020] 5. In the minimum clearance distance calculation, by retaining the effective tree barrier area, hierarchical clustering of tree key parts, extracting line reference points, combining K nearest neighbor algorithm to calculate static distance, and introducing dynamic swing amount of conductor and branch, the accuracy of tree barrier identification is ensured, and dynamic quantitative evaluation of tree barrier risk is realized, which makes risk judgment more in line with actual working conditions, and provides more reliable protection for safe operation of power distribution network lines. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A flowchart of a power distribution network line tree barrier identification method based on artificial intelligence provided by the embodiment of the present application; Figure 2 A system architecture diagram of a power distribution network line tree barrier identification system based on artificial intelligence provided by the embodiment of the present application; Figure 3 A flowchart of a spatiotemporal registration method for collected multi-modal data is provided for the embodiments of the present application; Figure 4 A flowchart of a semantic segmentation and three-dimensional feature extraction method for a preprocessed data set using a double-branch tree barrier identification model to obtain a tree region and a line region is provided for the embodiments of the present application; Figure 5 A structural diagram of an artificial intelligence-based power distribution network line tree barrier identification device is provided for the embodiments of the present application; Figure 6 A hardware structural diagram of an artificial intelligence-based power distribution network line tree barrier identification device is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should fall within the scope of protection of the present application.

[0023] Embodiment 1 The present embodiment discloses an artificial intelligence-based power distribution network line tree barrier identification method, which specifically comprises the following steps as shown in the figure: Figure 1 Step S1: Collecting multi-modal data of a power transmission line corridor by a UAV, including three-dimensional point cloud data and visible light images, performing spatiotemporal registration on the collected multi-modal data, and enhancing the registered multi-modal data by a generative adversarial network to obtain a preprocessed data set.

[0024] The power transmission line corridor refers to a strip-shaped area passed through by a power transmission line, covering line facilities such as conductors, towers, and insulators, and environmental elements such as surrounding trees and terrain; the multi-modal data set can include visible light images, infrared images, three-dimensional point clouds, etc., and can reflect the characteristics of the line and trees from different dimensions such as optics, thermotics, and spatial structure.

[0025] In some implementations, the spatiotemporal registration method for multi-modal data includes two types of time synchronization and spatial alignment; specifically, time synchronization can associate different modal data to the same time node by unifying the time stamps of each sensor; spatial alignment can use a feature point matching algorithm (such as Scale-Invariant Feature Transform, SIFT) to extract key feature points in the image and point cloud and establish a corresponding relationship, and map the data to the same spatial coordinate system.

[0026] ​In step S1, the core of the spatio-temporal registration is to eliminate the deviation of multi-modal data in time and space, ensure the accurate correlation of different types of data, including timestamp synchronization, feature point extraction and matching, coordinate unification and other links, and ensure that different types of data can be correlated and complementary information in subsequent processing.

[0027] For the visible light image and three-dimensional point cloud collected by the unmanned aerial vehicle at the same time, the image and the point cloud at the same flight time are bound by the timestamp, and then the edge feature points in the image and the geometric feature points in the point cloud are extracted by using the SIFT algorithm. The image pixel position and the point cloud spatial coordinates are associated by matching these feature points, and the spatio-temporal registration is completed.

[0028] In some implementations, when performing timestamp synchronization on three-dimensional point cloud data, the interpolation compensation algorithm can use linear interpolation or cubic spline interpolation, which is selected according to the data sampling frequency (such as linear interpolation when the sampling frequency is 10 Hz); the determination of the preset distance threshold needs to be combined with the sensor accuracy. For example, when the visible light camera carried by the unmanned aerial vehicle has an accuracy of 0.05 meters and the laser radar has an accuracy of 0.1 meters, the threshold can be set to 1.5 times the sum of the accuracies, i.e. 0.225 meters.

[0029] Feature point matching needs to be based on spatial constraints (such as collinear, coplanar constraints) to improve matching accuracy; the mapping matrix after registration is used to establish the spatial correspondence between the registered image and the registered point cloud, laying a foundation for subsequent cross-modal processing.

[0030] For example, the timestamp of the three-dimensional point cloud data in the multi-modal data has a 0.5 second delay from the visible light image, which is eliminated by the linear interpolation compensation algorithm; the Scale-Invariant Feature Transform (SIFT) two-dimensional feature points of the visible light image and the geometric feature points (such as vertex, edge point) of the three-dimensional point cloud are extracted, and after matching based on spatial constraints, the feature point pairs with a projection error exceeding 0.2 meters (preset threshold) are eliminated, and finally the registered data is converted to the WGS84 geographic coordinate system.

[0031] The specific steps of spatio-temporal registration of the collected multi-modal data in the above step S1 are as shown in Figure 3 , including: S101.1, performing timestamp synchronization on the three-dimensional point cloud data in the multi-modal data, and using an interpolation compensation algorithm to eliminate the collection delay of the sensor.

[0032] Among them, the timestamp synchronization is to align the collection time of the three-dimensional point cloud data with other modal data (such as visible light image), and the interpolation compensation algorithm is used to correct the time deviation caused by the difference in response speed of the sensor.

[0033] In some implementations, the timestamp synchronization can be achieved by obtaining the original timestamps of the three-dimensional point cloud sensor and the image sensor, converting the timestamps of all data to the same time scale based on a high-precision clock such as a GPS time, and using a linear interpolation algorithm to compensate for the data at the intermediate time when the time interval between two adjacent sampling points is less than a preset threshold (such as 0.1 seconds) to ensure the continuity of the time sequence.

[0034] The sensor acquisition delay is mainly caused by the difference in triggering mechanism of different devices, and the interpolation compensation needs to be based on the time correlation of the data to avoid introducing additional errors.

[0035] For example, the sampling time of the three-dimensional point cloud data is t1 and t3, and the acquisition time of the visible light image is t2 (t1 < t2 < t3). The point cloud data at time t2 is calculated by linear interpolation to accurately correspond the point cloud and the image in time and eliminate the 0.05 second acquisition delay.

[0036] S101.2, extract the two-dimensional feature points of the visible light image and the geometric feature points of the three-dimensional point cloud.

[0037] The two-dimensional feature points are the pixel points (such as edge intersection points and texture mutation points) in the visible light image that have recognition, and the three-dimensional geometric feature points are the points (such as vertexes and curvature extreme points) in the three-dimensional point cloud that reflect the spatial structure, both of which provide a basis for subsequent spatial matching.

[0038] In some implementations, the two-dimensional feature points can be extracted by the Scale-Invariant Feature Transform (SIFT) algorithm, which can maintain the stability of the features under scale and rotation changes, and the three-dimensional geometric feature points can be detected by the improved ISS (Intrinsic Shape Signatures) feature point detection algorithm, which selects representative geometric feature points by calculating the curvature and neighborhood distribution of the point cloud.

[0039] The extraction quality of the feature points directly affects the registration accuracy, and the number and recognition of the feature points need to be balanced to avoid excessive redundant points or missing key features.

[0040] S101.3, match the two-dimensional feature points and the geometric feature points based on the spatial constraint conditions, and eliminate the feature point pairs whose projection error exceeds the preset distance threshold after matching to obtain the registered multi-modal data; wherein the registered multi-modal data includes a registered image, a registered point cloud, and a mapping matrix of the registered image and the registered point cloud.

[0041] The spatial constraint conditions include collinear constraint and coplanar constraint, and are used to ensure that the two-dimensional and three-dimensional feature points have a real corresponding relationship in the physical space; the projection error refers to the pixel distance between the three-dimensional feature point projected to the image plane and the corresponding two-dimensional feature point, and the preset distance threshold is used to screen the effective matching pairs.

[0042] In some implementations, the spatial constraint conditions can be used to construct a projection relationship through camera internal and external parameters, project the three-dimensional geometric feature points to the image plane, and preliminarily match the two-dimensional feature points; the determination of the preset distance threshold needs to be combined with the image resolution, for example, when the image resolution is 1920*1080 pixels (pixel size 0.01 meters / pixel), the threshold can be set to the actual distance (0.03 meters) corresponding to 3 pixels.

[0043] The feature point pair matching needs to be performed through a random sample consensus (RANSAC) algorithm to remove outliers, so as to ensure the robustness of the matching result.

[0044] For example, the three-dimensional geometric feature points are projected to the image based on the camera parameters, the Euclidean distance calculation is performed on the two-dimensional feature points, the matching pairs with a distance less than 0.02 meters (2 pixels) are reserved, the RANSAC algorithm is further used to remove 30 abnormal point pairs with a projection error greater than 0.03 meters, and finally 200 groups of effective feature point pairs are obtained, thereby forming the registration image, the registration point cloud, and the mapping matrix of the two.

[0045] S101.4, converting the registered multi-modal data to a unified geographic coordinate system.

[0046] The unified geographic coordinate system provides a global spatial reference for the data, which is convenient for subsequent cross-regional tree barrier analysis and comparison. Common coordinate systems include the WGS84 coordinate system and the national 2000 geodetic coordinate system.

[0047] In some implementations, the conversion process needs to use the GPS positioning information and attitude data (such as heading angle and pitch angle) of the unmanned aerial vehicle, and convert the registration data from the local coordinate system (such as the unmanned aerial vehicle body coordinate system) to the target geographic coordinate system through a coordinate conversion formula (such as the Bursa model); the conversion parameters can be calibrated through the field control points, so as to ensure the conversion accuracy (such as the plane position mean error less than 0.5 meters).

[0048] It should be noted that the unified coordinate system is a prerequisite for multi-segment route data splicing and long-term monitoring data comparison, and the conversion accuracy needs to meet the distance calculation requirements of tree barrier identification.

[0049] Exemplarily, the registered multi-modal data is converted from the local coordinate system of the unmanned aerial vehicle to the WGS84 coordinate system, the conversion parameters are calibrated through the three known GPS control points, and the deviation of the finally converted point cloud data from the actual position is less than 0.3 meters, meeting the accuracy requirement of subsequent clearance distance calculation.

[0050] Based on the above technical solution, the space-time registration eliminates the time deviation of the data through time synchronization, establishes the spatial correlation of the two-dimensional and three-dimensional data through feature point extraction and matching, and realizes the accurate fusion of the multi-modal data in combination with the coordinate unification, thereby providing a space-time consistent high-quality data basis for subsequent data enhancement and tree barrier identification.

[0051] The specific steps of enhancing the registered multi-modal data through the generative adversarial network in the above step S1 include: Step S102.1, processing the registered image by using the Pix2PixHD network to obtain enhanced image data; Step S102.2, processing the registered point cloud by using the PointGAN network to complete the missing area of the registered point cloud, and obtaining enhanced point cloud data.

[0052] Step S102.3, the generative adversarial network includes the Pix2PixHD network and the PointGAN network, which are respectively used for processing the registered image and the registered point cloud.

[0053] The Pix2PixHD network is a high-resolution image generation model based on a conditional generative adversarial network, which is good at processing complex scene image fine generation and enhancement, and can improve the clarity of local details while preserving the overall structure of the image. The PointGAN network is a generative adversarial network for three-dimensional point cloud data, which can learn the spatial distribution characteristics of the point cloud, realize point cloud completion and optimization, and is suitable for irregular and unstructured three-dimensional data processing.

[0054] In some implementations, the pre-training process and the application process of the Pix2PixHD network can include the following steps: During training, registered image data of power line corridors in multiple scenes is collected, covering different light, weather and seasonal conditions, and is preprocessed to a uniform size; then paired samples of “low-quality image-high-quality reference image” are constructed (without manual labeling of missing details), the generator learns the mapping relationship from low quality to high quality, the discriminator distinguishes between generated images and real high-quality images, and the two are trained in opposition until the generator can output enhanced images close to the real ones.

[0055] Then, when the model is applied, the registered line corridor image is input into the trained network, the model will repair the blurred and light defect areas, enhance the texture details of the conductor and tree branches, and output enhanced images with improved quality.

[0056] In some implementations, the pre-training process and the application process of the PointGAN network can include the following steps: During training, complete power line corridor three-dimensional point clouds are first collected, and after preprocessing, the missing areas are marked by labeling to construct paired samples of "incomplete point clouds - complete point clouds". The generator generates point clouds in the missing area as input, and the discriminator judges whether the generated result conforms to the true distribution. Through adversarial training, the generator can complete the point cloud that conforms to the physical law.

[0057] When applied, the registered three-dimensional point cloud is input into the model, and the network automatically identifies sparse or missing areas. Based on the learned distribution characteristics, the data is completed, making the point cloud structure of towers, trees and other targets more complete.

[0058] The role of data augmentation using Pix2PixHD and PointGAN models is: The former solves the quality problem of registered images caused by the limitation of acquisition conditions by improving the clarity of image details, providing more distinct target features for subsequent semantic segmentation; the latter compensates for the information loss caused by occlusion and equipment limitations in the point cloud acquisition process by completing the missing areas of the point cloud, providing more comprehensive spatial feature support for three-dimensional feature extraction. The synergistic effect of the two improves the quality and integrity of the preprocessed data set, laying a solid foundation for the accuracy of subsequent tree barrier recognition.

[0059] It should be noted that image enhancement can improve texture detail clarity, and point cloud completion can compensate for information loss caused by occlusion and insufficient equipment precision, both of which can improve data quality.

[0060] Based on the above technical solutions, step S1 eliminates the spatiotemporal deviation of multi-modal data through fine spatiotemporal registration, and combines the generative adversarial network to enhance image and point cloud data, effectively improving the consistency, integrity and quality of the preprocessed data set, providing high-quality data support for subsequent tree barrier recognition.

[0061] Step S2, a double-branch tree barrier recognition model is used to perform semantic segmentation and three-dimensional feature extraction on the preprocessed data set to obtain tree and line regions.

[0062] The dual-branch tree barrier identification model is constructed based on a deep learning algorithm. One branch is used to process image type data (such as visible light images and infrared images) to identify the two-dimensional contours of trees and lines through semantic segmentation. The other branch is used to process three-dimensional point cloud data to obtain the spatial position and shape information of trees and lines through three-dimensional feature extraction. The basic architecture can include a convolutional neural network (CNN) for feature extraction of the image branch, a point cloud neural network (such as PointNet) for spatial feature learning of the point cloud branch, and the two are combined through a feature fusion layer to output the results. The output tree area and line area are both three-dimensional space regions, i.e., the spatial range occupied by trees and lines respectively in a three-dimensional coordinate system.

[0063] In some implementations, the training of the dual-branch model can use multi-modal sample data labeled with tree and line positions, and the model parameters are optimized through a back propagation algorithm to gradually learn the feature differences between trees and lines, thereby improving the accuracy of segmentation and extraction.

[0064] The advantage of the dual-branch structure is that it simultaneously utilizes the texture semantic information of images and the spatial structure information of point clouds, avoiding the limitations of single modal data, thereby more accurately dividing the target area.

[0065] For example, the image branch performs convolution, pooling, etc. on the enhanced visible light image through CNN to output two-dimensional semantic masks of trees and lines. The point cloud branch encodes the enhanced three-dimensional point cloud through PointNet to extract spatial coordinates and shape features. After the features of the two are fused, the model can output the boundary coordinates of trees and lines in three-dimensional space to form the corresponding regions.

[0066] Specifically, the dual-branch tree barrier identification model includes a first branch, a second branch, a cross-modal guidance module, a feature fusion module, and a tree barrier region generation module.

[0067] In the above step S2, the dual-branch tree barrier identification model is used to perform semantic segmentation and three-dimensional feature extraction on the preprocessed data set to obtain the specific steps of the tree region and the line region as shown in Figure 4 The specific steps are as follows: S201, processing the enhanced image data using the first branch to output two-dimensional semantic masks of trees and lines.

[0068] The first branch can be constructed based on a convolutional neural network (CNN) to extract semantic features from the enhanced image data and locate two-dimensional regions of trees and lines through semantic segmentation; the output two-dimensional semantic mask is a pixel-level classification result, with each pixel labeled as a tree (class 0) or a line (class 1, including components such as conductors, insulators, and towers), providing prior semantic information for subsequent three-dimensional feature extraction.

[0069] In some implementations, the first branch can adopt a U-Net or DeepLab series network, with the input being the enhanced image and the output two-dimensional semantic mask having the same resolution as the input image; during network training, a cross-entropy loss function is used to optimize parameters, with the labels being manually labeled tree and line pixel regions.

[0070] The core role of the first branch is to quickly locate the target region using semantic information such as texture and color of the image, providing a "region of interest" guide for three-dimensional processing and reducing interference from irrelevant regions.

[0071] For example, the enhanced visible light image contains towers, conductors, and trees, and the first branch outputs a two-dimensional semantic mask through multi-layer convolution and deconvolution operations: in the mask, the pixels in the tower and conductor regions are labeled as 1, the pixels in the tree region are labeled as 0, and the background region is labeled as 2, and the background region does not participate in subsequent processing.

[0072] In S202, a two-dimensional semantic mask is mapped to a three-dimensional point cloud space through a cross-modal guidance module to generate a three-dimensional attention mask and a three-dimensional attention weight.

[0073] The cross-modal guidance module is used to associate two-dimensional and three-dimensional features based on a mapping matrix M (obtained through spatial-temporal registration in S1) of the registered image and the registered point cloud; the three-dimensional attention mask is used to filter target points in the point cloud that belong to trees and lines, and the three-dimensional attention weight is used to quantify the importance of the target points in the point cloud.

[0074] In some implementations, the mapping matrix M is calculated based on camera intrinsic parameters (focal length, principal point coordinates) and extrinsic parameters (rotation matrix, translation vector), which can be obtained through a camera calibration tool (such as Zhang Zhengyou's calibration method).

[0075] The process of the cross-modal guidance module obtaining a three-dimensional attention mask and a three-dimensional attention weight of trees and lines is as follows: Obtain the mapping matrix M; Extract a target pixel coordinate set from the two-dimensional semantic mask output by the first branch Combine the target pixel coordinates to meet: wherein, ​​The coordinates in the two-dimensional semantic mask are The pixels belong to category c, where c=0 represents the tree category and c=1 represents the line category; The target pixel coordinate set is mapped by the mapping matrix M. Each point in the graph is converted into 3D point cloud coordinates. 3D point cloud coordinates satisfy: ; Based on the 3D point cloud coordinates, the points in the registered point cloud that belong to... Points that meet the criteria are marked as 1, otherwise they are marked as 0, generating a 3D attention mask. ; The three-dimensional attention weights are calculated based on the confidence level of the two-dimensional semantic mask and the three-dimensional Euclidean distance. The formula for its calculation is: in, This represents the classification confidence of two-dimensional pixels in a two-dimensional semantic mask. This represents the three-dimensional Euclidean distance, calculated based on the coordinates of a three-dimensional point. Indicates the distance attenuation coefficient. This represents the coordinates of the j-th point in the registered point cloud. This represents the attention weight of the j-th point in the registration point cloud.

[0076] The three-dimensional Euclidean distance is calculated based on the three-dimensional point coordinates.

[0077] Distance decay coefficient in 3D attention weight calculation It can be determined based on the average density of the point cloud. For example, when the average spacing of the point cloud is 0.1 meters, Set it to 0.3 meters to ensure that the weight decreases reasonably with distance.

[0078] This module is key to connecting the two-dimensional and three-dimensional branches. By "migrating" the semantic information of the image to the point cloud space, it solves the problem of missing semantic information in point cloud data and guides the three-dimensional feature extraction to focus on the target area.

[0079] S203 utilizes the second branch to process the enhanced point cloud data based on the 3D attention mask and weights, and outputs the 3D geometric features of trees and lines.

[0080] The second branch can be built based on point cloud neural networks (such as PointNet++) to process the enhanced point cloud data according to the three-dimensional attention mask, and capture the spatial morphology and positional relationship of trees and lines through three-dimensional attention weights, and output the three-dimensional geometric features of trees and lines; the three-dimensional geometric features can include spatial information such as the curvature and neighborhood distribution of the point cloud to reflect the three-dimensional structure of trees and lines.

[0081] The process of outputting the three-dimensional geometric features of the trees and the lines by the second branch specifically comprises: down-sampling and normalizing the enhanced point cloud data to obtain simplified point cloud features; masking the three-dimensional attention selecting a target point set from the simplified point cloud features ; point-by-point multiplying the target point set and the three-dimensional attention weight to obtain a weighted point cloud ; the weighted point cloud comprises spatial forms and positional relationships of the trees and the lines; extracting features of the weighted point cloud by using a pre-constructed PointNet++ network to obtain three-dimensional geometric features of the trees and the lines , comprising: outputting local features by a Set Abstraction layer in the pre-constructed PointNet++ network; max-pooling the local features to obtain global features; concatenating the local features and the broadcasted global features to obtain the three-dimensional geometric features of the trees and the lines .

[0082] The second branch focuses on the point cloud area of the trees and the lines through the attention mechanism, avoiding the interference of the background point cloud, and the weighting operation strengthens the feature expression of the key areas (such as the conductor sag point and the top of the tree), which can more accurately extract the three-dimensional geometric features of the trees and the lines, provide high-quality spatial feature input for subsequent cross-modal feature fusion, and thus improve the accuracy of the tree area and the line area division.

[0083] S204, fusing the high-level semantic features and the three-dimensional geometric features of the first branch by the feature fusion module to generate cross-modal fusion features.

[0084] The feature fusion module is used for dimensionally concatenating and recombining the high-level semantic features of the first branch and the three-dimensional geometric features of the second branch to generate cross-modal fusion features that fuse semantic information and spatial information, and compensate for the limitations of single-modal features; the cross-modal fusion features contain texture semantics and spatial structure information, providing more comprehensive feature support for subsequent area division; the high-level semantic features of the first branch are features obtained after mapping matrix conversion of the last feature map before outputting the two-dimensional semantic mask to the three-dimensional point cloud space; The process of generating the cross-modal fusion features that fuse semantic information and spatial information by the feature fusion module specifically comprises: concatenating the high-level semantic features of the first branch , and the three-dimensional semantic features aligned with the simplified point cloud features are obtained by converting to a three-dimensional point cloud space through a mapping matrix M ; The three-dimensional geometric features are aligned with the point number and feature dimension of the three-dimensional semantic features by interpolation or 1x1 convolution . The aligned features are spliced along the feature dimension to obtain spliced features . The spliced features are subjected to nonlinear transformation by a multilayer perceptron (MLP) layer with a ReLU activation function, and cross-modal fusion features are output . where W is a weight matrix and b is a bias.

[0085] The core of feature fusion is to align the dimensions and perform nonlinear transformation to complement the semantic features and spatial features, so as to improve the adaptability of the model to complex scenes.

[0086] In S205, the tree barrier region generation module divides the tree region and the line region based on the cross-modal fusion features.

[0087] The tree barrier region generation module is used to separate the three-dimensional space regions of trees and lines from the point cloud by point cloud clustering and category determination, and to remove noise points in the three-dimensional space region by boundary optimization to enhance the boundary clarity and region integrity of the tree region and the line region.

[0088] In some implementations, the point cloud clustering can use the DBSCAN algorithm, the neighborhood radius is set to 0.3 meters (determined according to 1.5 times the average distance of the point cloud), and the minimum number of points MinPts is set to 5 to ensure the stability of clustering; the category determination is realized by a softmax classifier, and the clustered point cloud is divided into a tree category and a line category based on the cross-modal fusion features; the boundary optimization can use statistical filtering to remove points with a distance from the cluster center exceeding 3 times the standard deviation, and the standard deviation is calculated based on the spatial distribution of the clustered point cloud.

[0089] The core of the tree barrier region generation module is to divide the three-dimensional regions of trees and lines based on the cross-modal fusion features, wherein the DBSCAN algorithm is mainly used for point cloud clustering and belongs to the post-processing step of the model inference stage. In a deep learning model, such an unsupervised clustering algorithm can be embedded as a part of feature processing, and its role is to spatially cluster the cross-modal fusion features output by the model, divide the point clouds with similar features into the same region, and thus separate the three-dimensional space regions of trees and lines.

[0090] In the training stage, the training of the model focuses on the optimization of the parameters of the first branch, the second branch and the feature fusion module. Through learning the feature rules of trees and lines in the sample, the output cross-modal fusion features can accurately reflect the semantic and spatial differences between the two. The DBSCAN algorithm itself does not participate in the learning of the model parameters, and only clusters and divides the features in the inference process after the training is completed, so it will not interfere with the gradient calculation and parameter update in the training stage, ensuring the normal progress of the training process.

[0091] For example, the cross-modal fusion features are clustered by DBSCAN to obtain 8 point clouds, of which 3 are determined as tree class and 5 are determined as line class by a softmax classifier; boundary optimization is performed on each point cloud set to remove noise points, and finally the tree region and line region are output.

[0092] Based on the above technical solution, S2 cooperates with the double-branch tree barrier identification model to first locate the target by extracting image semantic information using the first branch, then associate the three-dimensional point cloud through the cross-modal guidance module, extract spatial features combined with the second branch, and finally fuse multi-modal features to accurately divide the three-dimensional regions of trees and lines. The whole process fully takes advantage of the complementary advantages of image semantic information and point cloud spatial information, greatly improves the accuracy of target region recognition, and lays a reliable spatial positioning foundation for tree barrier risk assessment.

[0093] Step S3, calculate the minimum clearance distance of the tree region and the line region, and generate a tree barrier risk warning when the minimum clearance distance is less than zero.

[0094] The minimum clearance distance represents the shortest straight-line distance between the tree region and the line region, and is a core indicator for measuring the safety distance between the two. If the distance is less than zero, it means that the tree has invaded the safety range of the line.

[0095] In some implementations, the minimum clearance distance is usually determined by calculating the Euclidean distance between the boundary point sets of the two regions, that is, selecting a number of points from the boundary points of the tree region and the line region, calculating the distance between all pairs of points, and taking the minimum value as the result.

[0096] The quantitative calculation of the minimum clearance distance changes the tree barrier risk judgment from qualitative to quantitative, avoiding the subjectivity of manual judgment and improving the accuracy and reliability of the warning.

[0097] For example, 100 feature points are selected from the boundary points of the tree region, and 100 feature points are selected from the boundary points of the line region. The Euclidean distance between the 10000 pairs of points is calculated, and the minimum distance is the minimum clearance distance. If the value is -0.5 meters (less than zero), the system automatically generates a tree barrier risk warning message.

[0098] The process of calculating the minimum clearance distance of the tree region and the line region in the above step S3 specifically includes: S301, a line three-dimensional model is constructed, and a tree region within a preset distance range of the line three-dimensional model is reserved as a tree barrier region.

[0099] The line three-dimensional model is a refined line space model constructed based on the two-dimensional semantic mask output by the first branch and the three-dimensional geometric features output by the second branch, and contains the three-dimensional structures of components such as conductors, towers, and insulators; the tree barrier region is a part of the tree region that may pose a threat to the safety of the line, and by limiting it within a preset distance of the line three-dimensional model, the interference of irrelevant tree regions is reduced.

[0100] The preset distance range is determined according to the voltage level of the line. For example: In some implementations, the construction process of the line three-dimensional model includes: The line two-dimensional semantic mask of the first branch is bound to the line region point cloud through a mapping matrix to obtain a semantic enhanced line point cloud; A Poisson surface reconstruction algorithm is used for surface fitting, a local linear constraint is introduced for the conductor, and a voxel grid is encrypted to complete the tower; wherein the local linear constraint means that in the local neighborhood of the conductor point cloud (such as within a preset radius centered on a certain point), the geometric features such as the normal vector and the curvature of the point cloud are calculated to determine the linear trend that should be met in this region; in the process of solving the Poisson equation, a constraint is imposed on the surface reconstruction result of this region to tend to a straight line or a plane, limiting the excessive bending of the surface, and ensuring that the reconstruction result of each small section of the conductor is consistent with the actual linear form; The transformation matrix is calculated and the time sequence is aligned through ICP algorithm and tower stable feature point matching using multi-frame time sequence data of the unmanned aerial vehicle; wherein the tower stable feature points include but are not limited to the tower top point, the tower and cross arm connection node, and the insulator mounting point; If the conductor sag and tower spacing deviate from the design value by more than a preset threshold (such as a deviation threshold of 0.5 meters, determined according to the line design specification), the point cloud weight is adjusted to obtain a corrected model through a Bayesian optimization algorithm.

[0101] The accuracy of the line three-dimensional model directly affects the accuracy of the determination of the tree barrier region, and its construction process through semantic enhancement, surface reconstruction, and time sequence alignment can ensure high consistency with the actual line structure.

[0102] For example, based on the first branch semantic mask and the second branch point cloud features of a 10kV line, a line three-dimensional model is constructed, the conductor sag deviates from the design value by 0.3 meters, which is less than the 0.5 meter threshold, and no further correction is needed; the tree region within a 5-meter range around the model is marked as a tree barrier region, and then the tree point cloud 10 meters away from the line is removed, reducing the amount of invalid data by 60%.

[0103] S302, hierarchical clustering is performed on the trees in the tree barrier area to obtain a plurality of tree key parts; an arc drop lowest point and an insulator mounting point are extracted from the line three-dimensional model as a reference point set; a K nearest neighbor search algorithm is used to calculate the three-dimensional Euclidean distance between the tree key parts and the reference point set to obtain a static minimum distance .

[0104] The tree key parts refer to parts of the tree that pose a greater threat to the line safety, such as the top of the trunk, the tips of the main branches, etc.; the line reference point set is a feature point in the line that is most likely to come into contact with the tree, such as the conductor arc drop lowest point and the insulator mounting point; the static minimum distance is the shortest spatial distance between the two without external interference (such as wind).

[0105] In some implementations, hierarchical clustering can use the AGNES algorithm (Agglomerative Nesting), and the number of clustering layers is determined according to the height of the tree, such as 3 layers for a tree height of 5-10 meters and 5 layers for a tree height of 10-20 meters, and the distance threshold for each layer of clustering is 0.5 meters, or can be based on the average diameter of the tree branches to set, and finally the key parts such as the trunk, the first main branch, and the second main branch are obtained.

[0106] When extracting the line reference point set, the conductor arc drop lowest point is obtained by fitting the extreme points of the conductor curve in the line three-dimensional model, and the insulator mounting point can be obtained from the tower point cloud in combination with the structural features of the line three-dimensional model in a preset range of the connection node of the tower and the cross arm (the range is determined according to the insulator mounting size in the line design drawing), and the point set having “cylindrical” or “sheet” geometric features (corresponding to the umbrella skirt or connection fitting structure of the insulator) and having a connection transition with the cross arm point cloud and the tower point cloud is searched to obtain the insulator mounting point; Then the KD tree nearest neighbor search algorithm is used to calculate the static minimum distance.

[0107] It should be noted that hierarchical clustering focuses on the key parts of the tree that are prone to contact, avoiding redundancy caused by calculating the whole tree point cloud; the selection of the line reference point set is based on the structural features of the line, and the force-sensitive points and the discharge-prone areas are given priority to ensure that the static distance can reflect the true safety distance.

[0108] For example, a tree barrier area with a tree height of 15 meters is clustered into 5 layers to obtain key parts such as the trunk (1st layer), 3 first main branches (2nd-4th layers), and 2 second main branch tips (5th layer); 3 conductor arc drop lowest points and 2 insulator mounting points are extracted from the line three-dimensional model as the reference point set; the three-dimensional Euclidean distance between the key parts and the reference point set is calculated by the KD tree to obtain a static minimum distance = 2.3 meters.

[0109] S303, calculate the maximum swing amount of the conductor based on historical meteorological data; calculate the flexibility coefficient according to the diameter of the tree branch, multiply the flexibility coefficient by the average maximum wind speed v in the preset period to obtain the maximum swing amount of the branch; calculate the sum of the maximum swing amount of the conductor and the maximum swing amount of the branch to obtain the dynamic maximum distance; the diameter of the tree branch is obtained by extracting the geometric features of the three-dimensional point cloud of the tree region.

[0110] The dynamic maximum swing amount refers to the maximum displacement amount of the conductor and the tree branch under the action of external factors such as wind; the dynamic maximum distance is the sum of the swing amounts of the two, reflecting the distance change under extreme conditions.

[0111] In some implementations, the maximum swing amount of the conductor The calculation formula is: Wherein, k is the conductor model coefficient, such as LGJ-120 conductor k=0.002, according to the conductor material and cross-sectional area through experimental data fitting, v is the average maximum wind speed in the preset period, which can be obtained by taking the meteorological data of the nearest one month, the preset period is set to be smaller than one month, for example, three days, calculate the average wind speed of each three days, take the maximum value, unit m / s.

[0112] Then calculate the flexibility coefficient of the tree branch diameter d : Wherein, a is the basic flexibility coefficient, the default value is 0.8, which is determined by the experiment of similar tree branches; b is the diameter influence coefficient, the default value is 0.1, which is used to reflect the change of branch rigidity with diameter, d is the diameter of the branch, which is obtained by point cloud cylinder fitting; The maximum swing amount of the branch is obtained according to the flexibility coefficient and the average maximum wind speed v :

[0113] The calculation of the dynamic maximum distance considers the influence of environmental factors on the safety distance, which can avoid the risk misjudgment caused by only relying on the static distance, such as the large swing of the branch or the conductor under strong wind weather, which may cause contact.

[0114] For example, the average maximum wind speed v in the preset period is 10 m / s, the k of LGJ-120 conductor is 0.002, then the maximum swing amount of the conductor 0.2 meters; the diameter of a branch in the tree barrier region is d=0.1 meters, the flexibility coefficient a=0.8-0.1x0.1=0.79, the maximum swing amount of the branch =0.79x10=7.9 meters; the dynamic maximum distance =0.2+7.9=8.1 meters.

[0115] S304, subtract the static minimum distance from the dynamic maximum distance to calculate the minimum clearance distance, and generate a tree barrier risk warning when the distance is less than zero.

[0116] The minimum clearance distance is the static minimum distance minus the dynamic maximum distance, which is used to quantitatively evaluate the tree barrier risk; if the value is less than zero, it indicates that even with a certain distance in the static state, the tree and the line may still contact in extreme working conditions, and a warning needs to be issued.

[0117] In some implementations, the calculation formula of the minimum clearance distance is: The warning information can include a risk signal and a specific location.

[0118] The minimum clearance distance realizes dynamic quantitative evaluation of tree barrier risk by combining static and dynamic factors, and is more in line with actual working conditions than pure static distance judgment, which can provide accurate decision-making basis for line operation and maintenance.

[0119] For example, combined with the results of S302 and S303, the static minimum distance = 2.3 meters, the dynamic maximum distance = 8.1 meters, and the minimum clearance distance = 2.3-8.1 =-5.8 meters <0, the system generates a high-risk warning, and sends a warning signal to the terminal platform.

[0120] Based on the above technical solution, S3 filters the tree barrier area by constructing an accurate line three-dimensional model, calculates the static distance by combining hierarchical clustering and reference point extraction, introduces dynamic swing amount to evaluate the influence of extreme working conditions, and finally obtains the quantified minimum clearance distance and generates a warning. The whole process takes into account both static structure and dynamic environmental factors, realizes scientific evaluation and accurate warning of tree barrier risk, and provides reliable technical support for safe operation and maintenance of distribution network lines.

[0121] Based on the above steps S1-S3, the tree barrier identification method for distribution network lines based on artificial intelligence provided in the present application integrates multi-modal data acquisition, artificial intelligence processing and quantitative risk evaluation, which not only overcomes the inefficiency of traditional manual inspection, but also solves the limitations of single data identification, and can more accurately and efficiently find the tree barrier risk in the distribution network line, providing strong technical support for line operation and maintenance.

[0122] The above describes the scheme of the embodiments of the application mainly from the perspective of device implementation. It can be understood that each device, for example, an artificial intelligence-based power distribution network line tree barrier identification device, includes at least one of the corresponding hardware structure and software module for executing each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0123] Embodiment 2 This embodiment is based on the above embodiment 1, and discloses an artificial intelligence-based power distribution network line tree barrier identification system. This embodiment can divide the functional modules of an artificial intelligence-based power distribution network line tree barrier identification system according to the method examples in the above embodiment 1. For example, each functional module can be divided according to each function, or two or more functions can be integrated in one processing module, such as Figure 2 As shown in the figure, the system in this embodiment includes a data acquisition module, a data processing module, and a risk early warning module; wherein, The data acquisition module is configured to acquire multi-modal data of the power transmission line corridor by using a UAV, including three-dimensional point cloud data and visible light images. The data processing module is configured to perform spatio-temporal registration on the acquired multi-modal data, and enhance the registered multi-modal data by using a generative adversarial network to obtain a preprocessed data set. The double-branch tree barrier identification model is pre-constructed based on a deep learning algorithm. The risk early warning module is configured to calculate the minimum clearance distance of the tree region and the line region, and generate a tree barrier risk warning when the minimum clearance distance is less than zero.

[0124] Embodiment 3 The embodiment is based on the above-mentioned embodiment 1-2, and discloses an artificial intelligence-based power distribution network line tree barrier identification device. According to the method example of the above-mentioned embodiment 1, the functional units of the artificial intelligence-based power distribution network line tree barrier identification device are divided. For example, each functional unit can be divided according to each function, or two or more functions can be integrated in one processing unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit. It should be noted that the division of the units in the embodiment is illustrative, and is only a logical functional division. In actual implementation, another division mode can be used.

[0125] In the case of using integrated units, Figure 5 A possible structural diagram of an artificial intelligence-based power distribution network line tree barrier identification device (denoted as an artificial intelligence-based power distribution network line tree barrier identification device 50) involved in the above-mentioned embodiment is shown. The artificial intelligence-based power distribution network line tree barrier identification device 50 includes a processing unit 501 and a communication unit 502, and can also include a storage unit 503. Figure 5 The structural diagram shown can be used to illustrate the structure of the artificial intelligence-based power distribution network line tree barrier identification device involved in the above-mentioned embodiment.

[0126] When Figure 5 When the structural diagram shown is used to illustrate the structure of the artificial intelligence-based power distribution network line tree barrier identification device involved in the above-mentioned embodiment, the processing unit 501 is used to control and manage the actions of the artificial intelligence-based power distribution network line tree barrier identification device, the communication unit 502 is used for communication between the artificial intelligence-based power distribution network line tree barrier identification device and other devices, and the storage unit 503 is used to store the program code and data of the artificial intelligence-based power distribution network line tree barrier identification device.

[0127] For example, the communication unit 502 is used to receive the power line corridor multi-modal data collected by the unmanned aerial vehicle, and send the generated tree barrier risk warning information to the related terminal device. The processing unit 501 is used to perform spatio-temporal registration on the received multi-modal data, enhance the registered multi-modal data through a generative adversarial network to obtain a preprocessed data set, perform semantic segmentation and three-dimensional feature extraction on the preprocessed data set by using a double-branch tree barrier identification model to obtain a tree region and a line region, calculate the minimum clearance distance of the tree region and the line region, and generate a tree barrier risk warning when the distance is less than zero.

[0128] In a possible implementation, the processing unit 501 is further configured to construct a line three-dimensional model, perform hierarchical clustering on a tree barrier region to obtain a key part of a tree, extract a reference point set from the line three-dimensional model, and calculate a dynamic maximum swing amount of a conductor and a tree branch.

[0129] In a possible implementation, the communication unit 502 is further configured to receive line design values and other related information in the power grid line design drawing; and the processing unit 501 is further configured to perform optimization adjustment on the line model after timing correction according to the received line design values, to obtain a corrected line three-dimensional model.

[0130] The processing unit 501 can be a processor or a controller, and the communication unit 502 can be a communication interface, a transceiver, a transceiver, a transceiver circuit, a transceiver device, etc. The communication interface is a general term, and can include one or more interfaces. The storage unit 503 can be a memory. When the power grid line tree barrier identification device 50 based on artificial intelligence is a chip, the processing unit 501 can be a processor or a controller, the communication unit 502 can be an input interface and / or an output interface, a pin or a circuit, etc. The storage unit 503 can be a storage unit (for example, a register, a cache, etc.) in the chip, or can be a storage unit (for example, a read-only memory (ROM), a random access memory (RAM), etc.) located outside the chip.

[0131] The communication unit can also be referred to as a transceiving unit. The antenna and control circuit with transceiving function in the power grid line tree barrier identification device 50 based on artificial intelligence can be regarded as a communication unit 502 of the power grid line tree barrier identification device 50 based on artificial intelligence, and the processor with processing function can be regarded as a processing unit 501 of the power grid line tree barrier identification device 50 based on artificial intelligence. Optionally, the device for realizing the receiving function in the communication unit 502 can be regarded as a communication unit, and the communication unit is configured to perform the receiving steps in the embodiments of the present application, and the communication unit can be a receiver, a receiver, a receiving circuit, etc. The device for realizing the sending function in the communication unit 502 can be regarded as a sending unit, and the sending unit is configured to perform the sending steps in the embodiments of the present application, and the sending unit can be a transmitter, a transmitter, a sending circuit, etc.

[0132] Figure 5The integrated units in the above-mentioned embodiments, if realized in the form of software function modules and sold or used as independent products, can be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the present application essentially or in other words the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in the embodiments of the present application. The storage medium storing the computer software product includes a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0133] Embodiment 4 This embodiment is based on the above-mentioned embodiments 1-3, and discloses an artificial intelligence-based power distribution network line tree barrier identification device (denoted as an artificial intelligence-based power distribution network line tree barrier identification device 60). The hardware structure diagram is shown in Figure 6 The artificial intelligence-based power distribution network line tree barrier identification device 60 includes a processor 601, a memory 602 connected with the processor 601, and a transceiver 603.

[0134] The processor 601, the memory 602, and the transceiver 603 are connected through a bus. The transceiver 603 is used for communication with other devices or communication networks. The transceiver 603 can include a transmitter and a receiver. The device for realizing the receiving function in the transceiver 603 can be regarded as a receiver, and the receiver is used for executing the receiving steps in the embodiments of the present application. The device for realizing the sending function in the transceiver 603 can be regarded as a transmitter, and the transmitter is used for executing the sending steps in the embodiments of the present application.

[0135] Figure 6 The structure diagram shown can be used to illustrate the structure of the above-mentioned artificial intelligence-based power distribution network line tree barrier identification device, and can also illustrate a system chip in an artificial intelligence-based power distribution network line tree barrier identification device. In this case, the actions performed by the above-mentioned artificial intelligence-based power distribution network line tree barrier identification device can be realized by the system chip, and the specific actions performed can be referred to in the above, which will not be described here.

[0136] In the implementation process, each step in the method provided by embodiment 1 can be completed by the integrated logic circuit of hardware in the processor or the instructions in the form of software. The steps of the method disclosed in combination with embodiment 1 can be directly embodied as hardware processor execution completion, or executed by hardware and software modules in the processor.

[0137] The processor in the embodiment can include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, and the like, each of which is a computing device running software and can include one or more cores for executing software instructions to perform operations or processing. The processor can be a separate semiconductor chip, or can be integrated with other circuits as a semiconductor chip, for example, can be integrated with other circuits (such as coding and decoding circuits, hardware acceleration circuits, or various bus and interface circuits) to form a SoC (system on chip), or can be integrated as a built-in processor in an ASIC. The ASIC integrated with the processor can be packaged separately or packaged together with other circuits. In addition to including cores for executing software instructions to perform operations or processing, the processor can further include necessary hardware accelerators, such as field programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits implementing special logic operations.

[0138] The memory in the embodiment can include at least one of the following types: read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, and electrically erasable programmable read-only memory (EEPROM). In some scenarios, the memory can also be a compact disc read-only memory (CD-ROM) or other optical disk storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited thereto.

[0139] Embodiment 5 The embodiment provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method in Embodiment 1.

[0140] Embodiment 6 The embodiment provides a computer program product containing instructions, which, when executed on a computer, cause the computer to perform the method in the above embodiment 1.

[0141] Embodiment 7 The embodiment provides a chip, which comprises a processor and an interface circuit, the interface circuit is coupled with the processor, the processor is used for running a computer program or instructions to implement the method in the above embodiment 1, and the interface circuit is used for communicating with other modules outside the chip.

[0142] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by a software program, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product comprises one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as one or more servers, data centers, etc. integrated with one or more media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (solid state disk, SSD)) and the like.

[0143] Although the present application is described herein in conjunction with various embodiments, other variations of the disclosed embodiments can be understood and implemented by those skilled in the art with reference to the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "one" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. Some measures are described in mutually different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0144] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements shall be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for identifying tree obstacles in power distribution networks based on artificial intelligence, characterized in that, The method includes: Multimodal data of power transmission line corridors, including 3D point cloud data and visible light images, were collected using drones; Spatiotemporal registration is performed on the collected multimodal data, and the registered multimodal data is enhanced by a generative adversarial network to obtain a preprocessed dataset; A bi-branch tree obstacle recognition model is used to perform semantic segmentation and 3D feature extraction on the preprocessed dataset to obtain tree regions and line regions; the bi-branch tree obstacle recognition model is pre-built based on a deep learning algorithm. Calculate the minimum clearance distance between tree areas and road areas, and generate a tree obstacle risk warning when the minimum clearance distance is less than zero.

2. The method for identifying tree obstacles in power distribution networks based on artificial intelligence according to claim 1, characterized in that, The process of spatiotemporal registration of the collected multimodal data includes: The 3D point cloud data in the multimodal data is time-stamped and the sensor acquisition delay is eliminated using an interpolation compensation algorithm. Extract two-dimensional feature points from visible light images and geometric feature points from three-dimensional point clouds; Based on spatial constraints, two-dimensional feature points and geometric feature points are matched, and feature point pairs whose projection error exceeds a preset distance threshold are removed after matching, to obtain registered multimodal data; wherein, the registered multimodal data includes a registered image, a registered point cloud, and a mapping matrix between the registered image and the registered point cloud; The registered multimodal data is converted to a unified geographic coordinate system.

3. The method for identifying tree obstacles in power distribution networks based on artificial intelligence according to claim 2, characterized in that, The generative adversarial network includes a Pix2PixHD network and a PointGAN network. The process of enhancing the registered multimodal data through the generative adversarial network includes: The registered image is processed using a Pix2PixHD network to obtain enhanced image data; The registered point cloud is processed using a PointGAN network to fill in the missing regions and obtain enhanced point cloud data. The preprocessed dataset includes enhanced image data and enhanced point cloud data.

4. The method for identifying tree obstacles in power distribution networks based on artificial intelligence according to claim 3, characterized in that, The dual-branch tree obstacle recognition model includes a first branch, a second branch, a cross-modal guidance module, a feature fusion module, and a tree obstacle region generation module; wherein... The first branch is used to extract features from the enhanced visible light image data and output a two-dimensional semantic mask of trees and lines and their confidence scores; the lines include conductors, insulators and towers; The cross-modal guidance module is used to map the two-dimensional semantic mask output by the first branch to the three-dimensional point cloud space according to the mapping matrix, and obtain the three-dimensional attention mask and three-dimensional attention weight of trees and lines. The second branch is used to process the enhanced point cloud data according to the three-dimensional attention mask, and capture the spatial morphology and positional relationship of trees and lines through three-dimensional attention weights, and output the three-dimensional geometric features of trees and lines. The feature fusion module is used to perform dimensional splicing and feature recombination of the high-level semantic features of the first branch and the three-dimensional geometric features of the second branch to generate cross-modal fusion features that fuse semantic and spatial information; the high-level semantic features of the first branch are the features obtained by transforming the last layer feature map before outputting the two-dimensional semantic mask into the three-dimensional point cloud space through a mapping matrix. The tree barrier region generation module is used to perform point cloud clustering and category determination based on the cross-modal fusion features, to divide the three-dimensional spatial regions of trees and lines respectively, and to remove noise points in the three-dimensional spatial regions through boundary optimization, and output tree regions and line regions.

5. The method for identifying tree obstacles in power distribution networks based on artificial intelligence according to claim 4, characterized in that, The process by which the cross-modal guidance module obtains the 3D attention mask and 3D attention weights for trees and lines is as follows: Obtain the mapping matrix M; Two-dimensional semantic mask output from the first branch Extract the target pixel coordinate set The target pixel coordinates are combined satisfy: ； in, The coordinates in the two-dimensional semantic mask are The pixels belong to category c, where c=0 represents the tree category and c=1 represents the line category; The target pixel coordinate set is obtained through the mapping matrix M. Each point in the graph is converted into 3D point cloud coordinates. The three-dimensional point cloud coordinates satisfy: ； Based on the three-dimensional point cloud coordinates, the points in the registration point cloud belonging to... Points that meet the criteria are marked as 1, otherwise they are marked as 0, generating a 3D attention mask. ; The three-dimensional attention weights are calculated based on the confidence level of the two-dimensional semantic mask and the three-dimensional Euclidean distance. The three-dimensional Euclidean distance is calculated based on the three-dimensional point coordinates.

6. The method for identifying tree obstacles in power distribution lines based on artificial intelligence according to claim 4, characterized in that, The process of the second branch outputting the three-dimensional geometric features of trees and lines includes: The enhanced point cloud data is downsampled and normalized to obtain simplified point cloud features; Using 3D attention mask Target point set is selected from the simplified point cloud features. ; The target point set With the three-dimensional attention weights Perform point-by-point multiplication to obtain the weighted point cloud. The weighted point cloud includes the spatial morphology and positional relationships of trees and lines. The pre-built PointNet++ network is used to extract features from the weighted point cloud to obtain the 3D geometric features of trees and lines. ,include: Local features are output through the Set Abstraction layer in a pre-built PointNet++ network; Max pooling is used to extract global features from local features; The local features are concatenated with the broadcast global features to obtain the three-dimensional geometric features of the trees and lines. .

7. The method for identifying tree obstacles in power distribution networks based on artificial intelligence according to claim 4, characterized in that, The process by which the feature fusion module generates cross-modal fusion features that fuse semantic and spatial information includes: High-level semantic features of the first branch By transforming the point cloud into a 3D point cloud space through a mapping matrix M, 3D semantic features aligned with the simplified point cloud features are obtained. ; 3D geometric features are obtained through interpolation or 1×1 convolution. With three-dimensional semantic features Align the number of points and feature dimensions; The aligned features are concatenated along the feature dimensions to obtain the concatenated features. ; The spliced ​​features are processed by a multilayer perceptron (MLP) layer with ReLU activation. Perform nonlinear transformation to output cross-modal fusion features .

8. The method for identifying tree obstacles in power distribution networks based on artificial intelligence according to claim 4, characterized in that, The process of calculating the minimum clearance distance between tree areas and road areas includes: Tree areas within a preset distance range of the route's 3D model are retained to obtain tree barrier areas; the route's 3D model is constructed based on the outputs of the first and second branches. The trees in the tree barrier area are hierarchically clustered to obtain multiple key tree parts; The lowest point of sag and the insulator installation point are extracted from the three-dimensional model of the line as a set of reference points; The K-nearest neighbor search algorithm is used to calculate the three-dimensional Euclidean distance between key parts of the tree and the set of reference points, and the static minimum distance is obtained. ; Calculate the maximum sway of the conductor based on historical meteorological data; The flexibility coefficient is calculated based on the diameter of the tree branches, and the flexibility coefficient is multiplied by the average maximum wind speed v within a preset period to obtain the maximum sway of the branches; the diameter of the tree branches is obtained by extracting the geometric features of the three-dimensional point cloud of the tree area; Calculate the sum of the maximum swing of the conductor and the maximum swing of the branch to obtain the dynamic maximum distance; The minimum clearance distance is obtained by subtracting the maximum dynamic distance from the minimum static distance.

9. The method for identifying tree obstacles in power distribution networks based on artificial intelligence according to claim 8, characterized in that, The process of constructing the three-dimensional model of the line includes: By using a mapping matrix, the position labels of the lines in the two-dimensional semantic mask output by the first branch are bound to the line region to obtain a semantically enhanced line point cloud; The Poisson surface reconstruction algorithm is used to perform surface fitting on the semantically enhanced line point cloud to obtain an initial reconstruction model; Using multi-frame time-series data from UAVs, stable feature points of the towers are matched through an iterative nearest-point algorithm, and the transformation matrix of the line point cloud in adjacent frames is calculated; the stable feature points include tower fixed points, tower-crossarm connection nodes, and insulator installation points; The initial reconstruction model is time-aligned according to the transformation matrix to obtain the time-corrected line model. If the deviation between the conductor sag and tower spacing in the time-corrected line model and the line design value exceeds a preset threshold, the point cloud weights of the time-corrected line model are adjusted using a Bayesian optimization algorithm to obtain the final three-dimensional line model; the line design value is obtained from the distribution network line design drawings.

10. A tree obstacle identification system for power distribution lines based on artificial intelligence, characterized in that, include: The module comprises a data acquisition module, a data processing module, and a risk warning module; among which, The data acquisition module is used to collect multimodal data of the power transmission line corridor via drone, including three-dimensional point cloud data and visible light images; The data processing module is used to perform spatiotemporal registration on the collected multimodal data and to obtain a preprocessed dataset by generating an adversarial network to enhance the registered multimodal data. A bi-branch tree obstacle recognition model is used to perform semantic segmentation and 3D feature extraction on the preprocessed dataset to obtain tree regions and line regions; the bi-branch tree obstacle recognition model is pre-built based on a deep learning algorithm. The risk warning module is used to calculate the minimum clearance distance between tree areas and line areas, and to generate a tree obstacle risk warning when the minimum clearance distance is less than zero.

Images