Electric power tower and environment point cloud classification method and system supporting safe distance calculation

Abstract

The invention discloses an electric power tower and environment point cloud classification method and system supporting safe distance calculation, and the method comprises the steps: carrying out the space segmentation of laser point clouds of an electric power tower and peripheral facilities, and mapping the space coordinates xyz and three component color values RGB of each space unit to each laser point; sampling the laser points to generate a plurality of spatial sampling subsets; establishing an original matrix based on the space coordinates xyz of the laser points and the three component color values RGB; converting an original matrix generated by sample points in each spatial sampling subset and a preset number of spatial nearest neighbor points through a spatial conversion unit to generate a feature conversion matrix; training the deep learning network model based on the feature transformation matrix to generate a multi-modal input support deep learning model; acquiring multi-channel features extracted based on an original matrix on each convolution level in the multi-modal input deep learning model, and classifying the point clouds of the electric power tower and the surrounding facilities thereof; and based on a point cloud classification result, carrying out minimum safety distance calculation.

Description

Technical Field

[0001] The present invention relates to the technical field of live working safety protection, and more specifically, to a method and system for classifying power towers and environmental point clouds that support safety distance calculation. Background Art

[0002] The point data set of the product appearance surface obtained by measuring instruments such as radar laser scanning is called point cloud. Laser scanning technology has made great progress and has been widely used in fields such as CAD / CAM. The scanned spatial data point set, including information such as three-dimensional coordinates and colors, has the characteristics of scattered distribution and large data volume. Manual classification is very time-consuming, and it is difficult to accurately identify the category of laser point cloud in some locations with high density and complex topological relationships.

[0003] In recent years, thanks to the rapid development of Extra High Voltage (EHV) and Ultra High Voltage (UHV) in my country, the power security of the national economy has been effectively guaranteed. At the same time, the rapid growth of power load has also posed new challenges to power grid security, which has become a core element to ensure the stable operation and power supply reliability of the power system. In order to reconstruct the live working site and conduct rapid measurement and real-time monitoring based on the on-site three-dimensional information, laser point cloud data has been widely used in power grid operation monitoring. However, how to obtain the minimum safe distance to the ground, that is, to obtain the minimum distance to the ground potential element according to the coordinates of any given point, requires knowing the set of element points with ground potential in the laser point cloud. This requires determining the set of power towers and ground elements before distance calculation. By classifying all the laser point cloud data collected on site, the power towers can be separated from the conductors, and the ground elements covered by the surrounding environment of the live working, such as houses, trees and the ground, can also be effectively identified. This provides an effective means for some live working safety protection work based on distance measurement. Among them, when selecting the insulated soft ladder method and the drone combined with the electric lifting device operation method to enter the equipotential position, it is necessary to pre-calculate the minimum distance between the soft ladder and the insulating rope hanging point and the ground potential element. If the set of towers and ground elements can be extracted from the laser point cloud, the minimum safe distance to the ground can be quickly obtained based on the mature minimum distance calculation method, thereby effectively ensuring the real-time distance measurement of the operators.

[0004] However, there is a problem of low efficiency in manual classification when extracting poles and ground elements from large-scale laser point clouds, and it is difficult to directly use existing deep learning methods to cope with the problem of large scale of laser point clouds in operation scenes. Live working scenes generally cover at least all visible elements between two poles, such as trees, ground, roads, poles and wires. Methods based on manual classification can divide these elements into fine-grained levels with high precision, but this method will consume a lot of manpower and material resources. In addition, considering the rapid update of laser point clouds of power grids, these large amounts of manpower expenditures for laser point cloud data classification every year will significantly increase the maintenance cost of power grids. Deep learning methods use convolutional neural networks and Transformer methods to extract spatial or texture features of point clouds, providing an automated means for object segmentation of indoor and outdoor laser point clouds. However, due to the high computational cost of spatial convolution, existing deep learning methods are usually used to process scenes with millions of laser points, and are difficult to directly apply when processing larger-scale live working scenes. In addition, existing open source deep learning models mostly focus on the spatial relationship between points, and do not effectively consider the pixel color information on the laser points, making the model ineffective in segmenting complex objects such as poles.

[0005] Therefore, a technology is needed to automatically identify multi-modal power poles and surrounding facilities. Summary of the invention

[0006] The technical solution of the present invention provides a method and system for classifying power poles and environmental point clouds that support safe distance calculation, so as to solve the problem of how to automatically identify multi-modal power poles and surrounding facilities.

[0007] In order to solve the above problems, the present invention provides a method for classifying power towers and environmental point clouds that supports safe distance calculation, the method comprising:

[0008] The laser point cloud of the power tower and its surrounding facilities is spatially segmented, and the spatial coordinates xyz and three-component color values ​​RGB of each spatial unit after spatial segmentation are mapped to each laser point;

[0009] Sampling the laser point of each spatial unit to generate a plurality of spatial sampling subsets having the same number of samples as the sampled number; establishing an original matrix based on the spatial coordinates xyz and the three component color values ​​RGB of the laser point;

[0010] The original matrix generated by the sample points and a preset number of spatial nearest neighbor points in each spatial sampling subset is transformed by the constructed spatial transformation unit to generate a feature transformation matrix;

[0011] Training a deep learning network model based on the feature conversion matrix to generate a multimodal input support deep learning model that takes the feature conversion matrix as input and known classifications as output;

[0012] Acquire multi-channel features extracted based on the original matrix at each convolution level in the multimodal input support deep learning model, aggregate the multi-channel features into aggregate features, and classify the power pole tower and its surrounding facility point cloud through the aggregate features; calculate the minimum safety distance based on the classification results of the power pole tower and its surrounding facility point cloud.

[0013] Preferably, the laser point cloud of the power tower and its surrounding facilities is spatially segmented, and the spatial coordinates xyz and the three-component color values ​​RGB of each spatial unit after the spatial segmentation are mapped to each laser point; the laser point of each spatial unit is sampled to generate multiple spatial sampling subsets with the same number of sampled samples, including:

[0014] The laser point cloud of the power tower and its surrounding facilities is spatially segmented, and the laser points of each spatial unit after the spatial segmentation are randomly sampled, and the number of sampling points is less than a preset percentage of the total number of laser points;

[0015] Each sampling point is extended by a preset unit distance along the xy coordinate direction of each spatial unit to form a square sampling;

[0016] Project the laser points on the square sampling to obtain a laser point set under the rectangular parallelepiped MBR, and collect sample points of preset data from the laser point set under the rectangular parallelepiped MBR as a spatial sampling subset.

[0017] Preferably, the constructed spatial conversion unit converts the original matrix generated by the sample points in each spatial sampling subset and a preset number of spatial nearest neighboring points to generate a feature conversion matrix, including:

[0018] The three-dimensional spatial coordinates xyz and three-dimensional three-component color values ​​RGB generated by the sample points and a preset number of spatial nearest points are converted into a six-dimensional feature conversion matrix.

[0019] Preferably, the multimodal input supporting deep learning model comprises a seven-layer network, wherein the first four layers are edge convolution units, and the edge convolution units extract multi-channel features through a multi-layer perception network at different convolution levels and perform pooling output to form a multi-layer perception network output of the next convolution level;

[0020] The last three layers in the seven-layer network are aggregation layers, which are used to aggregate multi-channel features into aggregate features, and the aggregate features include context information at different levels.

[0021] Preferably, the method further comprises: calculating a minimum safety distance based on the classification result of the power pole tower and its surrounding facility point cloud.

[0022] Preferably, the calculating of the minimum safety distance includes:

[0023] Based on the classified ground, pole tower, building and vegetation information of the power pole tower and its surrounding facilities, the minimum safe distance of the target space unit to the ground is calculated.

[0024] Preferably, the calculating of the minimum safety distance of the target space unit to the ground includes:

[0025] The spatial coordinates xyz and three component color values ​​RGB of the laser point cloud are used as the input of the multimodal input support deep learning model to determine the category of the segmented laser point cloud;

[0026] Laser points with segmentation categories of towers, buildings, ground and trees are selected, and the nearest laser point is obtained for a given spatial query coordinate point through a three-dimensional spatial indexing method. The distance between the spatial query coordinate point and the nearest laser point is used as the minimum safe distance to the ground.

[0027] According to another aspect of the present invention, the present invention provides a power pole tower and environment point cloud classification system that supports safety distance calculation, the system comprising:

[0028] The initial unit is used to spatially segment the laser point cloud of the power tower and its surrounding facilities, and map the spatial coordinates xyz and the three-component color values ​​RGB of each spatial unit after spatial segmentation to each laser point;

[0029] A sampling unit is used to sample the laser point of each spatial unit to generate multiple spatial sampling subsets with the same number of samples; an original matrix is ​​established based on the spatial coordinates xyz and the three component color values ​​RGB of the laser point;

[0030] A conversion unit, used to convert an original matrix generated by sample points and a preset number of spatial nearest neighbor points in each spatial sampling subset through the constructed spatial conversion unit to generate a feature conversion matrix;

[0031] A training unit, used for training the deep learning network model based on the feature conversion matrix, and generating a multi-modal input supporting deep learning model having the feature conversion matrix as input and known classifications as output;

[0032] A result unit is used to obtain multi-channel features extracted based on the original matrix at each convolution level in the multimodal input support deep learning model, aggregate the multi-channel features into aggregate features, and classify the power pole tower and its surrounding facility point cloud through the aggregate features; based on the classification results of the power pole tower and its surrounding facility point cloud, calculate the minimum safety distance.

[0033] Preferably, the sampling unit is used to sample the laser points of each spatial unit to generate a plurality of spatial sampling subsets having the same number of samples as the sampled samples, and is also used to:

[0034] The laser points of each space unit after the space division are randomly sampled, and the number of sampling points is less than a preset percentage of the total number of laser points;

[0035] Each sampling point is extended by a preset unit distance along the xy coordinate direction of each spatial unit to form a square sampling;

[0036] Project the laser points on the square sampling to obtain a laser point set under the rectangular parallelepiped MBR, and collect sample points of preset data from the laser point set under the rectangular parallelepiped MBR as a spatial sampling subset.

[0037] Preferably, the conversion unit is used to convert the original matrix generated by the sample points and a preset number of spatial nearest neighbor points in each spatial sampling subset through the constructed spatial conversion unit to generate a feature conversion matrix, and is also used to:

[0038] The three-dimensional spatial coordinates xyz and three-dimensional three-component color values ​​RGB generated by the sample points and a preset number of spatial nearest points are converted into a six-dimensional feature conversion matrix.

[0039] Preferably, the multimodal input supporting deep learning model comprises a seven-layer network, wherein the first four layers are edge convolution units, and the edge convolution units extract multi-channel features through a multi-layer perception network at different convolution levels and perform pooling output to form a multi-layer perception network output of the next convolution level;

[0040] The last three layers in the seven-layer network are aggregation layers, which are used to aggregate multi-channel features into aggregate features, and the aggregate features include context information at different levels.

[0041] Preferably, the result unit is used to calculate the minimum safety distance based on the classification of the power pole tower and its surrounding facility point cloud.

[0042] Preferably, the result unit is used to calculate the minimum safety distance and is also used to:

[0043] Based on the ground, poles, towers, buildings and vegetation information after classification of the point cloud of the power poles and towers and their surrounding facilities, the minimum safe distance of the target space unit to the ground is calculated.

[0044] Preferably, the result unit is used to calculate the minimum safety distance of the target space unit to the ground, and is also used to:

[0045] The spatial coordinates xyz and three component color values ​​RGB of the laser point cloud are used as the input of the multimodal input support deep learning model to determine the category of the segmented laser point cloud;

[0046] Laser points with segmentation categories of towers, buildings, ground and trees are selected, and the nearest laser point is obtained for a given spatial query coordinate point through a three-dimensional spatial indexing method. The distance between the spatial query coordinate point and the nearest laser point is used as the minimum safe distance to the ground.

[0047] The technical solution of the present invention provides a method and system for classifying electric pole towers and environmental point clouds supporting safe distance calculation, wherein the method comprises: spatially segmenting the laser point cloud of the electric pole tower and its surrounding facilities, mapping the spatial coordinates xyz and three-component color values ​​RGB of each spatial unit after spatial segmentation to each laser point; sampling the laser points of each spatial unit to generate multiple spatial sampling subsets with the same number of sampled samples; establishing an original matrix based on the spatial coordinates xyz and three-component color values ​​RGB of the laser point; transforming the original matrix generated by the sample points and a preset number of spatial nearest neighbor points in each spatial sampling subset through a constructed spatial conversion unit to generate a feature conversion matrix; training a deep learning network model based on the feature conversion matrix to generate a multi-modal input supporting deep learning model that takes the feature conversion matrix as input and known classifications as output; obtaining multi-channel features extracted based on the original matrix at each convolution level in the multi-modal input supporting deep learning model, aggregating the multi-channel features into aggregated features, and classifying the point cloud of the electric pole tower and its surrounding facilities through the aggregated features; and calculating the minimum safe distance based on the classification results of the point cloud of the electric pole tower and its surrounding facilities. The technical solution of the present invention proposes a method and system for classifying power towers and environmental point clouds that support safe distance calculation, which solves the problem of how to segment the ground and towers of laser point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] A more complete understanding of exemplary embodiments of the present invention may be obtained by referring to the following drawings:

[0049] Figure 1 A flow chart of a method for classifying power towers and environmental point clouds supporting safety distance calculation according to a preferred embodiment of the present invention;

[0050] Figure 2A flowchart of a method for processing classified laser point cloud data according to a preferred embodiment of the present invention;

[0051] Figure 3 A schematic diagram of three-dimensional visualization of laser point cloud data of live working site for deep learning according to a preferred embodiment of the present invention;

[0052] Figure 4 Schematic diagram of a multi-modal spatial division method of laser point cloud at live working site according to a preferred embodiment of the present invention, in which data is divided into 32 subsets according to the xy plane, and each subset is extended in the z-axis direction to obtain a spatial three-dimensional polygonal schematic diagram;

[0053] Figure 5 It is an MBR view of a multi-modal space partitioning method of laser point cloud at a live working site according to a preferred embodiment of the present invention;

[0054] Figure 6 Schematic diagram of two types of channels of multimodal data aligned according to spatial points according to a preferred embodiment of the present invention, wherein (a) is an RGB channel and (b) is a spatial point coordinate diagram;

[0055] Figure 7 According to the multimodal data sampling method of the preferred embodiment of the present invention according to the spatial point alignment, a random sampling is generated to achieve the sampling of two types of channels, (a) to (c) are RGB channel sampling; (b) to (d) are spatial coordinate sampling. Among them, the point IDs of the two channels correspond one to one;

[0056] Figure 8 A structural diagram of a deep learning neural network considering local multimodal features of elements according to a preferred embodiment of the present invention, wherein the upper branch is used for classification of a single laser point cloud, and the lower branch is the focus of the present invention, a schematic diagram of user laser point cloud segmentation;

[0057] Fig. 9 According to a preferred embodiment of the present invention Figure 7 Schematic diagram of spatial transformation unit and edge convolution unit in neural network;

[0058] Fig.10 A schematic diagram of the structure of a processing system for classified laser point cloud data according to a preferred embodiment of the present invention; and

[0059] Fig.11 It is a structural diagram of a power tower and environmental point cloud classification system supporting safety distance calculation according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0060] Now, exemplary embodiments of the present invention are described with reference to the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to disclose the present invention in detail and completely and to fully convey the scope of the present invention to those skilled in the art. The terms used in the exemplary embodiments shown in the accompanying drawings are not intended to limit the present invention. In the accompanying drawings, the same units / elements are marked with the same reference numerals.

[0061] Unless otherwise specified, the terms (including technical terms) used herein have the commonly understood meanings to those skilled in the art. In addition, it is understood that the terms defined in commonly used dictionaries should be understood to have the same meanings as those in the context of the relevant fields, and should not be understood as idealized or overly formal meanings.

[0062] Figure 1 The present invention is a flowchart of a method for classifying power towers and environmental point clouds that supports safe distance calculation according to a preferred embodiment of the present invention.

[0063] The present invention provides a multi-modal electric power pole tower and surrounding facilities automatic identification method and system to solve the problem of how to segment the ground and the pole tower of laser point cloud data, and support the calculation of the minimum safe distance to the ground in combination with the segmentation result.

[0064] The present invention proposes a multi-modal electric power pole tower and surrounding facilities automatic identification method to solve the problem of how to segment the ground and the pole tower of laser point cloud data, and support the calculation of the minimum safe distance to the ground in combination with the segmentation result.

[0065] like Figure 1 As shown, the present invention provides a multi-modal power tower and surrounding facilities automatic identification method, the method comprising:

[0066] Step 101: spatially segment the laser point cloud of the power tower and its surrounding facilities, and map the spatial coordinates xyz and three-component color values ​​RGB of each spatial unit after spatial segmentation to each laser point;

[0067] Step 102: Sampling the laser point of each spatial unit to generate a plurality of spatial sampling subsets having the same number of samples as the sampled number; establishing an original matrix based on the spatial coordinates xyz and the three component color values ​​RGB of the laser point;

[0068] Preferably, the laser point cloud of the target object is spatially segmented, and the spatial coordinates xyz and the three-component color values ​​RGB of each spatial unit after the spatial segmentation are mapped to each laser point; the laser point of each spatial unit is sampled to generate multiple spatial sampling subsets with the same number of sampled samples, including:

[0069] The laser point cloud of the power tower and its surrounding facilities is spatially segmented, and the laser points of each spatial unit after the spatial segmentation are randomly sampled, and the number of sampling points is less than a preset percentage of the total number of laser points;

[0070] Each sampling point is extended by a preset unit distance along the xy coordinate direction of each spatial unit to form a square sampling;

[0071] Project the laser points on the square sampling to obtain the laser point set under the rectangular MBR, and collect the sample points of the preset data from the laser point set under the rectangular MBR as a spatial sampling subset.

[0072] The present invention is aimed at the spatial segmentation of laser point clouds of live working scenes. By randomly sampling the laser points of the laser point cloud scene, a certain number (such as S) of sampling points are obtained to ensure that the average number of point clouds under the S object segmentation is significantly less than U / N (N defaults to 4096) of the total number of laser points U; the present invention classifies the point clouds of power towers and their surrounding facilities. The point clouds include the ground, towers, buildings and vegetation. The point clouds include the ground, towers, buildings and vegetation.

[0073] The present invention extends each sampling point along the xy direction by r unit distances (r is 10 meters in the default geodetic coordinates) to form a sampling square;

[0074] The present invention projects the laser point cloud along the sampling square to obtain a laser point set under the rectangular parallelepiped MBR, from which N=4096 sample points are randomly sampled to form a spatial sampling subset;

[0075] The present invention uses a laser point cloud segmentation method to obtain multimodal data according to spatial point alignment, obtains the xyz coordinates of the sampled laser points and the RGB values ​​of the laser points, constructs a set of sampling points, and forms each batch of multimodal training and test samples for deep learning. The method includes:

[0076] The present invention extracts the xyz coordinates and RGB three-component color values ​​point by point for the sampling subset formed by the S N=4096 capacity sample points mentioned above;

[0077] In the present invention, each sampling subset constructs an Nx6 vector as a training sample, and combines S training samples into a batch as a training input for deep learning.

[0078] Step 103: transforming the original matrix generated by the sample points in each spatial sampling subset and a preset number of spatial nearest neighboring points through the constructed spatial transformation unit to generate a feature transformation matrix;

[0079] Preferably, the constructed spatial conversion unit converts the original matrix generated by the sample points in each spatial sampling subset and a preset number of spatial nearest neighboring points to generate a feature conversion matrix, including:

[0080] The three-dimensional spatial coordinates xyz and three-dimensional three-component color values ​​RGB generated by the sample points and a preset number of spatial nearest points are converted into a six-dimensional feature conversion matrix.

[0081] Step 104: training the deep learning network model based on the feature conversion matrix to generate a multi-modal input support deep learning model that takes the feature conversion matrix as input and the known classification as output;

[0082] Step 105: Obtain multi-channel features extracted based on the original matrix at each convolution level in the multimodal input support deep learning model, aggregate the multi-channel features into aggregate features, and classify the point cloud of the power pole tower and its surrounding facilities through the aggregate features.

[0083] Preferably, the multimodal input support deep learning model includes a seven-layer network, wherein the first four layers are edge convolution units, and the edge convolution units extract multi-channel features through a multi-layer perception network at different convolution levels and perform pooling output to form a multi-layer perception network output of the next convolution level;

[0084] The last three layers in the seven-layer network are aggregation layers, which are used to aggregate multi-channel features into aggregated features. The aggregated features include contextual information at different levels.

[0085] The present invention uses the classification and segmentation features of sample points with known classification and segmentation as output to train a deep learning model from multimodal input to output.

[0086] Based on the above sample input, the present invention constructs a deep learning neural network that considers the local multimodal characteristics of the elements, and trains a model from input to output (which can be oriented to the laser point cloud classification or segmentation problem), in which the point granularity classification considers the pole tower, building, ground and vegetation information of live working, and all other linear laser points are marked as unclassified. The deep learning model includes:

[0087] The spatial transformation unit of the present invention is essentially for realizing the normalization of the input six-dimensional (xyzRGB) space. In order to obtain the 6x6 transformation matrix, the method provided by the present invention connects the six dimensions of each input point with its k nearest spatial neighbors to obtain the transformation matrix.

[0088] In order to extract the neighborhood edge features of each point, the edge convolution unit of the present invention uses a multi-layer perceptron at different levels to extract features for the kNN of each point and output the features in a pooled form to form the input of the next layer of network. In terms of design, each edge convolution unit uses a multi-layer perceptron based on the kNN graph as a parameter to connect a pooling layer, generating a unit output of a specified dimension at each layer.

[0089] Step 106: Calculate the minimum safety distance based on the classification results of the power tower and its surrounding facility point cloud.

[0090] Preferably, the minimum safety distance calculation is performed, including:

[0091] Based on the ground, towers, buildings and vegetation information after classification of the point cloud of power towers and their surrounding facilities, the minimum safe distance of the target space unit to the ground is calculated.

[0092] Preferably, calculating the minimum safety distance of the target space unit to the ground includes:

[0093] The spatial coordinates xyz and three component color values ​​RGB of the laser point cloud are used as the input of the multimodal input support deep learning model to determine the category of the segmented laser point cloud;

[0094] Laser points with segmentation categories of towers, buildings, ground and trees are selected, and the nearest laser point is obtained for a given spatial query coordinate point through a three-dimensional spatial indexing method. The distance between the spatial query coordinate point and the nearest laser point is taken as the minimum safe distance to the ground.

[0095] The network structure of the present invention is designed as a 7-layer network, in which the first four layers are connected by edge convolutions and the last three layers are connected by aggregation operations. The characteristic of the network is that the edge convolution operations of the first four layers can effectively extract features for each point by referring to its kNN in the layer-by-layer convolution process, and finally aggregate the context information of points at different levels.

[0096] After effectively dividing the core elements such as poles, trees, buildings and the ground, the present invention can calculate the minimum safe distance to the ground. The process is divided into laser point cloud segmentation based on the output of the above model and minimum distance calculation based on the segmented point cloud.

[0097] The laser point cloud classification of the present invention processes the unsegmented laser point cloud through the above-obtained model. The points of the unsegmented point cloud are extracted by sampling one by one, and the segmented category of each point is obtained based on the xyz and RGB information of each point as the model input;

[0098] The present invention extracts laser points of segmentation categories such as poles, buildings, ground and trees, obtains the nearest laser point for a given spatial query coordinate point through a traditional three-dimensional spatial indexing method, and uses the distance between the query point and the point closest to it as the minimum safe distance to the ground for safety distance protection.

[0099] The present invention proposes a multimodal deep learning method for automatic classification of power poles and their surrounding facilities, which can be widely used in the calculation of safe distances for live working based on the segmentation of power facilities and ground elements, including: based on large-scale laser point cloud data spatial sampling, to process the spatial data segmentation problem of large-scale laser point clouds, and realize the point cloud data segmentation / merging classification problem under limited GPU video memory; a point cloud coordinate and color multimodal deep learning model is given, and with the help of point-by-point kNN feature expression, the semantic segmentation quality of the point cloud model is significantly improved, so that the segmentation of point cloud subsets of irregular shapes is not affected by traditional fixed segmentation methods; a sampling-based multimodal deep learning model is constructed for the classification problem of power poles and their surrounding facilities, providing a point cloud segmentation approach for power poles and ground facilities, and efficiently supporting the calculation of the minimum safe distance to the ground of spatial point information based on the segmented ground, poles, buildings and vegetation information.

[0100] The present invention proposes a point cloud coordinate and color multimodal deep learning model, which uses point-by-point kNN feature expression to significantly improve the semantic segmentation quality of the point cloud model, so that the segmentation of point cloud subsets of irregular shapes is not affected by traditional fixed segmentation methods; a sampling-based multimodal deep learning model is constructed for the classification problem of power towers and their surrounding facilities, which provides a point cloud segmentation approach for power towers and ground facilities, and can efficiently support the calculation of the minimum safe distance of spatial point information to the ground based on the segmented ground, towers, buildings and vegetation information.

[0101] The present invention provides a multimodal deep learning method for automatic classification of power poles and their surrounding facilities, which can be widely used in the calculation of safe distances for live working based on the segmentation of power facilities and ground elements. By spatially sampling large-scale laser point cloud data to process the spatial data segmentation problem of large-scale laser point clouds, this method can realize the point cloud data segmentation / merging classification problem under limited GPU video memory. Based on large-scale spatial segmentation sampling, the present invention provides a point cloud coordinate and color multimodal deep learning model, which significantly improves the semantic segmentation quality of the point cloud model with the help of point-by-point kNN feature expression, so that the segmentation of point cloud subsets of irregular shapes is not affected by traditional fixed segmentation methods. The typical application scenario of the present invention is the classification problem of power poles and their surrounding facilities. Based on the multimodal deep learning model, a point cloud segmentation method is provided for power poles and ground facilities. Based on the segmented ground, poles, buildings and vegetation information, the calculation of the minimum safe distance to the ground of spatial point information can be efficiently supported.

[0102] Figure 2 FIG. 1 is a flowchart of a method 100 for processing classified laser point cloud data according to an embodiment of the present invention. Figure 2 As shown, the present invention samples large-scale laser point cloud data spatially. The process obtains spatial data segmentation and covers the scene through spatial random sampling. Through this method, a certain number (such as S) of sampling points can be obtained to ensure that the average number of point clouds under the segmentation of S objects is significantly less than U / N (N defaults to 4096) of the total number of laser points U; sampling is performed on the above S to obtain N=4096 capacity sample points, form a sampling subset, extract its xyz coordinates and RGB three-component color values ​​point by point, and input the S sample points into the neural network as a batch; the present invention mainly includes a multimodal deep learning model, which significantly improves the semantic segmentation quality of the point cloud model by means of point-by-point kNN feature expression, so that the segmentation of point cloud subsets of irregular shapes is not affected by traditional fixed segmentation methods; based on the multimodal deep learning model, the present invention provides a point cloud segmentation method for power poles and ground facilities, and can efficiently support the calculation of the minimum safe distance of spatial point information to the ground based on the segmented ground, poles, buildings and vegetation information.

[0103] The present invention performs data sampling based on laser point cloud data with RGB color information to obtain deep learning training samples that are adapted to the specified GPU memory capacity. Spatial data segmentation and scene coverage are obtained through spatial random sampling. This method can obtain a certain number (such as S) of sampling points to ensure that the average number of point clouds under the segmentation of S objects is significantly less than U / N (N defaults to 4096) of the total number of laser points U. Including:

[0104] For the spatial segmentation of laser point clouds in live working scenes, the laser points of the laser point cloud scenes are randomly sampled to obtain a certain number (such as S) of sampling points, ensuring that the average number of point clouds under the S object segmentation is significantly less than U / N (N defaults to 4096) of the total number of laser points U. Given a laser point cloud dataset containing tens of millions of points, considering the complexity of the network structure and the size of the GPU memory, this parameter configuration can ensure that 16GB of memory can gradually incorporate all the laser point data of the live working scene, while ensuring that common datasets can be covered by at least one sampling point in one sampling.

[0105] Each sampling point is extended by r unit distances (r is 10 meters in the default geodetic coordinates) in both x and y directions to form a sampling square. The selection of r unit distances is suitable for the element segmentation of live working scenes. For example, the connection between a tower and a string of insulators or the connection between an overhead ground wire and a tower can extract distinguishing features within a range of about 10 meters.

[0106] Figure 3This is a diagram of different classifications of the laser point cloud targeted by the present invention, in which the tower body, the ground and the line are composed of three different grayscales, and the ground and ground buildings can be further subdivided. Figure 4 This is a schematic diagram of the spatial segmentation proposed by the present invention. First, the laser point cloud data is divided along the xy plane and extended by r unit distances. These xy planes form a column containing all point clouds extending by r unit distances on the xy plane. In order to ensure the spatial relationship of each sampling version, the random sampling method provided by the present invention randomly selects a number of sampling points as a sample set after the scene space is segmented to form Figure 5 The three-dimensional polygon shown.

[0107] In the present invention, after obtaining a spatial division sample, the data needs to be input into the deep learning network. Since the segmentation problem of laser point cloud focuses on the features between the spatial context of points, it is necessary to associate points with each other. When the input data scale is large, the limited GPU capacity is difficult to accommodate the association process between points, so it is necessary to effectively reduce the scale of spatial points while ensuring that data can be added to the network in batches to achieve progressive loss minimization. The present invention provides a method for randomly organizing samples for a progressive process.

[0108] Specifically, after a batch of sampling points is obtained, in order to smooth the quantity differences between sample batches, the laser point cloud is projected along the sampling square to obtain the laser point set under the rectangular MBR, from which N = 4096 sample points are randomly sampled to form a spatial sampling subset. Since it is difficult for the GPU to accommodate all the laser point data of live work at one time, the method can ensure that the laser points of a single batch are of the same size based on constant S and N. On this basis, the continuously randomly increased S points can sample all ranges in the scene along the xy plane to ensure the progressive stability of the network.

[0109] In step 102, multiple channels of the laser point cloud are merged to ensure that each batch of data contains xyz and RGB values ​​of the same channel on the basis of spatial alignment, so as to facilitate layer-by-layer feature extraction of the neural network.

[0110] The channel in the present invention may include spatial information xyz input dimensions, and RGB features associated with its points. A batch of training samples is formed by extracting information from it and merging information of at least 6 dimensions, including:

[0111] For the sampling subset formed by the above S N = 4096 capacity sample points, extract their xyz coordinates and RGB three-component color values ​​point by point. The two sets of three-component information need to consider the features of the nearest k neighboring points of each sample center point, so a spatial transformation is constructed to aggregate the 6 channels to achieve feature transformation from Nxkx6 to Nx6.

[0112] Figure 6 Two channels for the grounding part of a tower, Figure 6 a is the xyz coordinate of the space point, Figure 6 b is the RGB channel of each point. In order to ensure point alignment during training and testing, Figure 7 a is obtained by sampling Figure 7 The xyz channel of c, Figure 7 b corresponds to spatial sampling Figure 7 d. Since the two channels are sampled in a point-aligned manner, they can be guaranteed to be point-wise aligned and submitted to the deep learning network for training and testing.

[0113] Each sampling subset constructs an Nx6 vector as a training sample, and combines S training samples into a batch as the training input for deep learning.

[0114] The above Nx6 samples are grouped into a batch as input, and the known point-by-point sample classification and segmentation features are used as segmentation output to train a multi-modal input-output deep learning model. When facing the classification problem of a single laser point cloud, the above Nx6 samples are grouped into a batch as input, and the known classification is used as output to form a multi-modal input to support deep learning model training.

[0115] In step 103, based on the given input, a multi-layer perception deep learning model is constructed, which can extract spatially adjacent multi-channel features layer by layer, and generate aggregated features of each level across layers through aggregation operations, and finally enter classification or segmentation.

[0116] The key to the deep learning model proposed in this invention lies in the layer-by-layer aggregation of the preprocessing stage and the edge convolution. The spatial conversion unit and the edge convolution unit in the preprocessing stage of this process essentially consider the kNN object features of each sampling point at each convolution level, thereby providing a deep learning method that considers spatially adjacent multi-channel high-level features for large-scale point cloud segmentation problems. The framework includes:

[0117] like Figure 8 As shown in the top part: the classification framework consists of four layers of edge convolutional networks connected in sequence, two consecutive layers are connected through Nx64 multi-layer perceptrons, the top layer is aggregated with the output of the three layers of perceptrons below it through a Nx128 multi-layer perceptron, and finally through pooling, the classifier for classification c is obtained through the final mlp (512, 156, c) multi-layer perceptron. Similarly, using the spatial transformation unit as input, the deep learning of the point cloud segmentation part is as follows Figure 8As shown in the bottom part: cross-layer aggregation is performed through three layers of edge convolution processing, and then a point-by-point categorical vector is injected through an mlp(1024) multi-layer perceptron. Finally, it is aggregated through the convolution of Nx1088 and the three bottom edge convolution layers, and output to the point-by-point point cloud segmentation output through an mlp(256,256,128,p) multi-layer perceptron. Through this network, at any l-th layer, the network will form an independent graph G(l) = (V(l), E(l)), where the features of each sampling point xi on the l-th layer are characterized by the convolution of the features of its nearest k points on the same layer. This process requires a paired original kNN matrix as input and is applied in two stages.

[0118] In the spatial transformation part of preprocessing, such as Fig. 9 As shown in a, all sample points are transformed by their original kNN matrix, and the input for deep learning is obtained by performing matrix multiplication of the 6 components of the input with the kx6 nearest neighbor features.

[0119] In the convolution process, Fig. 9 As shown in Figure 2, any point on any layer obtains the same-layer features of its k nearest neighbors through the original kNN matrix. The features are converted by Nxkxan (where an refers to the dimension of the input features on a certain layer) and then pooled to obtain the k nearest neighbor aggregated features of each point.

[0120] The above segmentation method can be used to segment core elements such as towers, trees, buildings and ground. Fig.10 As shown, 901 performs spatial sampling to form a spatial segmentation polygon; 902 extracts the multi-channel laser point cloud sampling points by extracting multi-modal points in each polygon; 903 performs spatial conversion preprocessing on the sampled aligned multi-channel data, and 905 performs deep learning on the basis of layer-by-layer convolution at 904 to obtain a laser point cloud segmentation model; when it is necessary to measure the distance of the new laser point cloud data, the calculation of the minimum safe distance to the ground can be divided into two steps.

[0121] The process of the present invention is divided into laser point cloud segmentation based on the output of the above model and calculation of the minimum safe distance to the ground based on the segmented point cloud. The method includes:

[0122] Laser point cloud classification outputs the segmentation category of each point by inputting unclassified data into the model. The points of the unsegmented point cloud are extracted one by one by sampling, and the segmented category of each point is obtained based on the xyz and RGB information of each point as the model input, covering four major categories: towers, ground, buildings and lines.

[0123] The laser points of the segmentation categories of towers, buildings, ground and trees are extracted, and the nearest laser point is obtained for a given spatial query coordinate point through the traditional three-dimensional spatial indexing method. The distance between the query point and the point closest to it is used as the minimum safe distance to the ground for safety distance protection. All three-dimensional nearest distance calculation methods based on R-tree and spatial octree can be used to obtain the nearest neighbor, so as to obtain the coordinates of the nearest laser point of a given ground potential and its minimum safe distance to the ground for any given live working location point coordinates.

[0124] Since it is difficult to directly use existing deep learning methods to deal with the problem of large scale of laser point clouds in operation scenes, the present invention provides a multi-modal method and system for automatic identification of facilities around power poles and towers. When calculating the minimum safe distance to the ground, the live operation scene can be referred to, which generally covers at least all visible elements between two poles and towers, such as trees, ground, roads, poles and wires, etc. The point cloud spatial coordinates. The method provided by the present invention can effectively reduce the workload of manual classification. In particular, in the application of rapid update of laser point clouds in power grids, the maintenance cost of power grids can be significantly reduced.

[0125] The scope of protection of the present invention covers a method for multimodal spatial partitioning of laser point clouds at live working sites. The present invention is based on large-scale spatial sampling of laser point cloud data to handle the spatial data segmentation problem of large-scale laser point clouds and realize the point cloud data segmentation / merging classification problem under limited GPU video memory. Due to the adoption of the spatial sampling concept, the method is not strictly subject to the high computational cost of spatial convolution.

[0126] The protection scope of the present invention covers a multimodal spatial alignment method. The present invention aligns the multimodal data obtained by the laser point cloud segmentation method according to spatial points, obtains the xyz coordinates of the sampled laser points and the RGB values ​​of the laser points, constructs a set of sampling points, and forms each batch of multimodal training and test samples for deep learning, so that the proposed method can effectively consider the pixel color information on the laser points, and ensure the classification and classification results of complex objects.

[0127] The scope of protection of the present invention covers a multimodal deep learning model of point cloud coordinates and colors. The present invention constructs a sampling-based multimodal deep learning model for the classification of power towers and their surrounding facilities, providing a point cloud segmentation method for power towers and ground facilities. The model significantly improves the semantic segmentation quality of the point cloud model by using point-by-point kNN feature expression, so that the segmentation of irregular point cloud subsets is not affected by traditional fixed segmentation methods.

[0128] The present invention provides a method and system for calculating the minimum distance to power facilities based on ground potential. The model output can process point cloud extraction of pole towers, buildings, ground and vegetation information. When the insulating soft ladder method and the unmanned aerial vehicle combined with the electric lifting device operation method are selected to enter the position of equal potential, the minimum distance calculation between the soft ladder and the insulating rope hanging point and the ground potential elements can be supported.

[0129] Fig.11 It is a structural diagram of a power tower and environmental point cloud classification system supporting safety distance calculation according to a preferred embodiment of the present invention.

[0130] like Fig.11 As shown, the present invention provides a multi-modal power tower and surrounding facilities automatic identification system, the system comprising:

[0131] The initial unit 1101 is used to perform spatial segmentation on the laser point cloud of the power tower and its surrounding facilities, and map the spatial coordinates xyz and the three-component color values ​​RGB of each spatial unit after spatial segmentation to each laser point;

[0132] The sampling unit 1102 is used to sample the laser point of each spatial unit to generate multiple spatial sampling subsets with the same number of samples; establish an original matrix based on the spatial coordinates xyz and the three component color values ​​RGB of the laser point;

[0133] Preferably, the sampling unit 1102 is used to sample the laser points of each spatial unit to generate a plurality of spatial sampling subsets having the same number of samples as the sampled samples, and is also used to:

[0134] The laser points of each space unit after the space division are randomly sampled, and the number of sampling points is less than a preset percentage of the total number of laser points;

[0135] Each sampling point is extended by a preset unit distance along the xy coordinate direction of each spatial unit to form a square sampling;

[0136] Project the laser points on the square sampling to obtain the laser point set under the rectangular MBR, and collect the sample points of the preset data from the laser point set under the rectangular MBR as a spatial sampling subset.

[0137] A conversion unit 1103 is used to convert an original matrix generated by the sample points and a preset number of spatial nearest neighbor points in each spatial sampling subset through the constructed spatial conversion unit to generate a feature conversion matrix;

[0138] Preferably, the conversion unit 1103 is used to convert the original matrix generated by the sample points and a preset number of spatial nearest neighbor points in each spatial sampling subset through the constructed spatial conversion unit to generate a feature conversion matrix, and is also used to:

[0139] The three-dimensional spatial coordinates xyz and three-dimensional three-component color values ​​RGB generated by the sample points and a preset number of spatial nearest points are converted into a six-dimensional feature conversion matrix.

[0140] A training unit 1104 is used to train the deep learning network model based on the feature conversion matrix, and generate a multi-modal input support deep learning model that takes the feature conversion matrix as input and the known classification as output;

[0141] The result unit 1105 is used to obtain multi-channel features extracted based on the original matrix at each convolution level in the multi-modal input support deep learning model, aggregate the multi-channel features into aggregated features, and classify the point cloud of the power tower and its surrounding facilities through the aggregated features.

[0142] Preferably, the multimodal input support deep learning model includes a seven-layer network, wherein the first four layers are edge convolution units, and the edge convolution units extract multi-channel features through a multi-layer perception network at different convolution levels and perform pooling output to form a multi-layer perception network output of the next convolution level;

[0143] The last three layers in the seven-layer network are aggregation layers, which are used to aggregate multi-channel features into aggregated features. The aggregated features include contextual information at different levels.

[0144] Preferably, the result unit is used to calculate the minimum safety distance based on the classification of the power pole tower and its surrounding facility point cloud.

[0145] Preferably, the result unit, used for calculating the minimum safety distance to the ground, is further used for:

[0146] Based on the ground, towers, buildings and vegetation information after classification of the point cloud of power towers and their surrounding facilities, the minimum safe distance of the target space unit to the ground is calculated.

[0147] Preferably, the result unit is used to calculate the minimum safety distance of the target space unit to the ground, and is also used to:

[0148] The spatial coordinates xyz and three component color values ​​RGB of the laser point cloud are used as the input of the multimodal input support deep learning model to determine the category of the segmented laser point cloud;

[0149] Laser points with segmentation categories of towers, buildings, ground and trees are selected, and the nearest laser point is obtained for a given spatial query coordinate point through a three-dimensional spatial indexing method. The distance between the spatial query coordinate point and the nearest laser point is taken as the minimum safe distance to the ground.

[0150] A power pole tower and environmental point cloud classification system supporting safety distance calculation in a preferred embodiment of the present invention corresponds to a power pole tower and environmental point cloud classification method supporting safety distance calculation in another preferred embodiment of the present invention, which will not be repeated here.

[0151] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0152] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0153] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0155] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0156] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0157] The invention has been described above with reference to a few embodiments. However, it is readily apparent to a person skilled in the art that other embodiments than the ones disclosed above are equally within the scope of the invention, as defined by the appended patent claims.

[0158] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a / said / the [means, components, etc.]" are to be openly interpreted as at least one instance of the means, components, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not necessarily have to be performed in the exact order disclosed, unless explicitly stated otherwise.

Claims

1. A method for classifying power towers and environmental point clouds supporting safe distance calculation, the method comprising: The laser point cloud of the power tower and its surrounding facilities is spatially segmented, and the spatial coordinates xyz and three-component color values ​​RGB of each spatial unit after spatial segmentation are mapped to each laser point; Sampling the laser point of each spatial unit to generate a plurality of spatial sampling subsets having the same number of samples as the sampled number; establishing an original matrix based on the spatial coordinates xyz and the three component color values ​​RGB of the laser point; The original matrix generated by the sample points and a preset number of spatial nearest neighbor points in each spatial sampling subset is transformed by the constructed spatial transformation unit to generate a feature transformation matrix; Training a deep learning network model based on the feature conversion matrix to generate a multimodal input support deep learning model that takes the feature conversion matrix as input and known classifications as output; Acquire multi-channel features extracted based on the original matrix at each convolution level in the multimodal input support deep learning model, aggregate the multi-channel features into aggregate features, and classify the power pole tower and its surrounding facility point cloud by using the aggregate features; Based on the classification results of the power pole tower and its surrounding facility point cloud, a minimum safety distance is calculated.

2. The method according to claim 1, wherein the laser point cloud of the power tower and its surrounding facilities is spatially segmented, and the spatial coordinates xyz and the three-component color values ​​RGB of each spatial unit after the spatial segmentation are mapped to each laser point; The laser points of each spatial unit are sampled to generate multiple spatial sampling subsets with the same number of samples, including: The laser point cloud of the power tower and its surrounding facilities is spatially segmented, and the laser points of each spatial unit after the spatial segmentation are randomly sampled, and the number of sampling points is less than a preset percentage of the total number of laser points; Each sampling point is extended by a preset unit distance along the xy coordinate direction of each spatial unit to form a square sampling; Project the laser points on the square sampling to obtain a laser point set under the rectangular parallelepiped MBR, and collect sample points of preset data from the laser point set under the rectangular parallelepiped MBR as a spatial sampling subset.

3. The method according to claim 1, wherein the spatial conversion unit constructed by converting the original matrix generated by the sample points in each spatial sampling subset and a preset number of spatial nearest neighboring points to generate a feature conversion matrix comprises: The three-dimensional spatial coordinates xyz and three-dimensional three-component color values ​​RGB generated by the sample points and a preset number of spatial nearest points are converted into a six-dimensional feature conversion matrix.

4. According to the method of claim 1, the multimodal input supporting deep learning model comprises a seven-layer network, wherein the first four layers are edge convolution units, and the edge convolution units extract multi-channel features through a multi-layer perception network at different convolution levels and perform pooling output to form a multi-layer perception network output of the next convolution level; The last three layers in the seven-layer network are aggregation layers, which are used to aggregate multi-channel features into aggregate features, and the aggregate features include context information at different levels.

5. The method according to claim 1, wherein the minimum safety distance calculation comprises: Based on the classified ground, pole tower, building and vegetation information of the power pole tower and its surrounding facilities, the minimum safe distance of the target space unit to the ground is calculated.

6. The method according to claim 5, wherein the step of calculating the minimum safety distance of the target space unit to the ground comprises: The spatial coordinates xyz and three component color values ​​RGB of the laser point cloud are used as the input of the multimodal input support deep learning model to determine the category of the segmented laser point cloud; Laser points with segmentation categories of towers, buildings, ground and trees are selected, and the nearest laser point is obtained for a given spatial query coordinate point through a three-dimensional spatial indexing method. The distance between the spatial query coordinate point and the nearest laser point is used as the minimum safe distance to the ground.

7. A power tower and environment point cloud classification system supporting safety distance calculation, the system comprising: The initial unit is used to spatially segment the laser point cloud of the power tower and its surrounding facilities, and map the spatial coordinates xyz and the three-component color values ​​RGB of each spatial unit after spatial segmentation to each laser point; A sampling unit is used to sample the laser point of each spatial unit to generate multiple spatial sampling subsets with the same number of samples; an original matrix is ​​established based on the spatial coordinates xyz and the three component color values ​​RGB of the laser point; A conversion unit, used to convert an original matrix generated by sample points and a preset number of spatial nearest neighbor points in each spatial sampling subset through the constructed spatial conversion unit to generate a feature conversion matrix; A training unit, used for training the deep learning network model based on the feature conversion matrix, and generating a multi-modal input supporting deep learning model having the feature conversion matrix as input and known classifications as output; A result unit is used to obtain multi-channel features extracted based on the original matrix at each convolution level in the multimodal input support deep learning model, aggregate the multi-channel features into aggregate features, and classify the power pole tower and its surrounding facility point cloud through the aggregate features; based on the classification results of the power pole tower and its surrounding facility point cloud, calculate the minimum safety distance.

8. The system according to claim 7, wherein the sampling unit is used to sample the laser points of each spatial unit to generate a plurality of spatial sampling subsets having the same number of samples as the sampled samples, and is also used to: The laser points of each space unit after the space division are randomly sampled, and the number of sampling points is less than a preset percentage of the total number of laser points; Each sampling point is extended by a preset unit distance along the xy coordinate direction of each spatial unit to form a square sampling; Project the laser points on the square sampling to obtain a laser point set under the rectangular parallelepiped MBR, and collect sample points of preset data from the laser point set under the rectangular parallelepiped MBR as a spatial sampling subset.

9. The system according to claim 7, wherein the conversion unit is used to convert the original matrix generated by the sample points and a preset number of spatial nearest neighbor points in each spatial sampling subset through the constructed spatial conversion unit to generate a feature conversion matrix, and is also used to: The three-dimensional spatial coordinates xyz and three-dimensional three-component color values ​​RGB generated by the sample points and a preset number of spatial nearest points are converted into a six-dimensional feature conversion matrix.

10. The system according to claim 7, wherein the multimodal input supporting deep learning model comprises a seven-layer network, wherein the first four layers are edge convolution units, and the edge convolution units extract multi-channel features through a multi-layer perception network at different convolution levels and perform pooling output to form a multi-layer perception network output of the next convolution level; The last three layers in the seven-layer network are aggregation layers, which are used to aggregate multi-channel features into aggregate features, and the aggregate features include context information at different levels.

11. The system according to claim 7, wherein the result unit is used to calculate the minimum safety distance and is further used to: Based on the classified ground, pole tower, building and vegetation information of the power pole tower and its surrounding facilities, the minimum safe distance of the target space unit to the ground is calculated.

12. The system according to claim 11, wherein the result unit is used to calculate the minimum safety distance of the target space unit to the ground, and is also used to: The spatial coordinates xyz and three component color values ​​RGB of the laser point cloud are used as the input of the multimodal input support deep learning model to determine the category of the segmented laser point cloud; Laser points with segmentation categories of towers, buildings, ground and trees are selected, and the nearest laser point is obtained for a given spatial query coordinate point through a three-dimensional spatial indexing method. The distance between the spatial query coordinate point and the nearest laser point is used as the minimum safe distance to the ground.

Images