Method for generating shooting points for distribution network unmanned aerial vehicle inspection based on semantic segmentation of tower point cloud and related device

Abstract

The application discloses a kind of based on tower point cloud semantic segmentation's distribution network unmanned aerial vehicle inspection shooting point generation method and related device, the generation method includes: obtaining original point cloud data containing electric pole tower;The original point cloud data is clustered, and a plurality of point cloud clusters are obtained;Each point cloud cluster is identified, and tower point cloud is obtained;When the number of vertical pole is greater than 1, the tower point cloud is layered according to the number of vertical pole, and vertical pole point cloud is obtained;Each vertical pole point cloud is layered according to cross arm height, and cross arm point cloud is obtained;The position of two end points of the main axis direction of each cross arm point cloud is calculated;The position of two end points of the main axis direction of each cross arm point cloud is determined according to the position of two end points of the main axis direction of each cross arm point cloud.The application realizes the automatic generation of unmanned aerial vehicle inspection shooting point, solves the problem that manual marking inspection target point consumes much time, labor cost is high, needs to rely on professional knowledge and experience, and practical application popularization is difficult.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data processing, and particularly relates to a distribution network unmanned aerial vehicle inspection shooting point generation method based on tower point cloud semantic segmentation, an inspection method, equipment, a storage medium and a product. BACKGROUND

[0002] Power lines are widely distributed and mostly located in the wild. The harsh environment in the wild brings great pressure to the normal operation of the power system. Power inspection can timely find and eliminate potential faults and hidden dangers and prevent power accidents by regularly checking and maintaining power equipment and lines. The existing power inspection methods mainly include two types: one is traditional manual inspection, and the other is unmanned aerial vehicle inspection.

[0003] The traditional manual inspection method requires professional operation and maintenance personnel to go to the power equipment site to check the damage degree and operating condition of the power equipment one by one. Usually, the operation and maintenance personnel are required to climb to the power line tower to observe the operating state of the power equipment at close range. This method brings great safety risks to the operation and maintenance personnel, especially in harsh weather or complex site environment, and requires a large amount of manual cost.

[0004] In recent years, with the rapid development of remote sensing technology and communication technology, the application of unmanned aerial vehicles has been widely promoted and gradually applied to various industries. Unmanned aerial vehicles have the characteristics of flexibility, low cost and strong operability, and can well adapt to various complex environments and show obvious advantages in power inspection. Especially in high-risk and difficult-to-reach complex environments, unmanned aerial vehicles can carry visible light cameras, fly to the inspection target tower or component, shoot the current state of the equipment, and use these photos for subsequent defect identification and equipment maintenance. This not only eliminates the safety risks of the operation and maintenance personnel, but also greatly improves the inspection efficiency and promotes the intelligent and automated level of power inspection. However, the existing unmanned aerial vehicle inspection still requires the operator to operate the unmanned aerial vehicle to ensure that the unmanned aerial vehicle can shoot the components to be inspected, which still has high requirements for the operator.

[0005] In order to reduce the operation requirements of the operator on the unmanned aerial vehicle, the point cloud data of the power line tower is collected by the laser radar carried on the unmanned aerial vehicle in advance, and then the inspection track of the unmanned aerial vehicle is generated based on the point cloud data. The point cloud data is a series of unordered and discrete points in three-dimensional space, mainly including three-dimensional coordinates and color information, which can reflect the real geographical position information and environmental information in the actual scene. Therefore, by using the collected point cloud data, the real coordinates of the inspection target on the power line tower and the real coordinates required to hover during the unmanned aerial vehicle inspection process can be calibrated. The existing inspection target labeling method is mainly manual labeling. The inspection target points labeled manually can further form the inspection track of the unmanned aerial vehicle, so that the unmanned aerial vehicle can perform the power inspection task according to the predetermined inspection track. Although this method has significant improvement in safety and efficiency compared with manual inspection, it still consumes a lot of time and labor cost in the manual calibration link, and relies on the expert knowledge and professional experience in power grid inspection, which requires high personnel for drawing the inspection track, and leads to certain difficulties in practical application and popularization. SUMMARY

[0006] The purpose of the present application is to provide an inspection shooting point generation method, an inspection method, equipment, a medium and a product, to solve the problems of time-consuming, high labor cost, dependence on professional knowledge and experience, and difficulty in practical application and popularization in manual labeling of inspection target points.

[0007] The present application solves the above technical problems by the following technical solutions: a power distribution network unmanned aerial vehicle inspection shooting point generation method based on tower point cloud semantic segmentation, comprising:

[0008] Obtain the original point cloud data containing the power line tower;

[0009] Cluster the original point cloud data to obtain a plurality of point cloud clusters;

[0010] Identify the type of each point cloud cluster to obtain a tower point cloud;

[0011] When the number of vertical rods is greater than 1, layer the tower point cloud according to the number of vertical rods to obtain a vertical rod point cloud; when the number of vertical rods is equal to 1, use the tower point cloud as the vertical rod point cloud;

[0012] Layer each vertical rod point cloud according to the cross arm height to obtain a cross arm point cloud;

[0013] Calculate the positions of the two endpoints of the main axis direction of each cross arm point cloud;

[0014] Determine the inspection shooting point based on the positions of the two endpoints of the main axis direction of each cross arm point cloud.

[0015] Further, a trained tower pole recognition model is called to recognize the type of each point cloud cluster; wherein the tower pole recognition model comprises a multi-layer perceptron, an Encoder layer in a Pointnet classification network, a fully connected layer and a classification layer connected in sequence;

[0016] The type of each point cloud cluster is recognized, specifically including:

[0017] Statistical features of the point cloud cluster are calculated, and an input feature vector is formed by all the statistical features of the point cloud cluster;

[0018] The input feature vector is extracted by a multi-layer perceptron to obtain a first output feature quantity;

[0019] The product of the first output feature quantity and each point in the point cloud cluster is taken as the input of the Encoder layer in the Pointnet classification network, and the Encoder layer in the Pointnet classification network is used for feature learning and mapping to obtain a second output feature quantity;

[0020] The second output feature quantity is extracted by a fully connected layer to obtain a third output feature quantity;

[0021] The third output feature quantity is predicted by the classification layer to obtain the type of the point cloud cluster; wherein the type of the point cloud cluster includes wire, tower pole and environment;

[0022] The point cloud clusters of the type of tower pole are merged to obtain a tower pole point cloud.

[0023] Further, the statistical features of the point cloud cluster include the height, horizontal area, volume, cluster size, color value and distance between the cluster center point and the tower pole reference point of the point cloud cluster; wherein the calculation formula of each statistical feature is:

[0024] h k =z kmax -z kmin ;

[0025] A k =(x kmax -x kmin )×(y kmax -y kmin );

[0026] vol k =h k ×A k ;

[0027]

[0028]

[0029] wherein h k represents the height of the kth point cloud cluster, z kmax and z kmin represent the maximum and minimum vertical coordinate values of the points in the kth point cloud cluster, A k represents the area of the kth point cloud cluster, x kmax and x kmin represent the maximum and minimum horizontal coordinate values of the points in the kth point cloud cluster, y kmax and y kmin represent the maximum and minimum vertical coordinate values of the points in the kth point cloud cluster, vol k represents the volume of the kth point cloud cluster, color k represents the color value of the kth point cloud cluster, N k represents the cluster size of the kth point cloud cluster, r ki , g ki , b ki represents the color value of the ith point in the kth point cloud cluster, dist k represents the distance between the cluster center point and the tower reference point in the kth point cloud cluster, x ki , y ki , z ki represents the coordinate value of the ith point in the kth point cloud cluster, x r , y r , z r represents the coordinate value of the tower reference point.

[0030] Further, the tower point cloud is divided into vertical bars according to the number of vertical bars, including:

[0031] calculating the centroid of the tower point cloud;

[0032] constructing a first covariance matrix according to the centroid of the tower point cloud;

[0033] performing eigenvalue decomposition on the first covariance matrix, and taking the eigenvector corresponding to the minimum eigenvalue as the normal vector of the main plane; wherein the main plane refers to the plane formed by the vertical bars;

[0034] based on the normal vector of the main plane, dividing the tower point cloud into M t equal parts, obtaining M t vertical bar point clouds; wherein M t represents the number of vertical bars contained in the tower.

[0035] Further, the two end point positions of the main axis direction of each cross arm point cloud are calculated, including:

[0036] calculating the centroid of the cross arm point cloud;

[0037] constructing a second covariance matrix according to the centroid of the cross arm point cloud;

[0038] performing eigenvalue decomposition on the second covariance matrix, and taking an eigenvector corresponding to a maximum eigenvalue as a main axis direction of the cross arm point cloud;

[0039] determining two points located in the main axis direction and farthest from the centroid of the cross arm point cloud, and taking the two points as two end point positions of the main axis direction of the cross arm point cloud.

[0040] Further, the generating method further comprises calculating a yaw angle of the unmanned aerial vehicle and a pitch angle of the camera according to the inspection shooting point position, and a specific calculation formula is:

[0041]

[0042]

[0043] wherein, θ q represents the yaw angle of the unmanned aerial vehicle at the qth inspection shooting point position, x s,q and y s,q represent the horizontal coordinate and the vertical coordinate of the qth inspection shooting point position respectively, x s,q+1 and y s,q+1 represent the horizontal coordinate and the vertical coordinate of the q+1th inspection shooting point position respectively, represents the pitch angle of the camera at the qth inspection shooting point position, x qh , y qh , and z qh represent the coordinate position of the qth inspection shooting point position, x qt , y qt , and z qt represent the qth end point position or the qth inspection target position, and arctan represents an inverse tangent function.

[0044] The application provides a power line pole unmanned aerial vehicle inspection method, comprising:

[0045] obtaining an inspection shooting point position by using the power distribution network unmanned aerial vehicle inspection shooting point generating method based on pole point cloud semantic segmentation as described above;

[0046] generating an inspection flight path according to the inspection shooting point position;

[0047] controlling the unmanned aerial vehicle to perform an inspection task according to the inspection flight path.

[0048] The application provides an electronic device, including a memory, a processor and a computer program / instruction stored on the memory, the processor executes the computer program / instruction to realize the power distribution network unmanned aerial vehicle inspection shooting point generation method based on tower point cloud semantic segmentation or the power distribution tower unmanned aerial vehicle inspection method as described above.

[0049] The application provides a computer readable storage medium, which stores a computer program / instruction, the computer program / instruction is executed by a processor to realize the power distribution network unmanned aerial vehicle inspection shooting point generation method based on tower point cloud semantic segmentation or the power distribution tower unmanned aerial vehicle inspection method as described above.

[0050] The application provides a computer program product, which includes a computer program / instruction, the computer program / instruction is executed by a processor to realize the power distribution network unmanned aerial vehicle inspection shooting point generation method based on tower point cloud semantic segmentation or the power distribution tower unmanned aerial vehicle inspection method as described above.

[0051] Compared with the prior art, the application has the following advantages:

[0052] The application clusters the original point cloud data of the power distribution tower to obtain a plurality of point cloud clusters; the power distribution point cloud, the tower point cloud and the environment point cloud are determined by type identification on the point cloud clusters; the tower point cloud is layered according to the tower type and the cross arm height to determine the cross arm point cloud; finally, the positions of the target parts such as fittings and insulator strings located at both ends of the cross arm are determined from the cross arm point cloud, and then the inspection shooting point position is determined, thereby realizing the automatic generation of the unmanned aerial vehicle inspection shooting point position, solving the problems that manual labeling of the inspection target point consumes much time, the labor cost is high, professional knowledge and experience are needed, and the practical application and popularization are difficult. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only one embodiment of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0054] Figure 1 It is a power distribution network unmanned aerial vehicle inspection shooting point generation method flow chart based on tower point cloud semantic segmentation in the embodiment of the application;

[0055] Figure 2 It is a point cloud data clustering result visualization example 1 diagram in the embodiment of the application;

[0056] Figure 3 It is a point cloud data clustering result visualization example 2 diagram in the embodiment of the application;

[0057] Figure 4This is a diagram of the utility pole identification model architecture in an embodiment of the present invention;

[0058] Figure 5 This is Example 1 of the visualization of tower point clouds in an embodiment of the present invention;

[0059] Figure 6 This is Example 2 of the visualization of tower point clouds in an embodiment of the present invention;

[0060] Figure 7 This is an example diagram of the visualization of the pole-level layering in an embodiment of the present invention; wherein, different colors represent different layers;

[0061] Figure 8 This is an example diagram of highly layered visualization in an embodiment of the present invention; wherein, different colors represent different layers;

[0062] Figure 9 This is Example 1, which visualizes the tower point cloud and crossarm endpoints in an embodiment of the present invention.

[0063] Figure 10 This is Example 2 of the visualization of the tower point cloud and crossarm endpoints in an embodiment of the present invention;

[0064] Figure 11 This is an example diagram of visual verification of the inspection route KML file in an embodiment of the present invention. Detailed Implementation

[0065] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] The technical solutions of this application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0067] Example 1

[0068] like Figure 1 As shown in the figure, the method for generating inspection and shooting points of power distribution network drones based on pole point cloud semantic segmentation provided by the present invention includes the following steps:

[0069] Step S1: Obtain raw point cloud data containing utility poles and towers.

[0070] The drone, equipped with a lidar, was used to collect raw point cloud data containing utility poles and towers. The raw point cloud data includes multiple points, each of which includes coordinate and color information.

[0071] Step S2: clustering the original point cloud data to obtain a plurality of point cloud clusters.

[0072] In the specific embodiments of the present application, the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm is used to cluster the original point cloud data to obtain a plurality of point cloud clusters. The DBSCAN algorithm uses a neighborhood radius ε and a minimum number of points MinPts to perform spatial clustering on the original point cloud data, where ε defines the neighborhood range of a point, that is, all points within ε distance from a point are considered to be adjacent, and the ε-neighborhood (N ε (p)) can be represented as:

[0073] N ε (p) = {q e D | dist (p, q) < ε} (1)

[0074] where D represents the original point cloud data, p and q represent points in the original point cloud data, dist(p, q) represents the distance between points p and q, the neighborhood contains all points q that are within a distance of ε from point p; MinPts refers to the minimum number of points required in the ε neighborhood of a point p, if the number of points in the ε neighborhood of a point is greater than MinPts, the point is considered a core point, and all points in the ε neighborhood of the core point together form a point cloud cluster, thereby obtaining several point cloud clusters that are spatially separated and have different densities, as shown in Figure 2 and Figure 3 , where Cluster represents a cluster, Noise represents a noise point, different colors represent different point cloud clusters, the horizontal coordinate is the X coordinate of the UTM coordinate system, the vertical coordinate is the Y coordinate of the UTM coordinate system, and the vertical coordinate is the Z coordinate of the UTM coordinate system.

[0075] Although the power distribution network scenario is complex, the key parts on the tower are often at a certain spatial distance from the surrounding environment (such as buildings, vegetation, etc.) to meet the safety requirements of the power distribution network, and entities of different materials often exhibit different point densities in point cloud data. Based on this, the DBSCAN algorithm is used to separate the original point cloud data containing the tower and the surrounding environment to obtain point cloud clusters of different types, and the DBSCAN algorithm has the ability to identify noise points, which can filter out noise points in the original point cloud data.

[0076] Step S3: identifying the type of each point cloud cluster to obtain a tower point cloud.

[0077] In the specific embodiments of the present application, a trained wire tower identification model is called to identify the type of each point cloud cluster. As shown in Figure 4As shown, the power pole tower recognition model includes a multilayer perceptron, an Encoder layer in a Pointnet classification network, a fully connected layer, and a classification layer connected in turn. The trained power pole tower recognition model is used to identify the type of each point cloud cluster, specifically including:

[0078] Step S3.1: Calculate the statistical features of the point cloud cluster, and form an input feature vector from all the statistical features of the point cloud cluster.

[0079] In the specific embodiments of the present application, the statistical features of the point cloud cluster include the height, horizontal area, volume, cluster size, color value, and distance between the cluster center point and the pole tower reference point of the point cloud cluster, and the specific calculation formulas of the statistical features are as follows:

[0080] h k =z kmax -z kmin (2)

[0081] A k =(x kmax -x kmin )×(y kmax -y kmin ) (3)

[0082] vol k =h k ×A k (4)

[0083]

[0084]

[0085] wherein h k represents the height of the kth point cloud cluster, z kmax and z kmin represent the maximum and minimum vertical coordinate values (i.e. Z coordinate values) of the points in the kth point cloud cluster, A k represents the area of the kth point cloud cluster, x kmax and x kmin represent the maximum and minimum horizontal coordinate values (i.e. X coordinate values) of the points in the kth point cloud cluster, y kmax and y kmin represent the maximum and minimum vertical coordinate values (i.e. Y coordinate values) of the points in the kth point cloud cluster, vol k represents the volume of the kth point cloud cluster, color k represents the color value of the kth point cloud cluster, N k represents the cluster size of the kth point cloud cluster, r ki , g ki , and b kicolor value of the i-th point in the k-th point cloud cluster, dist k distance between the cluster center point and the tower reference point in the k-th point cloud cluster (i.e., center deviation), x ki ,y ki ,z ki coordinate value of the i-th point in the k-th point cloud cluster, x r ,y r ,z r coordinate value of the tower reference point. All coordinate values are based on the UTM (Universal Transverse Mercator) projection coordinate system.

[0086] The tower reference point is a theoretical coordinate point obtained in the tower construction design stage and is an input parameter of the method of the present application. Therefore, the input feature vector X I may be represented as {h k ,A k ,vol k ,N k ,color k ,dist k}.

[0087] Step S3.2: Feature extraction of the input feature vector using a multi-layer perception to obtain a first output feature quantity.

[0088] The multi-layer perception (MLP) processes the input feature vector of each point cloud cluster through multiple fully connected layers and activation functions, mapping it to a higher-dimensional feature space, which can be represented as:

[0089] MLP(X I ) = f(W 12 ·f(W 11 ·X I +b 11 )+b 12 ) (7)

[0090] where W 11 and b 11 represent the weight matrix and bias vector of the first layer of the multi-layer perception MLP, respectively, W 12 and b 12 represent the weight matrix and bias vector of the second layer of the multi-layer perception MLP, respectively, and f represents a nonlinear activation function ReLU, i.e., MLP(X I ) is the first output feature quantity.

[0091] Step S3.3: taking the product of the first output feature quantity and each point in the point cloud cluster as the input of the Encoder layer in the Pointnet classification network, performing feature learning and mapping by the Encoder layer in the Pointnet classification network to obtain a second output feature quantity.

[0092] As shown in Figure 4 , the first output feature quantity is an N k ×6 feature matrix, each row of the feature matrix is a feature vector (s1, s2, s3, s4, s5, s6), and there are N k rows in total. Each point in the point cloud cluster has 6 dimensions, i.e., 3 coordinate values and 3 color values. The 6 dimensions of the i-th point in the k-th point cloud cluster can be represented as (x ki ,y ki ,z ki ,r ki ,g ki ,b ki ). All points in the point cloud cluster form an N k ×6 matrix. The first output feature quantity is multiplied point by point with the matrix formed by all points in the point cloud cluster to obtain a Hadamard product, and the Encoder layer in the Pointnet classification network performs feature learning and mapping on the Hadamard product. Specifically, the Hadamard product is first mapped to a high-dimensional feature space by a multi-layer perceptron shared by the feature extraction layer, and then the high-dimensional features are aggregated into a global feature by a max-pooling operation of a symmetric function. The max-pooling operation is applied to the high-dimensional features of all points, and by taking the maximum value in each dimension, all local features are integrated into a global feature vector (i.e., the second output feature quantity), and each element of the global feature vector is the maximum value of the feature values of all points in the corresponding dimension.

[0093] Step S3.4: performing feature extraction on the second output feature quantity by a fully connected layer to obtain a third output feature quantity.

[0094] In this embodiment, the fully connected layer includes three, i.e., a first fully connected layer, a second fully connected layer and a third fully connected layer connected in sequence. The third output feature quantity includes three components, and the three components are a tower type score, a wire type score and an environment type score, respectively.

[0095] Step S3.5: performing prediction on the third output feature quantity by a classification layer to obtain the type of the point cloud cluster.

[0096] The classification layer includes a Softmax function and an output layer. The core role of the Softmax function in a multi-classification task is to convert each type score into a probability distribution, so that the sum of the probabilities of all types is 1. The Softmax function first performs an exponential transformation on each type score, converts it into a positive number, ensures that all converted results are positive numbers, and magnifies larger type scores and reduces smaller type scores. Then, the exponential transformed values are normalized to obtain the probability of each type. The specific formula is:

[0097]

[0098] where T i represents the probability of the i-th type, t i represents the i-th type score, and n represents the number of types.

[0099] Through analysis, it is found that the proportion of tower and pole point clouds in the overall point cloud is less than 10%. Therefore, if the type of a point cloud cluster is identified and the probability of the tower and pole type is greater than 0.1, it is considered that the type of the point cloud cluster is a tower and pole.

[0100] Step S3.6: merging the point cloud clusters of the type of tower and pole to obtain the tower and pole point cloud, as shown in Figure 5 and Figure 6 .

[0101] The training process of the wire tower recognition model is similar to the type recognition process. Before training, a sample data set needs to be constructed. Each sample in the sample data set includes an input sample and an output sample. The input sample is a vector composed of the statistical features of the point cloud cluster, and the output sample is the type of the corresponding point cloud cluster. The wire tower recognition model is trained and verified using the sample data set, and a trained wire tower recognition model is obtained.

[0102] Step S4: determining whether the number of vertical poles is greater than 1; when the number of vertical poles is greater than 1, the tower and pole point cloud is divided into pole levels according to the number of vertical poles to obtain vertical pole point clouds; when the number of vertical poles is equal to 1, the tower and pole point cloud is taken as the vertical pole point cloud.

[0103] The tower and pole in the distribution network usually includes vertical poles, cross arms, and fittings, insulator strings and other components hung on both ends of the cross arm. The inspection of the tower and pole is mainly the inspection of the fittings, insulator strings and other components on both ends of the cross arm. Therefore, it is necessary to determine the position of the fittings, insulator strings and other components in the tower and pole point cloud (i.e., the inspection target position). In order to determine the inspection target position, the tower and pole point cloud needs to be divided into pole levels to obtain vertical pole point clouds, and then the vertical pole point clouds are divided to obtain cross arm point clouds, and finally the inspection target position is determined in the cross arm point clouds. According to the number of vertical poles M t, the tower is divided into single-pole tower, double-pole tower and three-pole tower, for single-pole tower (i.e. the number of vertical poles M t equals 1), without stratifying the tower point cloud, directly taking the tower point cloud as the vertical pole point cloud; for double-pole tower or three-pole tower (i.e. the number of vertical poles M t is greater than 1), the tower point cloud needs to be stratified by pole level to obtain each vertical pole point cloud. In the specific embodiments of the present application, stratifying the tower point cloud by pole level according to the number of vertical poles comprises:

[0104] Step S4.1: calculating the centroid of the tower point cloud, the specific calculation formula is:

[0105]

[0106] Wherein, C t represents the centroid of the tower point cloud, N t represents the number of points in the tower point cloud, P ti represents the i-th point in the tower point cloud.

[0107] Step S4.2: constructing a first covariance matrix according to the centroid of the tower point cloud, the specific formula is:

[0108]

[0109] Wherein, Cov t represents the first covariance, i.e. the covariance of the tower point cloud, and the superscript T represents transposition.

[0110] Step S4.3: performing eigenvalue decomposition on the first covariance matrix, and taking the eigenvector corresponding to the minimum eigenvalue as the normal vector of the principal plane.

[0111] Wherein, the principal plane refers to the plane composed of vertical poles, and the normal vector of the principal plane is perpendicular to the principal plane.

[0112] Step S4.4: based on the normal vector of the principal plane, equally dividing the tower point cloud by M t , to obtain M t vertical pole point clouds.

[0113] Examples of stratifying the tower point cloud by pole level are shown in Figure 7 and Figure 8 .

[0114] Step S5: stratifying each vertical pole point cloud according to the cross arm height to obtain a cross arm point cloud.

[0115] According to the tower construction design scheme, the cross arm height is obtained, the range of the cross arm in each vertical rod point cloud is determined according to the cross arm height, and then the range is layered to obtain the cross arm point cloud. For example, the cross arm height is 0.2 meters, for each vertical rod point cloud, layering is performed in the direction parallel to the main plane and vertically upward at intervals of 0.2 meters to obtain each cross arm point cloud, and the tower structure segmentation is completed.

[0116] Step S6: Calculate the positions of the two end points of the main axis direction of each cross arm point cloud.

[0117] In the power inspection task of the unmanned aerial vehicle, the position information of key components such as fittings and insulator strings is the key and basis for inspection, and these key components are often installed or mounted at the two ends of the cross arm. Therefore, the two end point positions of the cross arm point cloud are determined, that is, the inspection target positions are determined. In the specific embodiment of the present application, the positions of the two end points of the main axis direction of each cross arm point cloud are calculated, including:

[0118] Step S6.1: Calculate the centroid of the cross arm point cloud, and the specific calculation formula is similar to formula (9);

[0119] Step S6.2: Construct a second covariance matrix according to the centroid of the cross arm point cloud, and the specific construction formula is similar to formula (10);

[0120] Step S6.3: Perform eigenvalue decomposition on the second covariance matrix, and take the eigenvector corresponding to the maximum eigenvalue as the main axis direction of the cross arm point cloud;

[0121] Step S6.4: Determine the two points located in the main axis direction and farthest from the centroid of the cross arm point cloud as the two end point positions of the main axis direction of the cross arm point cloud, that is, the two inspection target positions.

[0122] Determine the two points located in the main axis direction and farthest from the centroid of the cross arm point cloud, specifically: calculate the projection distance of each point in the cross arm point cloud and the centroid of the cross arm point cloud in the main axis direction; calculate the point farthest in the negative value range as one end point of the cross arm, that is, min{(P cai -C ca )·u m |i=1,2,…,N ca} corresponding point in the cross arm point cloud is one of the end points, P cai represents the i-th point in the cross arm point cloud, C ca represents the centroid of the cross arm point cloud, u m represents the main axis direction of the cross arm point cloud (i.e. the eigenvector corresponding to the maximum eigenvalue), N ca represents the number of points in the cross arm point cloud; calculate the point farthest in the positive value range as the other end point, that is, max{(P cai -C ca)·u m |i=1,2,…,N ca The point in the crossbeam point cloud corresponding to} is another endpoint.

[0123] From each crossarm point cloud, two inspection target locations (i.e., two endpoint locations) are determined, thus obtaining the locations of all inspection targets, such as... Figure 9 and Figure 10 As shown.

[0124] Step S7: Determine the inspection and shooting points based on the positions of the two endpoints of the main axis direction of each crossbow point cloud.

[0125] To ensure that the camera does not collide with the inspection target location during inspection filming, for each crossbow point cloud, it is necessary to extend outwards along the main axis from its two endpoints (i.e., the inspection target location) by d. s meters, with d s Meters are used as a safety distance. Therefore, the inspection and camera locations can be represented as:

[0126]

[0127] Among them, P hover Indicates the location of the inspection and filming point, P target It indicates the location of the inspection target, the safe distance, and the main axis direction of the crossbeam point cloud.

[0128] Step S8: Calculate the yaw angle of the drone and the pitch angle of the camera based on the inspection and shooting points.

[0129] Two inspection target positions (i.e., two endpoint positions) are determined from each crossarm point cloud, thus obtaining all inspection target positions. Based on each inspection target position, the corresponding inspection shooting point is determined, thus obtaining all inspection shooting points, i.e., the drone hovering position.

[0130] Drones typically inspect towers in the order of lower left, upper left, upper center, upper right, and lower right. The yaw angle during the inspection is determined by the two adjacent hovering positions or inspection shooting points in the inspection sequence, using the following formula:

[0131]

[0132] Where, θ q x represents the yaw angle of the drone at the q-th inspection and shooting point. s,q and y s,q Let x and y represent the x and y coordinates of the q-th inspection and shooting point, respectively. s,q+1 and y s,q+1 Let x and y represent the x and y coordinates of the (q+1)th inspection and shooting point, respectively, and arctan represent the arctangent function.

[0133] The pitch angle of the camera is calculated according to the inspection shooting point position, so as to ensure that the image of the key part is shot, and the calculation formula of the pitch angle of the camera is:

[0134]

[0135] wherein, represents the pitch angle of the camera (or a gimbal for controlling the movement of the camera) at the qth inspection shooting point position, x qh , y qh , z qh represents the coordinate position of the qth inspection shooting point position, x qt , y qt , z qt represents the qth inspection target position.

[0136] Embodiment 2

[0137] The power line tower unmanned aerial vehicle inspection method provided by the embodiment of the application comprises the following steps:

[0138] Step S1: obtaining the inspection shooting point position by using the power distribution network unmanned aerial vehicle inspection shooting point generation method based on tower point cloud semantic segmentation as described in Embodiment 1;

[0139] Step S2: generating an inspection flight path according to the inspection shooting point position in the order of left bottom, left top, middle top, right top to right bottom;

[0140] Step S3: controlling the unmanned aerial vehicle to perform an inspection task according to the inspection flight path.

[0141] In order to ensure the safe flight of the unmanned aerial vehicle, the KML file of the inspection flight path and the original point cloud data are jointly visualized to check whether the inspection flight path collides with the tower, the environment and the like, and whether the inspection shooting point position and the inspection flight path meet the overall safety requirements, as shown in Figure 11 When the inspection is confirmed to be correct, the inspection flight path KML file is imported into the unmanned aerial vehicle to perform the inspection task.

[0142] Embodiment 3

[0143] The embodiment of the application further provides an electronic device, which comprises a memory, a processor and a computer program / instruction stored in the memory, and the processor executes the computer program / instruction to implement the power distribution network unmanned aerial vehicle inspection shooting point generation method based on tower point cloud semantic segmentation or the power line tower unmanned aerial vehicle inspection method in the embodiments of the application.

[0144] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes according to programs and / or data stored in a read-only memory (ROM) or programs and / or data loaded from a storage section into a random access memory (RAM). The processor can be one multi-core processor or can include a plurality of processors. In some embodiments, the processor can include a general-purpose main processor and one or more special-purpose co-processors, such as a central processing unit, a graphics processing unit (GPU), a neural processing unit (NPU), a digital signal processor (DSP), and the like. In the RAM, various programs and data required for device operation are also stored. The processor, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0145] The above processor and memory are used together to execute programs / instructions stored in the memory, which, when executed by a computer, can implement the methods, steps, or functions described in the above embodiments.

[0146] Although not shown, the embodiments of the present application also provide a computer-readable storage medium having stored thereon computer programs / instructions, which, when executed by a processor, implement the power line distribution unmanned aerial vehicle inspection shooting point generation method based on tower point cloud semantic segmentation or the power pole unmanned aerial vehicle inspection method in the embodiments of the present application.

[0147] The storage medium of the embodiments of the present application includes permanent and non-permanent, removable and non-removable, and information storage can be realized by any method or technology. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0148] Read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile discs (DVD) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. According to the definition provided herein, computer readable medium does not include transitory computer readable medium, such as a modulated data signal and a carrier wave.

[0149] Although not shown, the embodiments of the present application also provide a computer program product, comprising: computer programs / instructions, which, when executed by a processor, implement the power distribution line unmanned aerial vehicle inspection shooting point generation method based on tower point cloud semantic segmentation or the utility pole unmanned aerial vehicle inspection method in the embodiments of the present application.

[0150] The above only discloses specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or modifications within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for generating UAV inspection and shooting points for power distribution networks based on semantic segmentation of tower point clouds, characterized in that, The generation method includes: Obtain raw point cloud data containing utility poles and towers; The original point cloud data is clustered to obtain multiple point cloud clusters; Type identification is performed on each of the point cloud clusters to obtain the tower point cloud; When the number of vertical poles is greater than 1, the pole point cloud is divided into pole-level layers according to the number of vertical poles to obtain the vertical pole point cloud; when the number of vertical poles is equal to 1, the pole point cloud is used as the vertical pole point cloud. The point cloud of each vertical pole is layered according to the height of the crossarm to obtain the point cloud of the crossarm; Calculate the positions of the two endpoints along the principal axis of each crossarm point cloud; The inspection and shooting points are determined based on the positions of the two endpoints of the main axis direction of each crossbeam point cloud. The method of classifying the pole point cloud according to the number of poles includes: Calculate the centroid of the tower point cloud; Construct the first covariance matrix based on the centroid of the tower point cloud; The first covariance matrix is ​​decomposed into eigenvalues, and the eigenvector corresponding to the smallest eigenvalue is used as the normal vector of the principal plane; wherein, the principal plane refers to the plane formed by the vertical rods; Based on the normal vector of the principal plane, the point cloud of the tower is analyzed. Divide into equal parts, and obtain A vertical pole dotted the cloud; among them... This indicates the number of vertical poles included in the tower.

2. The method for generating UAV inspection and shooting points for power distribution networks based on pole point cloud semantic segmentation according to claim 1, characterized in that, The trained utility pole and tower recognition model is invoked to identify the type of each point cloud cluster; wherein, the utility pole and tower recognition model includes a multilayer perceptron, an Encoder layer, a fully connected layer, and a classification layer in the Pointnet classification network connected in sequence; The process of identifying the type of each point cloud cluster specifically includes: Calculate the statistical features of the point cloud cluster, and construct an input feature vector from all the statistical features of the point cloud cluster; The input feature vector is extracted using a multilayer perceptron to obtain the first output feature quantity; The product of the first output feature and each point in the point cloud cluster is used as the input to the Encoder layer in the Pointnet classification network. The Encoder layer in the Pointnet classification network is used to perform feature learning and mapping to obtain the second output feature. The second output feature is extracted using a fully connected layer to obtain the third output feature. The classification layer is used to predict the third output feature quantity to obtain the type of the point cloud cluster; wherein, the type of point cloud cluster includes power lines, towers, and environment; Merge point cloud clusters of type "tower" to obtain tower point clouds.

3. The method for generating UAV inspection and shooting points for power distribution networks based on pole point cloud semantic segmentation according to claim 2, characterized in that, The statistical characteristics of the point cloud cluster include the cluster's height, horizontal area, volume, cluster size, color value, and the distance between the cluster center and the tower reference point; wherein, the calculation formula for each statistical characteristic is as follows: ； ； ； ； ； in, Indicates the first The height of a point cloud cluster and They represent the first The maximum and minimum ordinate values ​​of points in a point cloud cluster. Indicates the first The area of ​​a point cloud cluster, and They represent the first The maximum and minimum x-coordinate values ​​of points in a point cloud cluster and They represent the first The maximum and minimum ordinate values ​​of points in a point cloud cluster. Indicates the first The volume of a point cloud cluster Indicates the first The color value of a point cloud cluster Indicates the first The cluster size of a point cloud cluster Indicates the first The first point cloud cluster The color value of each point, Indicates the first The distance between the cluster center point and the tower reference point in a point cloud cluster Indicates the first The first point cloud cluster The coordinates of each point This indicates the coordinates of the tower's reference point.

4. The method for generating UAV inspection and shooting points for power distribution networks based on pole point cloud semantic segmentation according to claim 1, characterized in that, The calculation of the positions of the two endpoints along the principal axis of each crossbow point cloud includes: Calculate the centroid of the crossbow point cloud; Construct a second covariance matrix based on the centroid of the crossarm point cloud; The second covariance matrix is ​​decomposed into eigenvalues, and the eigenvector corresponding to the largest eigenvalue is taken as the principal axis direction of the cross-arm point cloud. Determine the two points located in the main axis direction that are farthest from the centroid of the crossarm point cloud, and use these two points as the two endpoints of the main axis direction of the crossarm point cloud.

5. The method for generating UAV inspection and shooting points for power distribution networks based on pole point cloud semantic segmentation according to claim 1, characterized in that, The generation method also includes calculating the yaw angle of the UAV and the pitch angle of the camera based on the inspection and shooting points. The specific calculation formula is as follows: ； ； in, Indicates that the drone is in Yaw angle at each inspection and shooting point and They represent the first The x and y coordinates of each inspection and photography point and They represent the first The x and y coordinates of each inspection and photography point Indicates the camera at the The tilt angle of each inspection and shooting point. Indicates the first The coordinates of each inspection and photography point. Indicates the first The endpoint position or the first There are several inspection target locations, where arctan represents the arctangent function.

6. A method for unmanned aerial vehicle (UAV) inspection of utility poles, characterized in that, The inspection method includes: The inspection and shooting points are obtained using the method for generating UAV inspection and shooting points for power distribution networks based on the semantic segmentation of pole point clouds as described in any one of claims 1 to 5; An inspection route is generated based on the inspection and photography locations. The drone is controlled to perform inspection tasks according to the inspection route.

7. An electronic device comprising a memory, a processor, and a computer program / instructions stored in the memory, characterized in that, The processor executes the computer program / instructions to implement the method for generating shooting points for power distribution network drone inspection based on pole point cloud semantic segmentation as described in any one of claims 1 to 5, or the method for drone inspection of power poles as described in claim 6.

8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the method for generating shooting points for power distribution network drone inspection based on pole point cloud semantic segmentation as described in any one of claims 1 to 5, or the method for drone inspection of power poles as described in claim 6.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the method for generating shooting points for power distribution network drone inspection based on pole point cloud semantic segmentation as described in any one of claims 1 to 5, or the method for drone inspection of power poles as described in claim 6.

Images