Distribution network unmanned aerial vehicle inspection shooting point generation method based on tower point cloud semantic segmentation and related device
Through the method based on the semantic segmentation of pole point clouds, the drone inspection shooting points are automatically generated, which solves the problem of time-consuming and labor-intensive manual labeling and relies on professional knowledge, and improves the efficiency and application convenience of drone inspection.
Patent Information
- Application Number
- CN202411139083.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-20
AI Technical Summary
In the existing drone inspection, it takes a lot of time and labor costs, and requires professional knowledge and experience, which leads to difficulties in practical application promotion.
Using a semantic segmentation method based on pole point cloud, the original point cloud data is obtained for clustering, type identification, and layering, and the inspection shooting points are automatically generated. Multi-layer perceptron and Pointnet classification networks are used to identify the pole tower point cloud, and the calculation is carried out. The end point position of the main axis direction of the cross-burst point cloud is determined to determine the inspection shooting point.
It realizes automatic generation of drone inspection shooting points, reduces the time and cost of manual labeling, reduces the dependence on professional knowledge, and improves the convenience of application promotion.
Smart Images

Figure CN119048785A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of data processing technology, and in particular relates to a distribution network drone inspection shooting point generation method based on tower point cloud semantic segmentation, an inspection method, equipment, a storage medium and a product. Background Art
[0002] Power lines are widely distributed and mostly located in the wild. The harsh environment in the wild has brought tremendous pressure to the normal operation of the power system. Power inspections regularly inspect and maintain power equipment and lines to promptly discover and eliminate potential faults and hidden dangers and prevent power accidents. There are two main methods of power inspections: one is traditional manual inspections, and the other is drone inspections.
[0003] The traditional manual inspection method requires professional maintenance personnel to go to the power equipment site to check the damage and operating status of the power equipment one by one. Usually, maintenance personnel are required to climb the power tower to observe the operating status of the power equipment at close range. This method brings great safety risks to maintenance personnel, especially in bad weather or complex on-site environments, and requires a lot of labor costs.
[0004] In recent years, with the rapid development of remote sensing technology and communication technology, the application of drones has been widely promoted and gradually applied to various industries. Drones are flexible, low-cost and highly maneuverable. They can adapt well to various complex environments and show obvious advantages in power inspections. Especially in complex environments with high risks and difficult to reach, drones can be equipped with visible light cameras, fly to the inspection target tower or component, take pictures of the current status of the equipment, and use these pictures for subsequent defect identification and equipment maintenance. This not only eliminates the safety risks of operation and maintenance personnel, but also greatly improves the inspection efficiency and promotes the intelligence and automation level of power inspections. However, the existing drone inspections require operators to operate the drones to ensure that the drones can take pictures of the components to be inspected, which still places high demands on operators.
[0005] In order to reduce the operator's requirements for the operation of the drone, the laser radar is used to carry the drone to collect the point cloud data of the power pole tower in advance, and then the inspection trajectory of the drone is generated based on the point cloud data. Point cloud data is a series of disordered and discrete points in three-dimensional space, mainly containing three-dimensional coordinates and color information, which can reflect the real geographical location information and environmental information in the actual scene. Therefore, using the collected point cloud data, the real coordinates of the inspection target on the power pole tower and the real coordinates where the drone needs to hover during the inspection process can be calibrated. The existing inspection target marking method is mainly manual marking. The inspection trajectory of the drone can be further formed through the manually marked inspection target points, so that the drone can perform the power inspection task according to the predetermined inspection trajectory. Although this method has significantly improved safety and efficiency compared with manual inspection, it still consumes a lot of time and labor costs in the manual calibration link, and relies on expert knowledge and professional experience in power grid inspection. It has high requirements for personnel who draw the inspection trajectory, resulting in certain difficulties in practical application and promotion. Summary of the invention
[0006] The purpose of the present invention is to provide a method for generating inspection shooting points, an inspection method, equipment, medium and product, so as to solve the problem that manual marking of inspection target points is time-consuming, has high labor costs, requires reliance on professional knowledge and experience, and is difficult to promote in practical applications.
[0007] The present invention solves the above technical problems through the following technical solutions: a method for generating shooting points for distribution network drone inspection based on semantic segmentation of tower point clouds, comprising:
[0008] Obtain original point cloud data containing electric poles and towers;
[0009] Clustering the original point cloud data to obtain multiple point cloud clusters;
[0010] Performing type identification on each of the point cloud clusters to obtain a tower point cloud;
[0011] When the number of vertical poles is greater than 1, the pole tower point cloud is layered according to the number of vertical poles to obtain a vertical pole point cloud; when the number of vertical poles is equal to 1, the pole tower point cloud is used as the vertical pole point cloud;
[0012] Layering each of the vertical pole point clouds according to the cross arm height to obtain a cross arm point cloud;
[0013] Calculate the two endpoint positions of the main axis direction of each cross arm point cloud;
[0014] The inspection shooting points are determined according to the two end point positions of the main axis direction of each cross arm point cloud.
[0015] Further, calling the trained electric pole tower recognition model to perform type recognition on each point cloud cluster; wherein the electric pole tower recognition model includes a multi-layer perceptron, an encoder layer in a Pointnet classification network, a fully connected layer, and a classification layer connected in sequence;
[0016] Performing type identification on each of the point cloud clusters, specifically including:
[0017] Calculating statistical features of the point cloud cluster, and forming an input feature vector from all statistical features of the point cloud cluster;
[0018] Using a multi-layer perceptron to extract features from the input feature vector to obtain a first output feature quantity;
[0019] The product of the first output feature quantity and each point in the point cloud cluster is used as the input of the Encoder layer in the Pointnet classification network, and the Encoder layer in the Pointnet classification network is used to perform feature learning and mapping to obtain a second output feature quantity;
[0020] Using a fully connected layer to extract features from the second output feature quantity to obtain a third output feature quantity;
[0021] The third output feature quantity is predicted by using the classification layer to obtain the type of the point cloud cluster; wherein the type of the point cloud cluster includes electric wires, poles and towers, and environment;
[0022] The point cloud clusters of the tower type are merged to obtain the tower point cloud.
[0023] Furthermore, the statistical features of the point cloud cluster include the height, horizontal area, volume, cluster size, color value, and distance between the cluster center point and the tower reference point of the point cloud cluster; wherein the calculation formula of each statistical feature is:
[0024] h k =z kmax -z kmin ;
[0025] A k =(x kmax -x kmin )×(y kmax -y kmin );
[0026] vol k =h k ×A k ;
[0027]
[0028]
[0029] Among them, h k Indicates the height of the k-th point cloud cluster, z kmax and z kmin Respectively represent the maximum vertical coordinate value and the minimum vertical coordinate value of the point in the k-th point cloud cluster, A k represents the area of the kth point cloud cluster, x kmax and x kmin Respectively represent the maximum and minimum abscissa values of the points in the k-th point cloud cluster, y kmax and kmin Respectively represent the maximum and minimum ordinate values of the points in the k-th point cloud cluster, vol k Indicates the volume of the kth point cloud cluster, color k Represents the color value of the kth point cloud cluster, N k represents the cluster size of the kth point cloud cluster, r ki ,g ki ,b ki Indicates the color value of the i-th point in the k-th point cloud cluster, dist k represents the distance between the center point of the kth point cloud cluster and the tower reference point, x ki ,y ki ,z ki represents the coordinate value of the i-th point in the k-th point cloud cluster, x r ,y r ,z r Indicates the coordinate value of the tower reference point.
[0030] Further, the tower point cloud is layered according to the number of vertical poles, including:
[0031] Calculating the centroid of the tower point cloud;
[0032] Constructing a first covariance matrix according to the centroid of the tower point cloud;
[0033] Performing eigenvalue decomposition on the first covariance matrix, and taking the eigenvector corresponding to the minimum eigenvalue as the normal vector of the main plane; wherein the main plane refers to the plane formed by the vertical rods;
[0034] Based on the normal vector of the main plane, the tower point cloud is M t Divide equally and get M t Vertical point clouds; among them, M t Indicates the number of vertical poles contained in the tower.
[0035] Further, calculating the two endpoint positions of the main axis direction of each crossarm point cloud includes:
[0036] Calculating the centroid of the crossarm point cloud;
[0037] constructing a second covariance matrix according to the centroid of the crossarm point cloud;
[0038] Performing eigenvalue decomposition on the second covariance matrix, and taking the eigenvector corresponding to the maximum eigenvalue as the principal axis direction of the cross-arm point cloud;
[0039] Two points located in the main axis direction and farthest from the centroid of the cross-arm point cloud are determined, and the two points are used as two end point positions in the main axis direction of the cross-arm point cloud.
[0040] Furthermore, the generation method also includes calculating the yaw angle of the drone and the pitch angle of the camera according to the inspection shooting points, and the specific calculation formula is:
[0041]
[0042]
[0043] Among them, θ q represents the yaw angle of the drone at the qth inspection point, x s,q and s,q They represent the horizontal and vertical coordinates of the qth inspection point, respectively. s,q+1 and s,q+1 They represent the horizontal and vertical coordinates of the q+1th inspection and shooting point, respectively. Indicates the pitch angle of the camera at the qth inspection point, x qh ,y qh ,z qh Indicates the coordinate position of the qth inspection shooting point, x qt ,y qt ,z qt represents the qth endpoint position or the qth inspection target position, and arctan represents the inverse tangent function.
[0044] The present invention provides a method for inspecting electric pole towers using an unmanned aerial vehicle, comprising:
[0045] The inspection shooting points are obtained by using the distribution network drone inspection shooting point generation method based on the tower point cloud semantic segmentation as described above;
[0046] Generate an inspection route according to the inspection shooting points;
[0047] According to the inspection route, the drone is controlled to perform the inspection task.
[0048] The present invention provides an electronic device, comprising a memory, a processor, and a computer program / instruction stored in the memory, wherein the processor executes the computer program / instruction to implement the distribution network drone inspection shooting point generation method or the power pole tower drone inspection method based on pole tower point cloud semantic segmentation as described above.
[0049] The present invention provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the distribution network drone inspection shooting point generation method based on tower point cloud semantic segmentation or the power pole tower drone inspection method as described above is implemented.
[0050] The present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the distribution network drone inspection shooting point generation method based on tower point cloud semantic segmentation or the power pole tower drone inspection method as described above.
[0051] Compared with the prior art, the advantages of the present invention are:
[0052] The present invention clusters original point cloud data of electric poles and towers to obtain multiple point cloud clusters; determines the electric wire point cloud, pole tower point cloud and environment point cloud by performing type recognition on the point cloud clusters; stratifies the pole tower point cloud according to the pole tower type and the crossarm height to determine the crossarm point cloud; finally, determines the positions of target parts such as hardware and insulator strings at both ends of the crossarm from the crossarm point cloud, and then determines the inspection shooting points, thereby realizing the automatic generation of drone inspection shooting points, and solving the problems of time-consuming and high labor cost of manual marking of inspection target points, relying on professional knowledge and experience, and being difficult to promote in actual application. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0054] Figure 1 It is a flow chart of a method for generating shooting points for distribution network drone inspection based on semantic segmentation of tower point cloud in an embodiment of the present invention;
[0055] Figure 2 1 is a visualization example of point cloud data clustering results in an embodiment of the present invention;
[0056] Figure 3 2 is a visualization example of point cloud data clustering results in an embodiment of the present invention;
[0057] Figure 41 is a diagram of the architecture of a power pole tower identification model in an embodiment of the present invention;
[0058] Figure 5 1 is an example of visualization of a tower point cloud in an embodiment of the present invention;
[0059] Figure 6 2 is an example of visualization of a tower point cloud in an embodiment of the present invention;
[0060] Figure 7 is an example diagram of the visualization of the pole-level stratification in an embodiment of the present invention; wherein different colors represent different stratifications;
[0061] Figure 8 is an example diagram of highly layered visualization in an embodiment of the present invention; wherein different colors represent different layers;
[0062] Fig. 9 1 is an example of visualization of the tower point cloud and cross arm endpoints in an embodiment of the present invention;
[0063] Fig.10 2 is an example of visualization of the tower point cloud and cross arm endpoints in an embodiment of the present invention;
[0064] Fig.11 This is an example diagram of visual verification of the inspection route KML file in an embodiment of the present invention. DETAILED DESCRIPTION
[0065] The following is a clear and complete description of the technical solutions in the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0066] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0067] Example 1
[0068] like Figure 1 As shown, a method for generating shooting points for distribution network drone inspection based on tower point cloud semantic segmentation provided by an embodiment of the present invention includes the following steps:
[0069] Step S1: Obtain original point cloud data containing electric poles and towers.
[0070] The original point cloud data including electric poles and towers are collected by using a laser radar mounted on a drone. The original point cloud data includes multiple points, and each point includes coordinate information and color information.
[0071] Step S2: Cluster the original point cloud data to obtain multiple point cloud clusters.
[0072] In a specific embodiment of the present invention, a density-based spatial clustering algorithm (Density-Based Spatial Clustering of Applications with Noise, DBSCAN) is used to cluster the original point cloud data to obtain multiple point cloud clusters. The DBSCAN algorithm uses the neighborhood radius ε and the minimum number of points MinPts to perform spatial clustering on the original point cloud data, where ε defines the neighborhood range of the point, that is, all points within a distance of ε from a point are considered to be adjacent, and the ε-neighborhood (N ε (p)) can be expressed as:
[0073] N ε (p)={q∈D|dist(p,q)≤ε} (1)
[0074] Where D represents the original point cloud data, p and q represent the points in the original point cloud data, dist(p,q) represents the distance between point p and point q, and the neighborhood contains all points q whose distance to point p does not exceed ε; MinPts refers to the minimum number of points required for a point p in its ε neighborhood. If the number of points of a point in its ε neighborhood is greater than MinPts, the point is considered to be a core point, and the core point and all the points in its ε neighborhood together constitute a point cloud cluster, thereby obtaining several point cloud clusters separated in space and with different densities, such as Figure 2 and Figure 3 As shown, Cluster represents cluster, Noise represents noise point, different colors represent different point cloud clusters, the horizontal coordinate is the X coordinate of the UTM coordinate system, the vertical coordinate is the Y coordinate of the UTM coordinate system, and the vertical coordinate is the Z coordinate of the UTM coordinate system.
[0075] Although the distribution network scene is complex, the key parts of the towers often have a certain spatial distance from the surrounding environment (such as buildings, vegetation, etc.) to meet the safety requirements of the distribution network, and entities of different materials often present different point densities in the point cloud data. Based on this, the DBSCAN algorithm is used to separate the original point cloud data containing the towers and the surrounding environment to obtain different types of point cloud clusters. At the same time, the DBSCAN algorithm has the ability to identify noise points and can filter out noise points in the original point cloud data.
[0076] Step S3: Perform type identification on each point cloud cluster to obtain the tower point cloud.
[0077] In a specific implementation of the present invention, the trained electric pole tower recognition model is called to perform type recognition on each point cloud cluster. Figure 4As shown in the figure, the telegraph pole recognition model includes a multi-layer perceptron, an encoder layer in the Pointnet classification network, a fully connected layer, and a classification layer connected in sequence. The trained telegraph pole recognition model is used to identify the type of each point cloud cluster, including:
[0078] Step S3.1: Calculate the statistical features of the point cloud cluster, and form an input feature vector from all the statistical features of the point cloud cluster.
[0079] In a specific embodiment of the present invention, the statistical features of the point cloud cluster include the height, horizontal area, volume, cluster size, color value and the distance between the cluster center point and the tower reference point of the point cloud cluster. The specific calculation formula of each statistical feature is:
[0080] h k =z kmax -z kmin (2)
[0081] A k =(x kmax -x kmin )×(y kmax -y kmin ) (3)
[0082] vol k =h k ×A k (4)
[0083]
[0084]
[0085] Among them, h k Indicates the height of the k-th point cloud cluster, z kmax and z kmin Respectively represent the maximum vertical coordinate value and the minimum vertical coordinate value (i.e., Z coordinate value) of the point in the k-th point cloud cluster, A k represents the area of the kth point cloud cluster, x kmax and x kmin They represent the maximum and minimum abscissa values (i.e., X coordinate values) of the points in the kth point cloud cluster, respectively. kmax and kmin They represent the maximum ordinate value and the minimum ordinate value (i.e., Y coordinate value) of the point in the k-th point cloud cluster, vol k Indicates the volume of the kth point cloud cluster, color k Represents the color value of the kth point cloud cluster, N k represents the cluster size of the kth point cloud cluster, r ki ,g ki ,b kiIndicates the color value of the i-th point in the k-th point cloud cluster, dist k represents the distance between the center point of the kth point cloud cluster and the tower reference point (i.e., center deviation), x ki ,y ki ,z ki represents the coordinate value of the i-th point in the k-th point cloud cluster, x r ,y r ,z r Indicates the coordinate value of the tower reference point. All coordinate values are based on the UTM (Universal Transverse Mercator) projection coordinate system.
[0086] The tower reference point is the theoretical coordinate point obtained in the tower construction design stage, and is the input parameter of the method of the present invention. I It can be expressed as {h k ,A k ,vol k ,N k ,color k ,dist k}.
[0087] Step S3.2: Use a multi-layer perceptron to extract features from the input feature vector to obtain a first output feature quantity.
[0088] The multi-layer perceptron (MLP) processes the input feature vector of each point cloud cluster through multiple fully connected layers and activation functions, mapping it to a higher-dimensional feature space, which can be expressed as:
[0089] MLP(X I )=f(W 12 ·f(W 11 ·X I +b 11 )+b 12 ) (7)
[0090] Among them, W 11 and b 11 Respectively represent the weight matrix and bias vector of the first layer of the multilayer perceptron MLP, W 12 and b 12 They represent the weight matrix and bias vector of the second layer of the multilayer perceptron MLP, respectively, and f represents the nonlinear activation function ReLU, that is, MLP(X I ) is the first output feature quantity.
[0091] Step S3.3: The product of the first output feature value and each point in the point cloud cluster is used as the input of the Encoder layer in the Pointnet classification network, and the Encoder layer in the Pointnet classification network is used to perform feature learning and mapping to obtain the second output feature value.
[0092] like Figure 4 As shown, the first output feature quantity is an N k ×6 feature matrix, each row of which is a feature vector (s1, s2, s3, s4, s5, s6), with a total of N k Each point in the point cloud cluster has 6 dimensions, namely 3 coordinate values and 3 color values. The 6 dimensions of the i-th point in the k-th point cloud cluster can be expressed as: (x ki ,y ki ,z ki ,r ki ,g ki ,b ki ), all points in the point cloud cluster constitute N k ×6 matrix. The first output feature quantity is multiplied point by point with the matrix composed of all points in the point cloud cluster to obtain the Hadamard product. The Encoder layer in the Pointnet classification network performs feature learning and mapping on the Hadamard product. Specifically, the Hadamard product is first mapped to a high-dimensional feature space through a multi-layer perceptron shared by the feature extraction layer, and then the high-dimensional features are aggregated into global features through the symmetric function maximum pooling. The maximum pooling operation is applied to the high-dimensional features of all points. By taking the maximum value in each dimension, all local features are integrated into a global feature vector (i.e., the second output feature quantity). Each element of the global feature vector is the maximum value of the feature values of all points in the corresponding dimension.
[0093] Step S3.4: Use the fully connected layer to extract features from the second output feature quantity to obtain a third output feature quantity.
[0094] In this embodiment, the fully connected layers include three, namely, the first fully connected layer, the second fully connected layer and the third fully connected layer connected in sequence. The third output feature quantity includes three components, which are the tower type score, the wire type score and the environment type score.
[0095] Step S3.5: Use the classification layer to predict the third output feature quantity to obtain the type of the point cloud cluster.
[0096] The classification layer includes the Softmax function and the output layer. The core role of the Softmax function in multi-classification tasks is to convert each type score into a probability distribution so that the sum of the probabilities of all types is 1. The Softmax function first performs an exponential transformation on each type score and converts it into a positive number to ensure that all conversion results are positive numbers, and that larger type scores are amplified and smaller type scores are reduced. These exponentially transformed values are then normalized to obtain the probability of each type. The specific formula is:
[0097]
[0098] Among them, T i represents the probability of the i-th type, t i represents the i-th type score, and n represents the number of types.
[0099] Through analysis, it is found that the proportion of pole tower point cloud in the overall point cloud is less than 10%. Therefore, if the type of a point cloud cluster is identified and the probability of the pole tower type is greater than 0.1, the type of the point cloud cluster is considered to be a pole tower.
[0100] Step S3.6: Merge the point cloud clusters of the tower type to obtain the tower point cloud, such as Figure 5 and Figure 6 shown.
[0101] The training process of the electric pole tower recognition model is similar to the type recognition process. Before training, a sample data set needs to be constructed. Each sample in the sample data set includes an input sample and an output sample. The input sample is a vector composed of the statistical features of the point cloud cluster, and the output sample is the type of the corresponding point cloud cluster. The sample data set is used to train and verify the electric pole tower recognition model, and a trained electric pole tower recognition model is obtained.
[0102] Step S4: Determine whether the number of vertical poles is greater than 1; when the number of vertical poles is greater than 1, perform pole-level stratification on the tower point cloud according to the number of vertical poles to obtain a vertical pole point cloud; when the number of vertical poles is equal to 1, use the tower point cloud as the vertical pole point cloud.
[0103] The pole towers in the distribution network usually include vertical poles, cross arms, and hardware, insulator strings and other components mounted on both ends of the cross arms. The inspection of the pole towers mainly involves the inspection of hardware, insulator strings and other components at both ends of the cross arms. Therefore, it is necessary to determine the position of hardware, insulator strings and other components in the pole tower point cloud (i.e., the inspection target position). In order to determine the inspection target position, it is necessary to first stratify the pole tower point cloud by pole level to obtain the vertical pole point cloud, and then stratify the vertical pole point cloud to obtain the cross arm point cloud, and finally determine the inspection target position in the cross arm point cloud. According to the number of vertical poles M tThe tower is divided into single-pole tower, double-pole tower and three-pole tower. For a single-pole tower (i.e., the number of vertical poles M t =1), there is no need to stratify the tower point cloud, and the tower point cloud is directly used as the vertical pole point cloud; for double-pole towers or three-pole towers (i.e., the number of vertical poles M t If the number of vertical poles is greater than 1), the pole tower point cloud needs to be layered at the pole level to obtain the point cloud of each vertical pole. In a specific embodiment of the present invention, the pole tower point cloud is layered at the pole level according to the number of vertical poles, including:
[0104] Step S4.1: Calculate the centroid of the tower point cloud. The specific calculation formula is:
[0105]
[0106] Among them, C t Represents the centroid of the tower point cloud, N t Represents the number of points in the tower point cloud, P ti Represents the i-th point in the tower point cloud.
[0107] Step S4.2: Construct the first covariance matrix according to the centroid of the tower point cloud. The specific formula is:
[0108]
[0109] Among them, Cov t It represents the first covariance, that is, the covariance of the tower point cloud, and the superscript T represents the transpose.
[0110] Step S4.3: Perform eigenvalue decomposition on the first covariance matrix, and use the eigenvector corresponding to the minimum eigenvalue as the normal vector of the principal plane.
[0111] The main plane refers to the plane formed by the vertical rods, and the normal vector of the main plane is perpendicular to the main plane.
[0112] Step S4.4: Based on the normal vector of the principal plane, perform M on the tower point cloud. t Divide equally and get M t A vertical point cloud.
[0113] An example of pole-level stratification of the tower point cloud is as follows: Figure 7 and Figure 8 shown.
[0114] Step S5: Layer each vertical pole point cloud according to the cross arm height to obtain the cross arm point cloud.
[0115] According to the tower construction design plan, the height of the cross arm is obtained, and the range of the cross arm in each vertical pole point cloud is determined according to the height of the cross arm, and then the range is layered to obtain the cross arm point cloud. For example, the height of the cross arm is 0.2 meters, and for each vertical pole point cloud, each cross arm point cloud is layered at intervals of 0.2 meters in the vertically upward direction parallel to the main plane to obtain each cross arm point cloud, completing the tower structure segmentation.
[0116] Step S6: Calculate the two endpoint positions of each cross arm point cloud in the main axis direction.
[0117] In the power inspection mission of the drone, the position information of key components such as hardware and insulator strings is the key and basis of the inspection. These key components are often installed or mounted at both ends of the crossarm. Therefore, determining the two endpoint positions of the crossarm point cloud is to determine the inspection target position. In a specific embodiment of the present invention, calculating the two endpoint positions of the main axis direction of each crossarm point cloud includes:
[0118] Step S6.1: Calculate the centroid of the cross arm point cloud. The specific calculation formula is similar to formula (9);
[0119] Step S6.2: construct a second covariance matrix according to the centroid of the crossarm point cloud, and the specific construction formula is similar to formula (10);
[0120] Step S6.3: performing eigenvalue decomposition on the second covariance matrix, and taking the eigenvector corresponding to the maximum eigenvalue as the principal axis direction of the crossarm point cloud;
[0121] Step S6.4: Determine two points that are located in the main axis direction and are farthest from the centroid of the crossarm point cloud, and use the two points as the two end points of the main axis direction of the crossarm point cloud, that is, the two inspection target positions.
[0122] Determine the two points that are located in the main axis direction and are farthest from the centroid of the crossarm point cloud. Specifically, calculate the projection distance between each point in the crossarm point cloud and the centroid of the crossarm point cloud in the main axis direction; calculate the farthest point in the negative value range as an endpoint of the crossarm, that is, min{(P cai -C ca )·u m |i=1,2,…,N ca} The point in the cross arm point cloud corresponding to it is one of the endpoints, P cai represents the i-th point in the crossarm point cloud, C ca represents the centroid of the cross arm point cloud, u m represents the principal axis direction of the crossarm point cloud (i.e., the eigenvector corresponding to the maximum eigenvalue), N ca Indicates the number of points in the crossarm point cloud; calculate the farthest point in the positive value range as the other endpoint, that is, max{(P cai -C ca)·u m |i=1,2,…,N ca}The point in the crossarm point cloud corresponding to it is the other endpoint.
[0123] Determine two inspection target positions (i.e., two endpoint positions) from each cross arm point cloud, thereby obtaining all inspection target positions, such as Fig. 9 and Fig.10 shown.
[0124] Step S7: Determine the inspection shooting points according to the two end points of the main axis direction of each cross arm point cloud.
[0125] In order to ensure that the camera does not collide with the inspection target position during inspection shooting, for each cross arm point cloud, it is necessary to extend d outward from its two endpoints (i.e., the inspection target position) along the main axis direction. s m, in d s meters as the safety distance. Therefore, the inspection shooting points can be expressed as:
[0126]
[0127] Among them, P hover Indicates the inspection shooting point, P target Indicates the inspection target position, safety distance, and main axis direction of the crossarm point cloud.
[0128] Step S8: Calculate the yaw angle of the drone and the pitch angle of the camera according to the inspection shooting points.
[0129] Two inspection target positions (i.e., two endpoint positions) are determined from each crossarm point cloud, thereby obtaining all inspection target positions, and the corresponding inspection shooting points are determined according to each inspection target position, thereby obtaining all inspection shooting points, i.e., the hovering position of the UAV.
[0130] UAVs usually inspect towers in the order of lower left, upper left, upper middle, upper right to lower right. The yaw angle of the UAV during the inspection is determined by the two adjacent hovering positions or inspection shooting points in the inspection sequence. The specific formula is:
[0131]
[0132] Among them, θ q represents the yaw angle of the drone at the qth inspection point, x s,q and s,q They represent the horizontal and vertical coordinates of the qth inspection point, respectively. s,q+1 and s,q+1 They represent the horizontal and vertical coordinates of the q+1th inspection and shooting point respectively, and arctan represents the inverse tangent function.
[0133] Calculate the camera's pitch angle according to the inspection shooting points to ensure that images of key parts are captured. The calculation formula for the camera's pitch angle is:
[0134]
[0135] in, Indicates the pitch angle of the camera (or the PTZ that controls the camera's motion) at the qth inspection point, x qh ,y qh ,z qh Indicates the coordinate position of the qth inspection shooting point, x qt ,y qt ,z qt Indicates the qth inspection target position.
[0136] Example 2
[0137] An electric pole tower drone inspection method provided by an embodiment of the present invention comprises the following steps:
[0138] Step S1: obtaining inspection shooting points using the method for generating inspection shooting points of distribution network drones based on semantic segmentation of tower point clouds as described in Example 1;
[0139] Step S2: Generate an inspection route according to the inspection shooting points in the order of lower left, upper left, upper middle, upper right to lower right;
[0140] Step S3: Control the UAV to perform the inspection task according to the inspection route.
[0141] In order to ensure the safe flight of the drone, the KML file of the inspection route and the original point cloud data are visualized together to check whether the inspection route collides with the tower, environment, etc., and whether the inspection shooting points and inspection route meet the overall safety requirements, such as Fig.11 When the inspection is confirmed to be correct, the inspection route KML file is imported into the drone to perform the inspection task.
[0142] Example 3
[0143] An embodiment of the present invention also provides an electronic device, which includes: a memory, a processor, and a computer program / instruction stored on the memory, and the processor executes the computer program / instruction to implement the distribution network drone inspection shooting point generation method based on pole tower point cloud semantic segmentation or the power pole tower drone inspection method in the embodiment of the present application.
[0144] Although not shown, the electronic device includes a processor, which can perform various appropriate operations and processes according to the program and / or data stored in the read-only memory (ROM) or the program and / or data loaded from the storage portion into the random access memory (RAM). The processor can be a multi-core processor, or it can include multiple processors. In some embodiments, the processor can include a general main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM, various programs and data required for device operation are also stored. The processor, ROM and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.
[0145] The processor and the memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.
[0146] Although not shown, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the distribution network drone inspection shooting point generation method based on tower point cloud semantic segmentation or the power pole tower drone inspection method in the embodiments of the present application.
[0147] Storage media in embodiments of the present invention include permanent and non-permanent, removable and non-removable items that can be implemented by any method or technology to store information. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
[0148] Readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.
[0149] Although not shown, an embodiment of the present invention also provides a computer program product, including: a computer program / instruction, which, when executed by a processor, implements the distribution network drone inspection shooting point generation method based on tower point cloud semantic segmentation or the power pole tower drone inspection method in the embodiment of the present application.
[0150] What is disclosed above is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or modifications within the technical scope disclosed in the present invention, which should be covered within the protection scope of the present invention.
Claims
1. A method for generating shooting points for distribution network drone inspection based on tower point cloud semantic segmentation, characterized in that: The generation method comprises: Obtain original point cloud data containing electric poles and towers; Clustering the original point cloud data to obtain multiple point cloud clusters; Performing type identification on each of the point cloud clusters to obtain a tower point cloud; When the number of vertical poles is greater than 1, the pole tower point cloud is layered according to the number of vertical poles to obtain a vertical pole point cloud; when the number of vertical poles is equal to 1, the pole tower point cloud is used as the vertical pole point cloud; Layering each of the vertical pole point clouds according to the cross arm height to obtain a cross arm point cloud; Calculate the two endpoint positions of the main axis direction of each cross arm point cloud; The inspection shooting points are determined according to the two end point positions of the main axis direction of each cross arm point cloud.
2. The method for generating distribution network drone inspection shooting points based on tower point cloud semantic segmentation according to claim 1 is characterized in that: Calling a trained electric pole tower recognition model to perform type recognition on each point cloud cluster; wherein the electric pole tower recognition model includes a multi-layer perceptron, an encoder layer in a Pointnet classification network, a fully connected layer, and a classification layer connected in sequence; The performing type identification on each of the point cloud clusters specifically includes: Calculating statistical features of the point cloud cluster, and forming an input feature vector from all statistical features of the point cloud cluster; Using a multi-layer perceptron to extract features from the input feature vector to obtain a first output feature quantity; The product of the first output feature quantity and each point in the point cloud cluster is used as the input of the Encoder layer in the Pointnet classification network, and the Encoder layer in the Pointnet classification network is used to perform feature learning and mapping to obtain a second output feature quantity; Using a fully connected layer to extract features from the second output feature quantity to obtain a third output feature quantity; The third output feature quantity is predicted by using the classification layer to obtain the type of the point cloud cluster; wherein the type of the point cloud cluster includes electric wires, poles and towers, and environment; The point cloud clusters of the tower type are merged to obtain the tower point cloud.
3. The method for generating distribution network drone inspection shooting points based on tower point cloud semantic segmentation according to claim 2 is characterized in that: The statistical features of the point cloud cluster include the height, horizontal area, volume, cluster size, color value, and distance between the cluster center point and the tower reference point of the point cloud cluster; wherein the calculation formula of each statistical feature is: h k =z kmax -z kmin ; TO k (x) kmax -x kmin )×(and kmax -and kmin ); vol k =h k ×A k ; Among them, h k Indicates the height of the k-th point cloud cluster, z kmax and z kmin Respectively represent the maximum vertical coordinate value and the minimum vertical coordinate value of the point in the k-th point cloud cluster, A k represents the area of the kth point cloud cluster, x kmax and x kmin Respectively represent the maximum and minimum abscissa values of the points in the k-th point cloud cluster, y kmax and kmin Respectively represent the maximum and minimum ordinate values of the points in the k-th point cloud cluster, vol k Indicates the volume of the kth point cloud cluster, color k Represents the color value of the kth point cloud cluster, N k represents the cluster size of the kth point cloud cluster, r ki ,g ki ,b ki Indicates the color value of the i-th point in the k-th point cloud cluster, dist k represents the distance between the center point of the kth point cloud cluster and the tower reference point, x ki ,y ki ,z ki represents the coordinate value of the i-th point in the k-th point cloud cluster, x r ,y r ,z r Indicates the coordinate value of the tower reference point.
4. The method for generating distribution network drone inspection shooting points based on tower point cloud semantic segmentation according to claim 1 is characterized in that: The pole-level stratification of the pole tower point cloud according to the number of vertical poles includes: Calculating the centroid of the tower point cloud; Constructing a first covariance matrix according to the centroid of the tower point cloud; Performing eigenvalue decomposition on the first covariance matrix, and taking the eigenvector corresponding to the minimum eigenvalue as the normal vector of the main plane; wherein the main plane refers to the plane formed by the vertical rods; Based on the normal vector of the main plane, the tower point cloud is M t Divide equally and get M t Vertical point clouds; among them, M t Indicates the number of vertical poles contained in the tower.
5. The method for generating distribution network drone inspection shooting points based on tower point cloud semantic segmentation according to claim 1 is characterized in that: The calculating of two endpoint positions of the main axis direction of each cross arm point cloud comprises: Calculating the centroid of the crossarm point cloud; constructing a second covariance matrix according to the centroid of the crossarm point cloud; Performing eigenvalue decomposition on the second covariance matrix, and taking the eigenvector corresponding to the maximum eigenvalue as the principal axis direction of the cross-arm point cloud; Two points located in the main axis direction and farthest from the centroid of the cross-arm point cloud are determined, and the two points are used as two end point positions in the main axis direction of the cross-arm point cloud.
6. The method for generating distribution network drone inspection shooting points based on tower point cloud semantic segmentation according to claim 1 is characterized in that: The generation method also includes calculating the yaw angle of the drone and the pitch angle of the camera according to the inspection shooting points. The specific calculation formula is: Among them, θ q represents the yaw angle of the drone at the qth inspection point, x s,q and s,q They represent the horizontal and vertical coordinates of the qth inspection point, respectively. s,q+1 and s,q+1 They represent the horizontal and vertical coordinates of the q+1th inspection and shooting point, respectively. Indicates the pitch angle of the camera at the qth inspection point, x qh ,y qh ,z qh Indicates the coordinate position of the qth inspection shooting point, x qt ,y qt ,z qt represents the qth endpoint position or the qth inspection target position, and arctan represents the inverse tangent function.
7. A method for inspecting electric pole towers by using drones, characterized in that: The inspection method comprises: Obtaining inspection shooting points using the method for generating inspection shooting points of distribution network UAVs based on semantic segmentation of tower point clouds as described in any one of claims 1 to 6; Generate an inspection route according to the inspection shooting points; According to the inspection route, the drone is controlled to perform the inspection task.
8. An electronic device comprising a memory, a processor, and a computer program / instruction stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the distribution network drone inspection shooting point generation method based on tower point cloud semantic segmentation according to any one of claims 1 to 6 or the power pole tower drone inspection method according to claim 7.
9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by the processor, it implements the distribution network drone inspection shooting point generation method based on tower point cloud semantic segmentation as described in any one of claims 1 to 6 or the power pole tower drone inspection method as described in claim 7.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by the processor, it implements the distribution network drone inspection shooting point generation method based on tower point cloud semantic segmentation as described in any one of claims 1 to 6 or the power pole tower drone inspection method as described in claim 7.
Citation Information
Patent Citations
A power line rapid layering extraction method based on LiDAR point cloud
CN109829199A
Automatic track generation method and device of unmanned aerial vehicle for inspection of power transmission line
CN111443730A
Method for calculating lightning protection angle based on high-precision three-dimensional point cloud data
CN113487601A
Fusion method and system of power transmission line image and point cloud data
CN114743021A
Patrol route generation method and device, electronic equipment and storage medium
CN116594428A
Cited By
Unmanned aerial vehicle autonomous electric power facility inspection method based on three-dimensional semantic driving
CN121430653A
