Point cloud classification method, device and equipment of power transmission line and storage medium

Abstract

The embodiment of the invention relates to the technical field of power operation and maintenance, and provides a point cloud classification method, device and equipment for a power transmission line and a storage medium, and the method comprises the steps: obtaining the original point cloud data of the point cloud of the power transmission line, and constructing multi-modal data based on the original point cloud data; obtaining grading parameters for different target scale characteristics, and constructing a multi-stage feature extraction network based on the grading parameters; performing multi-scale feature extraction on the multi-modal data through a multi-stage feature extraction network to obtain a multi-scale feature code; performing adaptive feature fusion on the multi-scale feature codes to obtain multi-scale fusion features; sampling point clouds of the power transmission line based on the multi-scale fusion features to obtain edge points, and enhancing feature expressions of the edge points to obtain target multi-scale features; and classifying the point clouds of the power transmission line based on the target multi-scale features to obtain a point cloud classification result. And point cloud classification of the power transmission line is realized based on scale adaptive domain feature extraction.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of power operation and maintenance, and in particular, to a point cloud classification method for transmission lines, a point cloud classification device for transmission lines, a corresponding electronic device, and a corresponding computer-readable storage medium. Background Art

[0002] Inspection of transmission lines is a core link in the safe operation and maintenance of power systems. LiDAR (such as LiDAR) technology can obtain high-precision three-dimensional point cloud data. However, the geometric scales of components such as conductors, insulators, and towers in the point cloud vary significantly (for example, the conductor diameter is about 2 cm - 5 cm, and the length of the insulator string can reach several meters). Precise classification of these components is the basis for defect detection and condition assessment.

[0003] In the related technologies of point cloud classification for transmission lines, classification can be achieved through hierarchical feature extraction and neighborhood aggregation based on point cloud deep learning frameworks such as PointNet++. However, in the above methods, a fixed radius (such as 0.5 m) is usually used to construct the local neighborhood. However, the optimal feature extraction scales of conductors and insulators are different, and there is a problem of insufficient multi-scale adaptability. Summary of the Invention

[0004] The embodiments of the present application provide a point cloud classification method, device, equipment, and storage medium for transmission lines, which can perform scale-adaptive domain feature extraction based on multi-modal feature fusion and the constructed multi-level feature extraction network, and realize intelligent classification of the laser point cloud of transmission lines.

[0005] In one aspect, the embodiments of the present application provide a point cloud classification method for transmission lines, and the method includes:

[0006] Obtain the original point cloud data of the point cloud of the transmission line, and construct multi-modal data based on the original point cloud data;

[0007] Obtain classification parameters for different target scale characteristics, and construct a multi-level feature extraction network based on the classification parameters;

[0008] Perform multi-scale feature extraction on the multi-modal data through the multi-level feature extraction network to obtain multi-scale feature encodings;

[0009] Perform adaptive feature fusion on the multi-scale feature encodings to obtain multi-scale fusion features;

[0010] Sample the point cloud of the transmission line based on the multi-scale fusion features to obtain edge points, and perform enhancement processing on the feature expressions of the edge points to obtain target multi-scale features;

[0011] Classify the point cloud of the transmission line based on the target multi-scale features to obtain a point cloud classification result.

[0012] In another aspect, an embodiment of the present application provides a point cloud classification device for a transmission line, and the device includes:

[0013] A modal data construction module, configured to obtain the original point cloud data of the point cloud of the transmission line and construct multi-modal data based on the original point cloud data;

[0014] A feature extraction network construction module, configured to obtain hierarchical parameters for different target scale characteristics and construct a multi-level feature extraction network based on the hierarchical parameters;

[0015] A scale feature extraction module, configured to perform multi-scale feature extraction on the multi-modal data through the multi-level feature extraction network to obtain a multi-scale feature encoding;

[0016] A scale feature fusion module, configured to perform adaptive feature fusion on the multi-scale feature encoding to obtain a multi-scale fusion feature;

[0017] An edge feature enhancement module, configured to sample the point cloud of the transmission line based on the multi-scale fusion feature to obtain edge points and perform enhancement processing on the feature expression of the edge points to obtain target multi-scale features;

[0018] A point cloud classification module, configured to classify the point cloud of the transmission line based on the target multi-scale features to obtain a point cloud classification result.

[0019] In yet another aspect, an embodiment of the present application further provides an electronic device, including: a processor, a memory, and a computer program stored on the memory and capable of running on the processor, where when the computer program is executed by the processor, it implements the point cloud classification method for the transmission line according to any one of the above.

[0020] In yet another aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the point cloud classification method for the transmission line according to any one of the above.

[0021] In yet another aspect, an embodiment of the present application further provides a computer program product containing instructions, which when running on a computer, causes the computer to execute the point cloud classification method for the transmission line described in the above aspects.

[0022] The point cloud classification method, device, equipment and storage medium for transmission lines provided by the embodiments of the present application construct multimodal data based on the original point cloud data of the transmission line point cloud, and construct a multi-level feature extraction network based on the hierarchical parameters for different target scale features. The multi-level feature extraction network performs multi-scale feature extraction on the multimodal data to obtain multi-scale feature encodings. Based on the multimodal feature fusion and the constructed multi-level feature extraction network, the scale features of different targets are adapted, realizing the scale-adaptive extraction of domain features, facilitating the subsequent implementation of multi-scale feature fusion. Further, adaptive feature fusion is performed on the multi-scale feature encodings to obtain multi-scale fusion features, and edge points are sampled from the point cloud based on the multi-scale fusion features. The feature expression of the edge points is enhanced to obtain target multi-scale features, and then the transmission line point cloud is classified based on the target degree scale features to obtain the point cloud classification result, realizing the intelligent classification of the laser point cloud of the transmission line while ensuring edge perception and multi-scale adaptability. Description of the Drawings

[0023] Figure 1 is a flowchart of the steps of a point cloud classification method for a transmission line provided by an embodiment of the present application;

[0024] Figure 2 is a flowchart of the steps of another point cloud classification method for a transmission line provided by an embodiment of the present application;

[0025] Figure 3 is a structural block diagram of a point cloud classification device for a transmission line provided by an embodiment of the present application;

[0026] Figure 4 is a structural block diagram of an electronic device provided by an embodiment of the present application;

[0027] Figure 5 is a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. Detailed Embodiments

[0028] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present application.

[0029] In the related technologies of transmission line point cloud classification, multi-level feature extraction can be implemented based on point cloud deep learning frameworks such as PointNet++. However, it usually uses a fixed neighborhood radius (such as 0.5m) and is not optimized for the scale differences between conductors and insulators. Specifically, in the multi-level feature extraction scheme based on PointNet++, the original reflection intensity values are directly used without compensating for material differences (such as different reflection coefficients between metals and non-metals), resulting in misjudgment of metal components, that is, the physical characteristics of reflection intensity are ignored; the fixed neighborhood radius cannot capture the multi-scale features of conductors and insulators at the same time. For example, the conductors are slender and the insulators have periodic texture features, etc., and only single-scale feature extraction can be achieved; moreover, its sampling mechanism is random sampling or FPS (Farthest Point Sampling), resulting in the loss of conductor edge points and affecting the classification integrity, that is, the aforementioned sampling mechanism has defects; in addition, the traditional label propagation algorithm has a slow convergence speed (more than 7 steps), which is difficult to meet the real-time requirements and has low label propagation efficiency.

[0030] The embodiments of this application perform scale-adaptive domain feature extraction based on multi-modal feature fusion and the constructed multi-level feature extraction network, and realize intelligent classification of the laser point cloud of the transmission line while ensuring edge perception and multi-scale adaptability.

[0031] In the embodiments of this application, for the problem of misjudgment of metal components, the influence of material differences can be eliminated through a reflection intensity compensation model; for the problem of single-scale feature extraction, a three-level feature extraction network can be constructed to adapt to the different scale features of conductors, insulators, towers, etc.; for the problem of edge point loss, an edge perception sampling mechanism can be proposed to preferentially retain feature boundary points and realize retaining the slender structural features of the conductor edge; for the problem of low label propagation efficiency, by introducing a momentum acceleration mechanism, the label propagation convergence steps can be reduced from 7 steps to 5 steps to achieve real-time optimization.

[0032] Specifically, referring to Figure 1 , the flowchart of the steps of a point cloud classification method for a transmission line provided by the embodiments of this application is shown, and it can specifically include the following steps:

[0033] Step S101, obtain the original point cloud data of the point cloud of the transmission line, and construct multi-modal data based on the original point cloud data;

[0034] The accuracy of transmission line point cloud classification is beneficial to ensuring the accuracy of transmission line inspection. Based on the collaborative relationship between transmission line point cloud classification and transmission line inspection, a technical closed-loop of intelligent power operation and maintenance is formed.

[0035] Point cloud classification is the core preprocessing step for inspection. The collected original point cloud data can usually be classified, for example, to distinguish conductors, insulators, towers, ground wires, etc., providing target areas for subsequent defect detection during inspection (such as insulator breakage, conductor strand breakage, etc.).

[0036] In some embodiments of the present application, multi-modal data can be constructed to break through the perception limitations of a single modality by fusing complementary data features, comprehensively improving the robustness, accuracy, and scene adaptability of classification and detection.

[0037] Optionally, the original point cloud data can be obtained by lidar or drone. In practical applications, multi-modal data can be constructed based on the acquired original point cloud data.

[0038] Among them, the original point cloud data is high-precision three-dimensional point cloud data collected by lidar or drone for each sampling point of the transmission line. Each sampling point may represent different objects or structural components, specifically depending on the lidar scanning scene and data processing method. For example, the transmission line body structure may include, but is not limited to, conductors, ground wires, insulators, towers, etc. The aforementioned structures can be used as sampling points during lidar scanning. Hereinafter, the "each sampling point" in the point cloud is simply referred to as "each point".

[0039] Exemplarily, assuming the original point cloud data is obtained based on lidar scanning, the original point cloud data usually includes the three-dimensional coordinates and original reflection intensity (Intensity) of each point in the point cloud of the transmission line. Among them, the three-dimensional coordinates refer to the spatial position of each point in the transmission line coordinate system, which can be (x, y, z), and the accuracy is usually less than or equal to 3 cm; the original reflection intensity represents the intensity of the laser echo, which is usually related to factors such as the target surface material, incident angle, distance, etc. Its numerical range is usually 0~255 (8-bit) or 0~65535 (16-bit), and the higher the value, the stronger the reflection.

[0040] The multi-modal data constructed based on the original point cloud data can include the reflection intensity after reflection intensity compensation, normal vector, and curvature. Among them, the reflection intensity after reflection intensity compensation is a corrected intensity value that can eliminate the influence of distance, incident angle, etc., which is beneficial to ensuring material classification, such as accurately distinguishing metal conductors and insulators. That is, the influence of material differences can be eliminated through the reflection intensity compensation model to avoid misjudgment of metal components; the normal vector refers to the orientation of the local surface of the point cloud, which can be mainly applied to surface geometry analysis, such as tower plane detection, conductor direction estimation, etc.; the curvature can be used to indicate the degree of bending of the local surface, which can be mainly applied to defect detection, such as insulator breakage, conductor depression, etc.

[0041] It should be noted that the embodiments of the present application do not limit the specific process of constructing multimodal data based on the original point cloud data.

[0042] Step S102: Obtain hierarchical parameters for different target scale characteristics, and construct a multi-level feature extraction network based on the hierarchical parameters.

[0043] In some embodiments of the present application, in order to achieve multi-scale target classification, different scale targets have different scale characteristics. At this time, in order to adapt to the different scale characteristics of multi-scale targets, a multi-level feature extraction network can be constructed.

[0044] Optionally, a multi-level feature extraction network can be constructed based on the hierarchical parameters of different target scale characteristics, that is, the multi-level feature extraction network can adapt to the scale characteristics of different targets through the hierarchical parameters to achieve multi-scale feature fusion.

[0045] Exemplarily, assume that the multi-scale targets are conductors, insulators, and towers. At this time, in order to adapt to the scale characteristics of the foregoing three scale targets, the obtained hierarchical parameters can be the hierarchical parameters respectively for the scale characteristics of conductors, insulators, and towers. At this time, a three-level feature extraction network that adapts to the scale characteristics of conductors, insulators, and towers can be constructed. It should be noted that the specific number of levels of the constructed multi-level feature extraction network can be determined based on the number of actual multi-scale targets. In this regard, the embodiments of the present application do not limit it.

[0046] Step S103: Perform multi-scale feature extraction on the multimodal data through the multi-level feature extraction network to obtain multi-scale feature encodings.

[0047] After the embodiments of the present application construct the multimodal data and the multi-level feature extraction network, scale-adaptive domain feature extraction can be performed based on multi-modal feature fusion and the constructed multi-level feature extraction network to achieve intelligent classification of the laser point cloud of the transmission line.

[0048] The multimodal data can fuse complementary data features, and the multi-level feature extraction network can adapt to multi-scale target features.

[0049] The scale-adaptive domain feature extraction can be manifested as performing scale-adaptive multi-scale feature extraction on the multimodal data through the constructed multi-level feature extraction network, so as to realize the subsequent classification of the transmission line point cloud based on the extracted multi-scale feature encodings.

[0050] Among them, multi-scale feature encoding can be used to indicate local details and global topological features. The local details can refer to millimeter-level details such as wire surface defects (e.g., broken strand point cloud depressions, etc.) and insulator inter-sheet gaps. The global topological features can refer to meter-level contexts such as the entire line orientation and the spatial relationship between towers, realizing the simultaneous capture of the wire microstructure (e.g., insulator strings, etc.) and the macroscopic layout of the towers. That is, multi-scale feature encoding can be used to indicate the geometric feature encoding of each point in the power transmission line point cloud.

[0051] It should be noted that for the specific extraction process of multi-scale features, the embodiments of this application do not limit this.

[0052] Step S104, perform adaptive feature fusion on the multi-scale feature encoding to obtain multi-scale fusion features;

[0053] There is modal heterogeneity in the geometric feature encoding of each point. To avoid feature conflicts caused by directly splicing three-dimensional coordinates, intensity, curvature, etc., the embodiments of this application can perform adaptive feature fusion on the multi-scale feature encoding to eliminate modal interference through adaptive weighted fusion, retain complementary information, and obtain multi-scale fusion features after completing adaptive feature fusion based on multi-scale feature extraction.

[0054] Optionally, the adaptive feature fusion performed can be achieved through dynamic weight allocation. For example, multi-scale fusion features are generated through cross-scale aggregation and the interaction between geometric features and reflection intensity. During the interaction between geometric features and reflection intensity, the geometric and intensity features can be dynamically weighted through an attention mechanism, that is, the weights of multi-modal features are dynamically balanced through an attention mechanism. The embodiments of this application do not limit this.

[0055] Step S105, sample the point cloud of the power transmission line based on the multi-scale fusion features to obtain edge points, and perform enhancement processing on the feature expression of the edge points to obtain target multi-scale features;

[0056] In the processing of the power transmission line point cloud, edge point perception sampling combined with multi-scale fusion features can effectively capture the boundary details of key structures such as wires, insulators, and towers, and avoid the loss of geometric features caused by uneven sampling.

[0057] In some embodiments of this application, in order to preferentially retain feature boundary points, the point cloud can be sampled to obtain edge points, and enhancement processing is performed on the feature expression of the edge points.

[0058] Exemplarily, the perceptual sampling of edge points can be achieved by means of multi-scale fusion features. Optionally, the feature gradient magnitude of the multi-scale fusion features can be calculated, and the point cloud is sampled based on the feature gradient magnitude to obtain edge points. The enhancement processing of the edge point feature expression can be achieved through pooling operations. Optionally, the feature expression can be enhanced through spatial pyramid pooling. The embodiments of the present application do not limit this.

[0059] Step S106: Classify the point cloud of the transmission line based on the target multi-scale features to obtain a point cloud classification result.

[0060] The feature expression of the edge points in the point cloud has been enhanced by the target multi-scale features. The edge points are obtained by sampling based on the multi-scale fusion features. The multi-scale fusion features are obtained by adaptively fusing features based on multi-scale feature encoding. The multi-scale feature encoding is obtained by extracting multi-scale features from multi-modal data through a multi-level feature extraction network. The multi-level feature extraction network is constructed based on hierarchical parameters for different target scale characteristics. The multi-modal data is constructed based on the original point cloud data of the transmission line point cloud.

[0061] Among them, in the constructed multi-modal data, the influence of material differences can be eliminated through reflection intensity compensation to avoid misjudgment of metal components. The constructed multi-level feature extraction network can adapt to different target scale features such as wires, insulators, and towers, which is beneficial for multi-scale target classification. The adaptive feature fusion performed is beneficial for eliminating modal interference and retaining complementary information. The edge point perceptual sampling and feature expression enhancement processing performed are beneficial for retaining slender structure features.

[0062] In some embodiments of the present application, the classification of the point cloud based on the target multi-scale features obtains a point cloud classification result on the basis of ensuring that metal components are not misjudged, multi-scale adaptive feature extraction, eliminating modal interference, and retaining slender structure features, which can improve the classification accuracy of the transmission line point cloud and thus ensure the accuracy of subsequent transmission line inspections.

[0063] In the embodiments of the present application, multi-modal data is constructed based on the original point cloud data of the transmission line point cloud, and a multi-level feature extraction network is constructed based on hierarchical parameters for different target scale features. The multi-modal data is subjected to multi-scale feature extraction through the multi-level feature extraction network to obtain multi-scale feature encoding. Based on multi-modal feature fusion and the constructed multi-level feature extraction network, the scale features of different targets are adapted to achieve scale-adaptive extraction of domain features, facilitating subsequent multi-scale feature fusion. Further, adaptive feature fusion is performed on the multi-scale feature encoding to obtain multi-scale fusion features. Edge points are sampled from the point cloud based on the multi-scale fusion features, and the feature expression of the edge points is enhanced to obtain target multi-scale features. Then, the transmission line point cloud is classified based on the target degree scale features to obtain a point cloud classification result, realizing intelligent classification of the laser point cloud of the transmission line while ensuring edge perception and multi-scale adaptability.

[0064] Referring to Figure 2 , the flowchart of steps of another point cloud classification method for a transmission line provided by the embodiments of the present application is shown, which may specifically include the following steps:

[0065] Step S201, construct multi-modal data for each point in the point cloud based on the three-dimensional coordinates and reflection intensity of each point in the point cloud;

[0066] In the embodiments of the present application, multi-modal data can be constructed to break through the perception limitations of a single modality by fusing complementary data features, comprehensively improving the robustness, accuracy, and scene adaptability of classification and detection.

[0067] Specifically, the original point cloud data may include the three-dimensional coordinates and reflection intensity of each point in the point cloud for the transmission line. At this time, the multi-modal data for each point can be constructed based on the three-dimensional coordinates and original reflection intensity of each point in the point cloud.

[0068] Optionally, the constructed multi-modal data may include the reflection intensity after compensation for each point. The goal of reflection intensity compensation is to eliminate the influence of lidar distance attenuation and incident angle on the intensity to obtain the true reflectivity. That is, the compensated reflection intensity can mainly be used to compensate for material differences and enhance the physical characteristics of the reflection intensity, which is beneficial to ensuring the accuracy of material classification, such as distinguishing between metal wires and insulators.

[0069] In some embodiments of the present application, the original reflection intensity of each point can be compensated for reflection intensity through a reflection intensity compensation model to obtain the compensated reflection intensity of each point. Specifically, it can be realized through the input and output of a preset reflection intensity compensation model, which is to input the original reflection intensity of each point into the preset reflection intensity compensation model and output the compensated reflection intensity of each point, where the original reflection intensity is the reflection intensity before compensation.

[0070] Exemplarily, in the foregoing preset reflection intensity compensation model, the material reflection coefficient and the Euclidean distance from each point to the lidar sensor can be used to process the original reflection intensity of each point to obtain the compensated reflection intensity of each point. That is, the compensated reflection intensity can be determined based on the original reflection intensity, material reflection coefficient, and Euclidean distance of each point.

[0071] The preset reflection intensity compensation model can be shown by the following formula:

[0072]

[0073] In the formula, I i ’ refers to the compensated reflection intensity of each point; I i refers to the original reflection intensity of each point; refers to the material reflection coefficient, which can be dynamically determined by a pre-trained classifier. For example, the material reflection coefficient of a metal material is taken as 1.2, and the material reflection coefficient of a non-metal material is taken as 0.8; d i refers to the Euclidean distance from the point i in the point cloud to the lidar sensor, and d0 = 100m.

[0074] It should be noted that reflection intensity compensation can improve the classification accuracy of metal components. For example, in the test set , it helps to improve the classification accuracy of subsequent point cloud classification.

[0075] Among them, the preset reflection intensity compensation model can dynamically determine the material reflection coefficient based on a material classifier. For the implementation of the material classifier, its network structure can adopt a framework based on PointNet++, and the output layer can be a 3-layer MLP ( ); the training data can be an annotated data set containing 12 typical materials of transmission lines, such as conductors, insulators, fittings, etc.; its performance index requirements can be manifested as the accuracy of the test set for material allocator testing ≥ 95% ( ), and the inference speed is 1000 points / ms to meet the requirements of real-time inspection.

[0076] Optionally, the constructed multi-modal data can include the normal vectors of each point. The normal vector refers to the orientation of the local surface of the point cloud. As an important representation of geometric features, it can be applied to surface geometric analysis, such as pole plane detection, conductor orientation estimation, etc.

[0077] In some embodiments of the present application, a spherical domain with a preset radius can be constructed for each point, and then for the spherical domain of each point, the covariance matrix of the points in the spherical domain is calculated, and the eigenvector corresponding to the target value in the covariance matrix calculation result is used as the normal vector of the current point.

[0078] Among them, the preset radius value can be a specific value that can effectively balance noise suppression and detail retention, and the selected target value can be the minimum eigenvalue. That is, the eigenvector corresponding to the minimum eigenvalue in the calculation result of the covariance matrix can be used as the normal vector of the current point.

[0079] Exemplarily, the calculation of the normal vector can be implemented by using the PCA (Principal Component Analysis) algorithm. Assume that each point in the transmission line point cloud is , at this time, a spherical domain with a preset radius value can be constructed for each of the foregoing points. For example, a spherical domain with R = 0.5 m. Then, for the spherical domain of the i-th point, the covariance matrix can be calculated using all the points in the spherical domain of the i-th point to determine the normal vector of the i-th point.

[0080] Assume that the points included in the spherical domain of the i-th point are , and the covariance matrix calculation formula can be as follows:

[0081]

[0082]

[0083] In the formula, λ refers to the eigenvalue obtained by orthogonal decomposition; N can refer to the number of neighboring points of the i-th point, that is ; can refer to the centroid of the neighboring points, that is, the mean value; can refer to the vector after centering.

[0084] Among them, the eigen-decomposition of the covariance matrix can obtain three orthogonal eigenvectors. The eigenvector corresponding to the minimum eigenvalue is the direction of the normal vector. For example, there are three orthogonal eigenvectors λ1, λ2, and λ3, and λ1 ≥ λ2 ≥ λ3. At this time, the eigenvector corresponding to λ3 can be used as the normal vector of the i-th point .

[0085] Optionally, the constructed multi-modal data can include the curvature of each point. The curvature can be used to indicate the degree of bending of the local surface and characterize the local geometric complexity, and can be specifically used to distinguish different components. For example, the surface of the wire is smooth and the curvature is low, while the texture of the insulator is complex and the curvature is high.

[0086] In some embodiments of the present application, the curvature of each point can be determined based on the normal vector of each point.

[0087] Exemplarily, the curvature calculation formula can be as follows:

[0088]

[0089] wherein, C i may refer to the curvature value of the i-th point. The greater the curvature value, the more severe the surface undulation; ε refers to the error value, usually ε = 10 -6 ; λ1, λ2, and λ3 refer to the three eigenvalues obtained by orthogonal decomposition of the covariance matrix. The eigenvector corresponding to the minimum eigenvalue can be the normal vector direction , and the normal vector direction usually corresponds to the minimum change direction of the surface, which reflects the minimum change direction of the local surface. At this time, the curvature can be determined based on the aforementioned minimum change direction.

[0090] In the embodiments of the present application, the compensated reflection intensity of each point, the normal vector of each point, and the curvature of each point can be used as the multi-modal data of each point to complete the construction of the multi-modal data, and then fuse the complementary data features to break through the perception limitations of a single modality.

[0091] In practical applications, the reflection intensity is used to compensate for material differences and enhance the physical characteristics of the reflection intensity; the normal vector is an important representation of geometric features, and the curvature is used to distinguish different components, that is, the normal vector and the curvature can assist in extracting the geometric feature encoding of each point at multiple scales. Optionally, the normal vector and the curvature can be fused with the reflection intensity feature in a hierarchical network to assist the network in distinguishing the surface characteristics of different components. In this regard, the embodiments of the present application do not impose any limitations.

[0092] In the embodiments of the present application, multi-modal data, namely the compensated reflection intensity, the normal vector, and the curvature, are obtained through a reflection intensity compensation model, normal vector calculation, and curvature calculation, converting the original point cloud into a feature vector with clear physical meaning and multi-modal fusion, providing an optimized input for the subsequent network; moreover, the reflection intensity compensation can improve the classification accuracy of metal components, that is, the F1-score of the test set ≥ 0.93, thereby improving the classification accuracy.

[0093] Step S202, construct a multi-level feature extraction network using the domain radius of each level and the width of each level;

[0094] To achieve multi-scale target classification, different scale targets have different scale features. At this time, in order to adapt to the different scale features of multi-scale targets, a multi-level feature extraction network can be constructed.

[0095] Specifically, a multi-level feature extraction network can be constructed based on the hierarchical parameters of different target scale characteristics, that is, the multi-level feature extraction network can adapt to the scale characteristics of different targets through hierarchical parameters to achieve multi-scale feature fusion.

[0096] In the process of multi - target scale feature extraction of transmission line point cloud, the selection of neighborhood radius directly affects the feature expression ability, calculation efficiency, and adaptability to different scale structures. For example, when the neighborhood radius is within the first numerical range, it affects the extraction of local microscopic features, such as scratches on the wire surface and gaps between insulator sheets, which are in the millimeter level; when the neighborhood radius is within the second numerical range, it affects the extraction of component - level features, such as the overall attitude of the insulator string and wire joints, which are in the decimeter level; when the neighborhood radius is within the third numerical range, it affects the extraction of macroscopic topological features, such as the overall structure of the tower and the line trend, which are in the meter level. Among them, the first numerical range is a small - radius range, such as 0.1 - 0.3m; the second numerical range is a medium - radius range, such as 0.3 - 0.8m; the third numerical range is a large - radius range, such as greater than 1.0m.

[0097] In the multi - target scale feature extraction of transmission line point cloud, the width of each level directly affects the network's representation ability, calculation efficiency, and sensitivity to different scale targets.

[0098] Optionally, the feature extraction grading parameters can include the neighborhood radius and width of each level for each target scale characteristic. Among them, the width of each level can be determined based on the number of feature channels. The number of feature channels can be the number of channels of an MLP (Multilayer Perceptron), which usually refers to the number of neurons in each layer, that is, the width of each level is the number of neurons in each layer.

[0099] In some embodiments of the present application, a multi - level feature extraction network can be constructed using the neighborhood radius and width of each level.

[0100] Optionally, assuming that the multi - scale targets are wires, insulators, and towers, in order to adapt to the scale characteristics of the above - mentioned three scale targets at this time, the constructed multi - level feature extraction network is a three - level feature extraction network. The obtained grading parameters can specifically include the first grading parameter for the local detail features of the wire, the second grading parameter for the medium - scale structure of the insulator, and the third grading parameter for the scale contour of the tower. At this time, the three - level feature extraction network can be constructed using the first grading parameter, the second grading parameter, and the third grading parameter. The constructed three - level feature extraction network is used to extract the features of wires, insulators, and towers.

[0101] Among them, the first grading parameter can include the first neighborhood radius and the first number of feature channels, the second grading parameter can include the second neighborhood radius and the second number of feature channels, the third grading parameter can include the third neighborhood radius and the third number of feature channels. The value of the first neighborhood radius is within the first numerical range, the value of the second neighborhood radius is within the second numerical range, and the value of the third neighborhood radius is within the third numerical range.

[0102] Exemplarily, in the constructed three-level feature extraction network, it can adapt to the scale features of different targets through hierarchical parameters to achieve multi-scale feature fusion. The correspondence between the configuration of the hierarchical parameters and the targets can be shown in Table 1 below:

[0103] Table 1 Correspondence between hierarchical parameters and targets

[0104]

[0105] As shown in Table 1 above, the L1 layer of the three-level feature extraction network can be used to extract the features of the wire. Its main design purpose is to capture the local detailed features of the wire. The value of the first neighborhood radius adapted to the wire scale characteristics can be 0.3m, and its MLP channel number can be ; the L2 layer can be used to extract the features of the insulator. Its main design purpose is to extract the scale structure in the insulator. The value of the second neighborhood radius adapted to the insulator can be 0.3m, and its MLP channel number can be ; the L3 layer can be used to extract the features of the tower. Its main design purpose is to perceive the large-scale contour of the tower. The value of the third neighborhood radius adapted to the tower can be 1.2m, and its MLP channel number can be .

[0106] In the embodiment of the present application, the three-level feature extraction network captures details to global features through L1 (0.3m neighborhood), L2 (0.6m), and L3 (1.2m) layers, which can reduce the classification error of the slender features of the wire, the medium-scale features of the insulator, and the large-scale features of the tower by 12.7% and achieve multi-scale adaptability. It should be noted that the specific construction method of the multi-layer feature extraction network is not limited in the embodiment of the present application.

[0107] Step S203, perform multi-scale feature extraction on the multi-modal data through the multi-level feature extraction network to obtain multi-scale feature encoding;

[0108] In the embodiment of the present application, scale-adaptive domain feature extraction can be performed based on multi-modal feature fusion and the constructed multi-level feature extraction network to achieve intelligent classification of the laser point cloud of the transmission line.

[0109] The multi-modal data can fuse complementary data features, and the multi-level feature extraction network can adapt to multi-scale target features.

[0110] The scale-adaptive domain feature extraction can be manifested as performing scale-adaptive multi-scale feature extraction on the multi-modal data through the constructed multi-level feature extraction network, so as to realize the subsequent classification of the transmission line point cloud based on the extracted multi-scale feature encoding.

[0111] Specifically, for the three-level feature extraction network constructed above, the first hierarchical parameter can be used to extract the local detailed features of the conductor, and / or the second hierarchical parameter can be used to extract the mesoscale structure of the insulator, and / or the third hierarchical parameter can be used to extract the large-scale contour of the tower to obtain multi-scale features. The obtained multi-scale features can be used to indicate features at different levels, and then a channel splicing operation can be performed on the multi-scale features to obtain a multi-scale feature encoding.

[0112] Among them, the multi-scale feature encoding can be used to indicate local details and global topological features. The local details can refer to millimeter-level details such as conductor surface defects (such as broken strand point cloud depressions, etc.) and insulator inter-sheet gaps. The global topological features can refer to meter-level contexts such as the entire line trend and the spatial relationship between towers, realizing the simultaneous capture of the micro-structure of the conductor (such as insulator strings, etc.) and the macro-layout of the tower. That is, the multi-scale feature encoding can be used to indicate the geometric feature encoding of each point in the point cloud of the transmission line.

[0113] Exemplarily, the feature encoding formula can be as follows:

[0114]

[0115] In the formula, i represents the index of the current center point, that is, the target point for which features are to be extracted, and also each point in the point cloud; k represents the index of other points in the neighborhood of the center point i, that is, the points adjacent to the center point i in the spherical neighborhood; represents the relative position vector between the center point i and the adjacent point k; is the Euclidean distance feature; represents the channel splicing operation, which is used to realize multi-scale feature fusion.

[0116] Step S204, perform adaptive feature fusion on the multi-scale feature encoding to obtain a multi-scale fusion feature;

[0117] In the embodiments of the present application, adaptive feature fusion can be performed on the multi-scale feature encoding to adaptively weighted fuse to eliminate modal interference and retain complementary information, obtaining a multi-scale fusion feature after completing adaptive feature fusion on the basis of multi-scale feature extraction.

[0118] In some embodiments of the present application, the adaptive feature fusion performed can be achieved through dynamic weight allocation. For example, multi-scale fusion features can be generated through cross-scale aggregation and the interaction between geometric features and reflection intensity. Among them, during the interaction between geometric features and reflection intensity, the geometric and intensity features can be dynamically weighted through an attention mechanism, that is, the weights of multi-modal features can be dynamically balanced through the attention mechanism. The embodiments of the present application do not limit this.

[0119] Optionally, multi-scale feature encoding can be used to indicate the geometric feature encoding of each point at each level in the point cloud. Cross-scale feature aggregation can specifically be manifested as sampling and weighted summation of the geometric feature encodings of each point at different levels to obtain multi-scale fusion information; the interaction between geometric features and reflection intensity can specifically be manifested as using an attention mechanism to assign weights to geometric features and reflection intensity features for the multi-scale fusion information to obtain multi-scale fusion features.

[0120] Exemplarily, cross-scale feature aggregation is to perform upsampling and weighted summation on features at different levels, such as levels L1 - L3 in Table 1 above, to obtain multi-scale fusion information; the interaction between geometric features and reflection intensity can be to dynamically assign weights to geometric and reflection features through an attention mechanism on the fused features to enhance the representation ability of key modalities.

[0121] The formula for cross-scale feature aggregation can be shown as follows:

[0122]

[0123]

[0124] In the formula, F agg can refer to the result of weighted summation; UpSample can represent performing upsampling, and F S can refer to the level, where s = 1, 2, 3; a s refers to the weight coefficient; W s refers to the weight matrix.

[0125] On the fused features, the weights of multi-modal features can be dynamically balanced through an attention mechanism, and the formula of its attention mechanism can be shown as follows:

[0126]

[0127] In the formula, A g and A r can be the attention weights of the set and reflection channels respectively; F geo refers to the geometric feature; F flect can refer to the reflection intensity feature; G can refer to the fused feature; W g can refer to the linear transformation matrix; W geo can refer to the weight matrix.

[0128] In the embodiments of the present application, on the basis of performing upsampling and weighted summation to achieve weighted fusion of features at different scales, the weights of geometric and reflection features can be dynamically assigned through an attention mechanism to enhance the representation ability of multi-scale targets.

[0129] Step S205: Calculate the feature gradient magnitude of the multi-scale fusion features, and sample the point cloud based on the feature gradient magnitude to obtain edge points;

[0130] In the processing of transmission line point clouds, edge point perception sampling combined with multi-scale fusion features can effectively capture the boundary details of key structures such as wires, insulators, and towers, and avoid the loss of geometric features caused by uneven sampling.

[0131] In some embodiments of the present application, in order to preferentially retain feature boundary points, the edge points in the point cloud can be sampled, and the feature expression of the edge points can be enhanced.

[0132] Optionally, it can be implemented by processing with an improved lightweight network. Specifically, edge-aware sampling can be performed based on the improved lightweight network to retain key points, and the retained key points are edge points.

[0133] Specifically, edge-aware sampling can be manifested as sampling the point cloud to obtain edge points, which can be specifically manifested as calculating the feature gradient magnitude of the multi-scale fusion features, and then sampling the point cloud based on the feature gradient magnitude to obtain edge points.

[0134] Among them, the feature gradient magnitude is used to indicate the degree of change in the feature space, specifically reflecting the severity of the change in the feature space.

[0135] Edge points refer to the points in the feature boundary region. In some embodiments of the present application, sampling the edge points of the point cloud based on the feature gradient magnitude can be manifested as sampling the target points in the point cloud whose feature gradient magnitude is greater than a preset threshold to obtain edge points.

[0136] Exemplarily, the feature gradient amplitude calculation formula can be as follows:

[0137]

[0138] In the formula, F i can refer to the feature gradient values of each point in the point cloud; L can refer to the loss function used to calculate the gradient; F ic can refer to the value of the feature vector of the point i in the point cloud in the c-th channel, where c is one of the total number of channels D; D can refer to the total number of channels of the feature (i.e., the feature dimension).

[0139] The edge point sampling implemented based on the feature gradient magnitude can be realized based on the dynamic sampling probability formula, and this dynamic sampling probability formula is the dynamic sampling probability formula based on the feature gradient magnitude, which can be specifically as follows:

[0140]

[0141] In the formula, pi can refer to the sampling probability of each point in the point cloud; F j can refer to the feature gradient value of the points adjacent to the central point i in the spherical neighborhood; D knn can refer to the average distance of the k-nearest neighbors.

[0142] In the above dynamic sampling probability formula, the numerator term can be used to preferentially retain the points in the feature boundary region, specifically, the points with a gradient magnitude greater than a preset threshold can be preferentially retained; the denominator term can be used to suppress the over-sampling in the dense region.

[0143] Optionally, in the above calculation process of the feature gradient magnitude, the gradient calculation can adopt the high-order automatic differentiation technique.

[0144] Exemplarily, first, a forward computational graph can be constructed to retain the second-order derivative information, such as the Hessian matrix; then, the gradient magnitude correction formula can be adopted to implement the correction of the gradient magnitude, and its specific formula can be as follows:

[0145]

[0146] In the formula, is the feature second-order derivative matrix, which can be used as a correction term, and this correction term can be used to smooth the gradient mutation caused by noise.

[0147] In the embodiments of the present application, after adding Gaussian noise (σ = 0.1m), the gradient correction formula can reduce the edge detection error by 18.3%, realizing the anti-noise performance test, that is, the gradient mutation caused by noise can be suppressed based on the high-order automatic differentiation technique, such as the Hessian matrix correction method, so as to enhance the anti-noise ability.

[0148] Step S206, perform enhancement processing on the feature expression of the edge points to obtain the target multi-scale features;

[0149] In some embodiments of the present application, the feature expression of the edge points can be enhanced based on spatial pyramid pooling.

[0150] Optionally, an edge-aware grid can be generated based on the edge points, and then pooling operations can be performed on each voxel in the edge-aware grid through a spatial pyramid to enhance the feature dimension of the feature expression of the edge points and obtain the target multi-scale features.

[0151] Exemplarily, 1m can be used 3The voxel is divided into 2×2×2 = 8 sub-regions (the side length of the sub-cube is 0.5 m), and then a dual-channel pooling operation can be performed. For example, based on the max-pooling operation to retain significant features and the average-pooling operation to maintain distribution features, so as to output the feature dimension, such as the feature expression of edge points with 8×(64 + 64) = 1024, realizing the multi-dimensional feature expression of edge points, and further achieving the purpose of enhancing the feature dimension of the feature expression of edge points.

[0152] Step S207: Classify the point cloud based on the target multi-scale features to obtain the point cloud classification result.

[0153] In the embodiment of the present application, the classification of the point cloud based on the target multi-scale features is to obtain the point cloud classification result on the basis of ensuring that the metal components are not misjudged, multi-scale adaptive feature extraction, eliminating modal interference, and retaining the slender structure features, which can improve the classification accuracy of the transmission line point cloud, and further ensure the accuracy of subsequent transmission line inspections.

[0154] Optionally, the classification result can be optimized through a k-NN graph (k-Nearest Neighbors Graph, a graph structure representing the topological relationship between data points) and momentum acceleration, and the point cloud classification result is output.

[0155] Among them, the output point cloud classification result can specifically be manifested as outputting the final class label.

[0156] In some embodiments of the present application, a momentum acceleration mechanism can be introduced to reduce the number of label propagation convergence steps from 7 steps to 5 steps, so that the inference speed of the edge computing unit is 1000 points / ms, meeting the inspection requirement of 50 m / s, and realizing the real-time optimization of dynamic label propagation.

[0157] Specifically, based on the target multi-scale features, a graph structure for indicating the topological relationship between each point in the point cloud can be constructed, and its topological relationship can be determined based on the edge weights of the edges formed between each point. Then, a preset momentum update value can be used to perform dynamic label optimization operations on the edge weights to obtain the point cloud classification result.

[0158] Exemplarily, assuming that the constructed k-NN graph is G=(V,E,W) (k = 15), the calculation formula of its edge weights can be as follows:

[0159]

[0160] In the formula, k refers to the index of other points in the neighborhood of the center point i, that is, the points adjacent to the center point i in the spherical neighborhood; F i can refer to the feature gradient values of each point in the point cloud, F jIt can refer to the characteristic gradient value of the points adjacent to the central point i in the spherical neighborhood; x i It can refer to the abscissa coordinates of each point in the point cloud, x j It can refer to the abscissa coordinates of the adjacent points; w ij It can refer to the edge weights between each point i in the point cloud and its adjacent points; exp can refer to the natural exponential function; (.) is an indicator function to ensure that only the k-nearest neighbor nodes are connected; 0.5 and 0.3 are the standard deviation parameters of the feature space and the geometric space respectively.

[0161] The dynamic label optimization operation can be manifested as using the similarity between nodes in the k-NN graph to propagate the known labels (such as labeled points) to the unknown nodes in an iterative manner, and finally realizing the globally consistent label assignment. Its iterative optimization process can be manifested as:

[0162]

[0163] In the formula, Y can refer to the convergence value; is the degree matrix, used to achieve weight normalization; t is the number of iterations. When t = 5, the convergence threshold 。

[0164] Among them, the momentum-accelerated label propagation can be realized by combining the iterative optimization formula with the momentum term to accelerate convergence.

[0165] In the above iterative optimization process, a momentum acceleration mechanism can be introduced, which can be specifically realized through the momentum update formula. The specific formula can be as follows:

[0166]

[0167] In the formula, P can refer to the preset momentum update value. The specific value of P can be set based on actual needs, and the embodiments of the present application do not limit this.

[0168] It has been verified that the momentum acceleration mechanism provided by the embodiments of the present application can reduce the convergence steps of label propagation from 7 steps to 5 steps, and reduce the classification error rate by 12.7%, thereby improving the convergence efficiency.

[0169] In some embodiments of the present application, the embodiments of the present application can also provide a power transmission line intelligent inspection system, and the aforementioned power transmission line intelligent inspection system can implement the point cloud classification method of the power transmission line provided by the embodiments of the present application.

[0170] Exemplarily, the hardware configuration of the power transmission line intelligent inspection system can include a lidar and an edge computing unit. For the technical parameters of the lidar, its wavelength can be 1550nm to effectively penetrate vegetation; its point frequency can 800 kHz to meet the inspection speed of 50 m / s; its ranging accuracy can be For the configuration of the edge computing unit, its CUDA (Compute Unified Device Architecture Core) cores can be configured to 512, its memory can be configured to 32 GB LPDDR4x, and its computing power can be configured to 30 TOPS (INT8).

[0171] The software architecture of the intelligent transmission line inspection system can include a real-time preprocessing module and an online learning module. The real-time preprocessing module can be mainly used for point cloud denoising, and can retain 99.7% of the effective points based on statistical filtering to ensure the registration accuracy 3 cm to meet the DL / T 1242-2022 standard; the online learning module can support incremental update of the material classifier, such as daily update every 5 minutes, and the online learning module can implement model version management, such as retaining the latest 10 versions.

[0172] It should be noted that the above configuration of the intelligent transmission line inspection system is one of the configurations of the embodiments of the present application, and it can also be other configurations that can implement the point cloud classification method of the transmission line provided by the embodiments of the present application; moreover, the specific process of the intelligent transmission line inspection system implementing the point cloud classification method of the transmission line is not limited in the embodiments of the present application.

[0173] In the embodiments of the present application, domain feature extraction with scale adaptability is performed based on multi-modal feature fusion and the constructed multi-level feature extraction network. While ensuring edge perception and multi-scale adaptability, intelligent classification of the laser point cloud of the transmission line is achieved. Among them, for the problem of misjudgment of metal components, the influence of material differences can be eliminated through the reflection intensity compensation model; for the problem of single-scale feature extraction, a three-level feature extraction network can be constructed to adapt to different scale features of conductors, insulators, towers, etc.; for the problem of edge point loss, an edge perception sampling mechanism can be proposed to preferentially retain feature boundary points and achieve the retention of the slender structural features of the conductor edge; for the problem of low label propagation efficiency, a momentum acceleration mechanism can be introduced to reduce the label propagation convergence steps from 7 steps to 5 steps to achieve real-time optimization.

[0174] It should be noted that, for the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present application are not limited by the described action sequence, because according to the embodiments of the present application, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present application.

[0175] Referring to Figure 3 , a structural block diagram of a point cloud classification device for a transmission line provided by an embodiment of the present application is shown, which may specifically include the following modules:

[0176] The modality data construction module 301 is configured to obtain the original point cloud data of the point cloud of the transmission line and construct multi-modal data based on the original point cloud data;

[0177] The feature extraction network construction module 302 is configured to obtain hierarchical parameters for different target scale characteristics and construct a multi-level feature extraction network based on the hierarchical parameters;

[0178] The scale feature extraction module 303 is configured to perform multi-scale feature extraction on the multi-modal data through the multi-level feature extraction network to obtain multi-scale feature encodings;

[0179] The scale feature fusion module 304 is configured to perform adaptive feature fusion on the multi-scale feature encodings to obtain multi-scale fusion features;

[0180] The edge feature enhancement module 305 is configured to sample the point cloud of the transmission line based on the multi-scale fusion features to obtain edge points, and perform enhancement processing on the feature expressions of the edge points to obtain target multi-scale features;

[0181] The point cloud classification module 306 is configured to classify the point cloud of the transmission line based on the target multi-scale features to obtain a point cloud classification result.

[0182] In some embodiments of the present application, the original point cloud data includes the three-dimensional coordinates and original reflection intensities of each point; the modality data construction module 301 may include the following sub-modules:

[0183] The multi-modal data construction sub-module is configured to input the original reflection intensities of each point into a preset reflection intensity compensation model, and output the compensated reflection intensities of each point; construct a spherical domain with a preset radius for each point, and for the spherical domain of each point, calculate the covariance matrix of the points in the spherical domain, and use the eigenvector corresponding to the target value in the covariance matrix calculation result as the normal vector of the current point; determine the curvature of each point based on the direction of the normal vector of each point; use the compensated reflection intensities of each point, the normal vectors of each point, and the curvatures of each point as multi-modal data.

[0184] In some embodiments of the present application, the hierarchical parameters include the domain radius and the width of each level for each target scale characteristic; the feature extraction network construction module 302 may include the following sub-modules:

[0185] A multi-level feature extraction network construction sub-module, configured to construct a multi-level feature extraction network by using the domain radius and the width of each level.

[0186] In some embodiments of the present application, the multi-level feature extraction network is a three-level feature extraction network, and the hierarchical parameters include a first hierarchical parameter for the local detail features of the wire, a second hierarchical parameter for the medium-scale structure of the insulator, and a third hierarchical parameter for the large-scale contour of the tower;

[0187] The scale feature extraction module 303 may include the following sub-modules:

[0188] A multi-scale feature extraction sub-module, configured to extract the local detail features of the wire by using the first hierarchical parameter, and / or extract the medium-scale structure of the insulator by using the second hierarchical parameter, and / or extract the large-scale contour of the tower by using the third hierarchical parameter to obtain multi-scale features; based on performing a channel splicing operation on the multi-scale features, a multi-scale feature encoding is obtained.

[0189] In some embodiments of the present application, the multi-scale feature encoding is used to indicate the geometric feature encoding of each point at each level in the point cloud; the scale feature fusion module 304 may include the following sub-modules:

[0190] A multi-scale feature fusion sub-module, configured to sample and perform weighted summation on the geometric feature encodings of each point at different levels to obtain multi-scale fusion information; and assign weights to the geometric feature and the reflection intensity feature to the multi-scale fusion information through an attention mechanism to obtain multi-scale fusion features.

[0191] In some embodiments of the present application, the edge feature enhancement module 305 may include the following sub-modules:

[0192] An edge feature enhancement sub-module, configured to calculate the feature gradient amplitude of the multi-scale fusion features to obtain the feature gradient amplitude; sample the target points in the point cloud of the transmission line whose feature gradient amplitude is greater than a preset threshold to obtain edge points; generate an edge perception grid based on the edge points, and perform a pooling operation on each voxel in the edge perception grid through a spatial pyramid to enhance the feature dimension of the feature expression of the edge points to obtain target multi-scale features.

[0193] In some embodiments of the present application, the point cloud classification module 306 may include the following sub-modules:

[0194] A point cloud classification sub-module is used to construct a graph structure for indicating the topological relationship between each point in the point cloud based on the target multi-scale features; the topological relationship is determined based on the edge weights of the edges formed between each point; a preset momentum update value is used to perform a dynamic label optimization operation on the edge weights to obtain the point cloud classification result.

[0195] In the embodiments of the present application, by constructing multi-modal data based on the original point cloud data of the transmission line point cloud, and constructing a multi-level feature extraction network based on the hierarchical parameters for different target scale features, the multi-modal data is subjected to multi-scale feature extraction through the multi-level feature extraction network to obtain multi-scale feature encoding. Based on the multi-modal feature fusion and the constructed multi-level feature extraction network, the scale features of different targets are adapted to achieve the scale adaptive extraction of domain features, facilitating the subsequent multi-scale feature fusion; further, the multi-scale feature encoding is subjected to adaptive feature fusion to obtain multi-scale fusion features, and edge points are sampled from the point cloud based on the multi-scale fusion features. The feature expression of the edge points is enhanced to obtain the target multi-scale features, and then the transmission line point cloud is classified based on the target degree scale features to obtain the point cloud classification result, realizing the intelligent classification of the laser point cloud of the transmission line while ensuring edge perception and multi-scale adaptability.

[0196] For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiments.

[0197] The embodiments of the present application also provide an electronic device. Refer to Figure 4 The provided electronic device 400 includes a memory 410, a processor 420, and a computer program 411 stored on the memory 410 and capable of running on the processor 420. When the computer program 411 is executed by the processor, it realizes each process of the above-mentioned method embodiment for classifying the point cloud of the transmission line and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0198] The embodiments of the present application also provide a computer-readable storage medium. Refer to Figure 5 The computer program 411 is stored on the provided computer-readable storage medium 500. When the computer program 411 is executed by the processor, it realizes each process of the above-mentioned method embodiment for classifying the point cloud of the transmission line and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0199] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts between each embodiment can be referred to each other.

[0200] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of the embodiments of the present application are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described here can be implemented in an order different from that shown or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or modules does not necessarily have to be limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products or devices. The division of modules in the embodiments of the present application is only a logical division, and there may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the shown or discussed couplings or direct couplings or communication connections between each other can be through some interfaces, and the indirect couplings or communication connections between modules can be electrical or other similar forms, which are not limited in the embodiments of the present application. And the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed to multiple circuit modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present application.

[0201] In the above embodiments, the descriptions of each embodiment have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0202] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices and modules can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated here.

[0203] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the shown or discussed couplings or direct couplings or communication connections between each other can be through some interfaces, and the indirect couplings or communication connections between devices or modules can be electrical, mechanical or other forms.

[0204] The module described as a separation component may or may not be physically separated. The component shown as a module may or may not be a physical module, that is, it may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0205] In addition, in each embodiment of this application, each functional module can be integrated in a processing module, or each module can exist physically alone, or two or more modules can be integrated in one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0206] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0207] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a Solid State Disk (SSD)).

[0208] Embodiments of the present application are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0209] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks; these computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0210] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they know the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0211] Finally, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0212] The above has introduced in detail the technical solutions provided by the embodiments of the present application. Specific examples are used in the embodiments of the present application to elaborate on the principles and implementation manners of the embodiments of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the embodiments of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the embodiments of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the embodiments of the present application.

Claims

1. A point cloud classification method for a transmission line, characterized in that, The method includes: Obtain the original point cloud data of the point cloud of the transmission line, and construct multimodal data based on the original point cloud data; Obtain hierarchical parameters for different target scale characteristics, and construct a multi-level feature extraction network based on the hierarchical parameters; Perform multi-scale feature extraction on the multimodal data through the multi-level feature extraction network to obtain multi-scale feature encoding; Perform adaptive feature fusion on the multi-scale feature encoding to obtain multi-scale fusion features; Sample the point cloud of the transmission line based on the multi-scale fusion features to obtain edge points, and enhance the feature expression of the edge points to obtain target multi-scale features; Classify the point cloud of the transmission line based on the target multi-scale features to obtain a point cloud classification result.

2. The method according to claim 1, characterized in that, The original point cloud data includes the three-dimensional coordinates and original reflection intensity of each point; constructing multimodal data based on the original point cloud data includes: Input the original reflection intensity of each point into a preset reflection intensity compensation model, and output the compensated reflection intensity of each point; Construct a spherical domain with a preset radius for each point. For the spherical domain of each point, calculate the covariance matrix of the points in the spherical domain, and use the eigenvector corresponding to the target value in the covariance matrix calculation result as the normal vector of the current point; Determine the curvature of each point based on the direction of the normal vector of each point; Use the compensated reflection intensity of each point, the normal vector of each point, and the curvature of each point as multimodal data.

3. The method according to claim 1, wherein The hierarchical parameters include the domain radius and width of each level for each target scale characteristic; constructing a multi-level feature extraction network based on the hierarchical parameters includes: Use the domain radius and width of each level to construct a multi-level feature extraction network.

4. The method according to claim 1 or 3, characterized in that, The multi-level feature extraction network is a three-level feature extraction network. The hierarchical parameters include the first hierarchical parameter for the local detail features of the conductor, the second hierarchical parameter for the mesoscale structure of the insulator, and the third hierarchical parameter for the scale contour of the tower; The process of performing multi-scale feature extraction on the multimodal data through the multi-level feature extraction network to obtain multi-scale feature encoding includes: Extract the local detail features of the conductor using the first hierarchical parameter, and / or extract the mesoscale structure of the insulator using the second hierarchical parameter, and / or extract the large-scale contour of the tower using the third hierarchical parameter to obtain multi-scale features; Based on performing a channel splicing operation on the multi-scale features, obtain multi-scale feature encoding.

5. The method according to claim 1, characterized in that, The multi-scale feature encoding is used to indicate the geometric feature encoding of each point at each level in the point cloud; the process of performing adaptive feature fusion on the multi-scale feature encoding to obtain multi-scale fusion features includes: Sample and perform weighted summation on the geometric feature encodings of each point at different levels to obtain multi-scale fusion information; Allocate weights to the geometric features and reflection intensity features of the multi-scale fusion information through an attention mechanism to obtain multi-scale fusion features.

6. The method according to claim 1, characterized in that, Sampling the point cloud of the transmission line based on the multi-scale fusion feature to obtain edge points, and enhancing the feature representation of the edge points to obtain target multi-scale features, including: Calculating the feature gradient magnitude of the multi-scale fusion feature to obtain the feature gradient magnitude; Sampling the target points in the point cloud whose feature gradient magnitude is greater than a preset threshold to obtain edge points; Generating an edge-aware grid based on the edge points, and performing pooling operations on each voxel in the edge-aware grid through a spatial pyramid to enhance the feature dimension of the feature representation of the edge points, thereby obtaining target multi-scale features.

7. The method according to claim 1, wherein Classifying the point cloud of the transmission line based on the target multi-scale features to obtain a point cloud classification result, including: Based on the target multi-scale features, constructing a graph structure for indicating the topological relationship between each point in the point cloud; the topological relationship is determined based on the edge weights of the edges formed between each point; Performing a dynamic label optimization operation on the edge weights using a preset momentum update value to obtain the point cloud classification result.

8. A point cloud classification device for a transmission line, characterized in that, The device includes: A modal data construction module, configured to obtain the original point cloud data of the point cloud of the transmission line, and construct multi-modal data based on the original point cloud data; A feature extraction network construction module, configured to obtain hierarchical parameters for different target scale characteristics, and construct a multi-level feature extraction network based on the hierarchical parameters; A scale feature extraction module, configured to perform multi-scale feature extraction on the multi-modal data through the multi-level feature extraction network to obtain multi-scale feature encodings; A scale feature fusion module, configured to perform adaptive feature fusion on the multi-scale feature encodings to obtain multi-scale fusion features; An edge feature enhancement module, configured to sample the point cloud of the transmission line based on the multi-scale fusion feature to obtain edge points, and enhance the feature representation of the edge points to obtain target multi-scale features; A point cloud classification module, configured to classify the point cloud of the transmission line based on the target multi-scale features to obtain a point cloud classification result.

9. An electronic device, characterized in that, Including: A processor, a memory, and a computer program stored on the memory and capable of running on the processor, where when the computer program is executed by the processor, it implements the point cloud classification method of the transmission line according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the point cloud classification method of the transmission line according to any one of claims 1 to 7.

Images