Pole part point positioning method and device based on point cloud hierarchical registration

Abstract

The invention provides a pole part point positioning method and device based on point cloud hierarchical registration. The method comprises the following steps: firstly, determining target point cloud data and reference point cloud data of multiple resolution levels; then determining transformation parameters for aligning the target point cloud data to the reference point cloud data; generating a transformation function based on the transformation parameters, and converting each point in the target point cloud into a coordinate system corresponding to the reference point cloud by using the transformation function; and finally, based on the coordinates of the key part points in the reference point cloud, performing neighborhood search in the target point cloud in the coordinate system corresponding to the reference point cloud, and taking adjacent points closest to the key part points as key part points in the target point cloud. And taking the coordinate of the key part point in the target point cloud in the coordinate system corresponding to the target point cloud as the final positioning coordinate of the key part point in the target point cloud.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of electric power, and in particular to a tower pole component point positioning method and device based on point cloud hierarchical registration. BACKGROUND

[0002] In the field of key component point extraction of electric power towers, existing methods mainly rely on traditional image processing techniques and rule-based methods. Traditional image processing techniques usually rely on photos or videos taken by high-resolution cameras for analysis, which mainly use edge detection, feature matching, and other algorithms to identify and locate target objects; the disadvantage of this method is that it is sensitive to lighting conditions, difficult to adapt to complex environmental changes, and has poor recognition effect in occlusion situations. The point cloud registration method based on ICP is one of the most widely used point cloud registration methods, which finds the correspondence between the source point cloud and the target point cloud, and then calculates the optimal transformation matrix to make the two point clouds coincide as much as possible. Although ICP and its improved versions (such as Generalized-ICP, G-ICP) perform well in many application scenarios, their performance will decrease significantly in the presence of large non-rigid deformation or local detail changes. The template matching-based method uses pre-defined rule templates to match specific types of structural features in some cases. This method relies on the quality and diversity of the template library, and may not be able to correctly match if it encounters unseen structures or deformations. SUMMARY

[0003] The present disclosure at least provides a tower pole component point positioning method and device based on point cloud hierarchical registration to solve the problems of low efficiency, poor precision, and poor applicability in positioning key component points of towers and poles in the prior art.

[0004] According to an aspect of the present disclosure, a tower pole component point positioning method based on point cloud hierarchical registration is provided, comprising:

[0005] obtaining a reference point cloud pre-labeled with key component points and a target point cloud to be positioned with key component points;

[0006] performing multi-level down-sampling processing on the target point cloud to obtain target point cloud data of a plurality of preset resolution levels; performing multi-level down-sampling processing on the reference point cloud to obtain reference point cloud data of a plurality of preset resolution levels;

[0007] for the smallest resolution level, registering the target point cloud data and the reference point cloud data of this resolution level to obtain transformation parameters for aligning the target point cloud data to the reference point cloud data;

[0008] In ascending order, for each resolution level except the smallest resolution level, performing the following operations: performing feature alignment processing on the target point cloud data of the resolution level using the transformation parameters corresponding to the previous resolution level of the resolution level, and inputting the target point cloud data after the feature alignment processing into the trained deep learning model to make the deep learning model output the transformation parameters corresponding to the resolution level for aligning the target point cloud data to the reference point cloud data; wherein the loss function of the deep learning model includes a smoothness regularization term, an isometry regularization term, and a rigidity prior term;

[0009] Generating a transformation function based on the transformation parameters;

[0010] Converting each point in the target point cloud into the coordinate system corresponding to the reference point cloud using the transformation function;

[0011] Based on the coordinates of the key component points in the reference point cloud, performing neighborhood search in the target point cloud located in the coordinate system corresponding to the reference point cloud, and taking the nearest neighbor points of the key component points as the key position points in the target point cloud; and taking the coordinates of the key position points in the target point cloud in the coordinate system corresponding to the target point cloud as the final positioning coordinates of the key position points in the target point cloud.

[0012] In a possible implementation, the transformation parameters corresponding to the smallest resolution level include rigid transformation parameters; and the transformation parameters corresponding to each resolution level except the smallest resolution level include rigid transformation parameters and non-rigid displacement fields.

[0013] In a possible implementation, the smoothness regularization term is as follows:

[0014] L_smooth = λ * Σ ||∇u||²

[0015] Wherein, L_smooth is the smoothness regularization term, u is the displacement field, ∇u is the displacement field gradient, and λ is the weight coefficient.

[0016] In a possible implementation, the isometry regularization term is as follows:

[0017] L_isom = μ * Σ ||D_tgt_s - D_tgt||²

[0018] Wherein, L_isom is the isometry regularization, D_tgt_s is the distance matrix of a point in the target point cloud data and its neighborhood points, D_tgt is the distance matrix of the corresponding point after transformation and its neighborhood points, and μ is the weight.

[0019] In a possible implementation, the tower component point positioning method based on point cloud hierarchical registration further comprises:

[0020] determining the index of the key part point in the target point cloud;

[0021] recording and returning the index of the key part point in the target point cloud and the final positioning coordinates.

[0022] In a possible implementation, before the multi-level down-sampling processing of the target point cloud and the reference point cloud, the tower component point positioning method based on point cloud hierarchical registration further comprises the following steps:

[0023] standardizing the reference point cloud and the target point cloud respectively; wherein the standardization processing comprises at least one of the following: decentralization processing, scale normalization processing.

[0024] In a possible implementation, before processing the target point cloud data after the feature alignment processing by using the deep learning model, the tower component point positioning method based on point cloud hierarchical registration further comprises the following steps:

[0025] configuring parameters of the deep learning model;

[0026] The parameters comprise at least one of the following: model depth, model width, applicable device, learning rate, iteration number, training stop condition.

[0027] In a possible implementation, for the minimum resolution level, the target point cloud data and the reference point cloud data of the resolution level are registered to obtain the transformation parameter for aligning the target point cloud data to the reference point cloud data, comprising:

[0028] using the method based on FPFH feature matching or using the deep learning model to register the target point cloud data and the reference point cloud data of the minimum resolution level to obtain the transformation parameter for aligning the target point cloud data to the reference point cloud data.

[0029] In a possible implementation, the transformation function is generated based on the transformation parameter, comprising:

[0030] The transformation function is generated by using the following formula:

[0031] T(p)=R*p+t+u(p)

[0032] Wherein, T(p) represents the transformation function corresponding to the point p in the target point cloud, R represents the rotation matrix, t represents the translation vector, u(p) represents the non-rigid displacement field corresponding to the point p in the target point cloud, and (R, t) is the rigid transformation parameter.

[0033] According to another aspect of the present disclosure, a tower component point positioning device based on point cloud hierarchical registration is provided, comprising:

[0034] a point cloud acquisition module configured to acquire a reference point cloud pre-labeled with key component points and a target point cloud to be labeled with key component points;

[0035] a hierarchical module configured to perform multi-level down-sampling on the target point cloud to obtain target point cloud data of a plurality of preset resolution levels, and perform multi-level down-sampling on the reference point cloud to obtain reference point cloud data of the plurality of preset resolution levels;

[0036] a matching module configured to, for the smallest resolution level, perform registration on the target point cloud data and the reference point cloud data of the resolution level to obtain transformation parameters for aligning the target point cloud data to the reference point cloud data;

[0037] in ascending order, for each resolution level other than the smallest resolution level, performing the following operations: performing feature alignment processing on the target point cloud data of the resolution level by using the transformation parameters corresponding to the previous resolution level of the resolution level, and inputting the target point cloud data after the feature alignment processing into the trained deep learning model to enable the deep learning model to output transformation parameters corresponding to the resolution level for aligning the target point cloud data to the reference point cloud data; wherein a loss function of the deep learning model comprises a smoothness regular term, an equidistance regular term and a rigidity prior term;

[0038] a function generation module configured to generate a transformation function based on the transformation parameters;

[0039] an alignment module configured to convert each point in the target point cloud to a coordinate system corresponding to the reference point cloud by using the transformation function;

[0040] a positioning module configured to, based on coordinates of key component points in the reference point cloud, perform neighborhood search in the target point cloud in the coordinate system corresponding to the reference point cloud, take the nearest neighbor point to the key component points as a key part point in the target point cloud, and take coordinates of the key part point in the target point cloud in the coordinate system corresponding to the target point cloud as final positioning coordinates of the key part point in the target point cloud.

[0041] The tower pole component point positioning method and device based on point cloud hierarchical registration of the present disclosure first acquires a reference point cloud in which key component points are pre-labeled, and a target point cloud in which key component point positioning is to be performed; then performs multi-level down-sampling processing on the target point cloud to obtain target point cloud data of a plurality of preset resolution levels; performs multi-level down-sampling processing on the reference point cloud to obtain reference point cloud data of a plurality of preset resolution levels; then, for the smallest resolution level, performs registration on the target point cloud data and the reference point cloud data of the resolution level to obtain a transformation parameter for aligning the target point cloud data to the reference point cloud data; for each resolution level other than the smallest resolution level, in ascending order, performs the following operations: performs feature alignment processing on the target point cloud data of the resolution level by using the transformation parameter corresponding to the previous resolution level of the resolution level, and inputs the target point cloud data after the feature alignment processing into a trained deep learning model to enable the deep learning model to output a transformation parameter corresponding to the resolution level for aligning the target point cloud data to the reference point cloud data; wherein the loss function of the deep learning model includes a smoothness regular term, an equidistance regular term, and a rigidity prior term; then generates a transformation function based on the transformation parameter, and converts each point in the target point cloud to a coordinate system corresponding to the reference point cloud by using the transformation function; finally, based on the coordinates of the key component points in the reference point cloud, performs neighborhood search in the target point cloud located in the coordinate system corresponding to the reference point cloud, and takes the nearest neighbor point to the key component points as the key part points in the target point cloud; and takes the coordinates of the key part points in the target point cloud in the coordinate system corresponding to the target point cloud as the final positioning coordinates of the key part points in the target point cloud.

[0042] The scheme of the present disclosure can better cope with the non-rigid deformation and local detail changes existing in the power tower by introducing a hierarchical optimization strategy and a non-rigid regularization loss mechanism, thereby realizing more accurate key component point positioning, improving the robustness and automation degree of registration, and being particularly suitable for objects such as power towers with complex geometric shapes.

[0043] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings serve to better understand the present scheme and do not constitute a limitation on the present disclosure. Among them:

[0045] Figure 1 is a flowchart of the tower pole component point positioning method based on point cloud hierarchical registration in the embodiments of the present disclosure;

[0046] Figure 2is a schematic diagram of a reference point cloud in the embodiment of the disclosure;

[0047] Figure 3 is a structural schematic diagram of a tower pole component point positioning device based on point cloud hierarchical registration in the embodiment of the disclosure. DETAILED DESCRIPTION

[0048] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and structures are omitted in the following description for the sake of clarity and conciseness.

[0049] The present disclosure proposes a tower pole component point positioning method and device based on point cloud hierarchical registration to solve the problems of low efficiency, poor precision, and poor applicability in the current technology when positioning key components of a tower pole. The present disclosure combines high-precision, deformation-resistant deep learning registration with nearest neighbor search based on prior knowledge to achieve seamless connection from "overall alignment" to "precise positioning of key points". After registration is completed, the extraction process of target key component points converts complex three-dimensional perception problems into efficient, reliable, and automated coordinate extraction tasks based on high-quality registration results. Compared with existing technologies that rely on manual work or independent component recognition models, the scheme of the present disclosure has achieved a qualitative leap in automation degree, extraction precision, adaptability to real-world complexity, and richness of diagnostic information, providing strong technical support for intelligent operation and maintenance of power systems. In summary, the present disclosure surpasses existing key component point methods in multiple aspects through innovative technical solutions, providing more accurate, reliable, and efficient solutions, especially suitable for complex structures such as power poles.

[0050] The technical solutions of the present disclosure will be described below through specific embodiments.

[0051] As shown in Figure 1 is a flowchart of the tower pole component point positioning method based on point cloud hierarchical registration of the present embodiment. The execution subject of the present embodiment is a computing device or component with data processing capability. Specifically, the method of the present embodiment can include the following steps:

[0052] S110, obtaining a reference point cloud of a tower pole pre-labeled with key component points, and a target point cloud of a tower pole to be positioned with key component points.

[0053] The above reference point cloud is a tower point cloud with key component point labeling, denoted as P_reference and KP_re. The point cloud data is derived from a power tower with complete structure, clear components, and key component points pre-determined by artificial labeling or high-precision modeling. These key component points include but are not limited to: insulator string connection points, cross arm end points, tower body main material connection nodes, lightning conductor support points, etc.

[0054] The target point cloud is a tower point cloud to be positioned for key component points, denoted as P_target. The point cloud data is derived from a power tower to be detected or analyzed, and the positions of its key component points are unknown and need to be extracted and positioned by the method of the embodiment.

[0055] Two point cloud data sets can be obtained by scanning the target power tower using a three-dimensional laser scanner (LiDAR) or a laser radar system mounted on a drone.

[0056] The reference point cloud and component point format are as shown in Figure 2 , where the component point coordinates are coded in geographic information. Figure 2

[0057] After obtaining the reference point cloud and the target point cloud, the reference point cloud and the target point cloud can be standardized respectively; wherein the standardization processing includes at least one of the following: decentralization processing, scale normalization processing.

[0058] The above standardization processing can facilitate subsequent operations, and the point cloud can be down-sampled as needed to reduce the amount of calculation while retaining key features.

[0059] S120, performing multi-level down-sampling processing on the target point cloud to obtain target point cloud data of a plurality of preset resolution levels; performing multi-level down-sampling processing on the reference point cloud to obtain reference point cloud data of a plurality of preset resolution levels.

[0060] In the point cloud registration task, especially when facing targets such as power towers with complex hierarchical structures (such as tower base, main material, cross arm, insulator string, etc.) and possible local deformation, direct global optimization often faces the following challenges: this embodiment introduces a hierarchical optimization mechanism, allowing the model to gradually optimize the transformation relationship between point clouds from rough to fine. This strategy helps improve the stability of registration, especially when dealing with non-rigid deformation.

[0061] This strategy is usually implemented in combination with a point cloud pyramid (Point Cloud Pyramid) or feature pyramid (Feature Pyramid) structure, and the specific steps are as follows:

[0062] ​Constructing point cloud pyramid: multi-level down-sampling (such as voxel grid filtering, random sampling) is performed on the reference point cloud and the target point cloud respectively, to generate multiple resolution levels from coarse to fine (for example, L3 is the coarsest, and L0 is the finest). Each level represents a different scale of geometric structure: high levels (coarse) capture the overall contour and approximate pose; low levels (fine) retain more detailed features.

[0063] In S130, for the smallest resolution level, the target point cloud data and the reference point cloud data of the resolution level are registered to obtain a transformation parameter for aligning the target point cloud data to the reference point cloud data.

[0064] The transformation parameter corresponding to the smallest resolution level includes a rigid transformation parameter, such as a global transformation matrix.

[0065] In a specific implementation, the target point cloud data and the reference point cloud data of the smallest resolution level are registered by using a method based on FPFH feature matching or by using the deep learning model to obtain the transformation parameter for aligning the target point cloud data to the reference point cloud data.

[0066] This step is the coarse registration stage, for example, L3→L2 registration. In the coarsest level, a lightweight matching algorithm (such as rough matching based on FPFH features) or a deep learning model is used for preliminary alignment to obtain a roughly correct global transformation matrix (rotation and translation). This stage has fast calculation speed and strong noise resistance.

[0067] The transformation matrix obtained in the coarse registration stage (and in all subsequent level optimization stages) is applied to the target point cloud P_target, and the purpose is to transform it from its original coordinate system to the coordinate system of the reference point cloud. This transformation matrix defines the pose (rotation and translation, and subsequent local deformation) of the target point cloud relative to the reference point cloud.

[0068] In S140, for each resolution level except the smallest resolution level, in ascending order, the following operations are performed: feature alignment processing is performed on the target point cloud data of the resolution level by using the transformation parameter corresponding to the previous resolution level of the resolution level, and the target point cloud data after the feature alignment processing is input into the trained deep learning model, so that the deep learning model outputs the transformation parameter corresponding to the resolution level for aligning the target point cloud data to the reference point cloud data; wherein the loss function of the deep learning model includes a smoothness regular term, an equidistance regular term, and a rigidity prior term.

[0069] In specific implementation, a deep learning model suitable for point cloud data processing can be selected, for example, a neural deformation pyramid (NDP) model, which is specially designed for processing point cloud data in three-dimensional space and can effectively capture complex correspondence between point clouds.

[0070] After selecting the deep learning model, the model needs to be configured with the following key parameters:

[0071] Depth (model depth): defines the number of layers or modules of the model. Width (model width): determines the number of neurons in each layer. Device (applicable device): select the running device (CPU or GPU) according to available resources. Learning Rate (learning rate): controls the speed of model parameter update. Iterations (number of iterations): sets the maximum number of model parameter updates in the training process. Break Threshold Ratio (stop threshold ratio): in the training process, when the change of the loss function is less than a certain proportion, the training is terminated in advance to prevent overfitting. Among them, the above stop threshold ratio is the iteration stop condition.

[0072] The transformation parameters corresponding to each resolution level except the smallest resolution level include rigid transformation parameters and non-rigid displacement fields.

[0073] The power tower may produce local non-rigid deformation due to wind load, thermal expansion and contraction, foundation settlement or slight damage in actual operation. Traditional rigid or near-rigid registration methods (such as ICP) assume that the entire object only rotates and translates, which cannot effectively handle such local deformation, resulting in positioning deviation of key components. To solve this problem, the present disclosure introduces a non-rigid regularization loss term (including a smoothness regularization term, an equidistance regularization term, and a rigidity prior term) in the loss function of the deep learning registration model, allowing the model to learn and compensate for local deformation while maintaining the overall structure. The core idea of the non-rigid regularization loss is to constrain the smoothness and physical reasonableness of the deformation field to prevent the model from predicting excessive distortion or unnatural transformation.

[0074] The regularization loss function design includes a smoothness regularization term, an equidistance regularization term, and a rigidity prior term.

[0075] The above smoothness regularization term is as follows:

[0076] L_smooth = λ * Σ ||∇u||²

[0077] Where L_smooth is the smoothness regularization term, u is the displacement field, ∇u is the displacement field gradient, and λ is the weight coefficient.

[0078] The smoothness regularization term acts to penalize the drastic changes of displacement field, encouraging the displacement of neighboring points to be similar, ensuring the local smoothness of deformation, and conforming to the physical law.

[0079] The isometry regularization term is as follows:

[0080] L_isom = μ * Σ ||D_tgt_s - D_tgt||²

[0081] Where L_isom is the isometry regularization, D_tgt_s is the distance matrix of a point in the target point cloud data and its neighborhood points, D_tgt is the distance matrix of the corresponding point after transformation and its neighborhood points, and μ is the weight.

[0082] The isometry regularization term acts to constrain the deformation to maintain the local geometric structure (such as angle, distance) as much as possible, preventing excessive stretching or compression.

[0083] The rigid prior term is an optional item that imposes stronger rigid constraints on known rigid regions (such as tower main materials) or adds a residual term of global rigid transformation to the loss.

[0084] The total loss function of the deep learning model is:

[0085] L_total = L_match + α * L_smooth + β * L_isom + γ * L_rigid

[0086] Where L_match is the main matching loss (such as Chamfer Distance, Earth Mover's Distance), α, β, γ are the hyperparameters of each regularization term, which can be tuned through the validation set.

[0087] Through the synergistic effect of hierarchical optimization strategy and non-rigid regularization loss, the present disclosure constructs an efficient, robust and physically reasonable deep learning registration framework: hierarchical optimization solves the problem of "how to efficiently and stably find the correct solution space", providing high-quality initial conditions for subsequent fine registration. Non-rigid regularization loss solves the problem of "how to accurately align key points when there is local deformation", improving the model's ability to adapt to real-world complexity.

[0088] Specifically, step S140 can include middle-level optimization (L2 → L1) and fine registration (L1 → L0).

[0089] Middle-level optimization (L2 → L1): The transformation parameters obtained in the previous stage are used as initial values, which are mapped to a more refined level for more accurate local optimization. This stage begins to focus on the alignment of main structural components (such as cross arms and main materials).

[0090] The "transform parameters obtained in the last stage" refer to the global transform matrix calculated in the coarse registration stage for aligning the target point cloud from the coarsest level (L3) to the reference point cloud coarsest level (L3).

[0091] The "projecting to a finer level" specifically performs matrix operations. Specifically as follows:

[0092] Obtain the point cloud of the finer level: Take out the point cloud data P_target_L2 of the target point cloud at the L2 level from the pyramid. Take out the point cloud data P_reference_L2 of the reference point cloud at the L2 level from the pyramid.

[0093] Apply the transform:

[0094] Directly apply the global transform matrix obtained in the last stage to P_target_L2.

[0095] Therefore, in the middle-level optimization (L2→L1) stage, the algorithm no longer starts from zero to search, but in this good initial state, it performs incremental and finer optimization.

[0096] Fine registration (L1→L0): At the level of the original resolution or close to the original resolution, the final fine optimization is performed. At this time, the input is the point cloud well initialized by the previous stages, and the optimization process is more stable, which can accurately align key component points such as insulator connection points, bolt hole positions, etc.

[0097] In the L1→L0 stage, a deep learning model is used to perform final optimization on the high-resolution point cloud using the good initial alignment provided by the previous stages. It constrains the smoothness and reasonableness of deformation through L_smooth, L_isom, etc. loss, predicts fine rigid transformation parameters (R, t) and dense non-rigid displacement field u(p). Finally, this complete transform T_final (i.e. the following transform function) is applied to the original point cloud (i.e. the original target point cloud), realizing high-precision alignment containing local deformation.

[0098] Through the execution of steps S120-S140, the configured deep learning model is used to register the reference point cloud and the target point cloud, and through a series of optimization steps, the best spatial transformation parameters are found to make the reference point cloud align with the target point cloud as much as possible.

[0099] Steps S120-S140 implement an information transfer mechanism, and the optimization result (transform parameters) of each level is used as the initial transform of the next level, forming a cascading optimization. A differentiable sampling layer can be introduced to make the entire pyramid structure end-to-end trainable, which is convenient for implementation in a deep learning framework.

[0100] S150, generating a transformation function based on the transformation parameters.

[0101] Generating a transformation function based on the transformation parameters includes:

[0102] Generating a transformation function based on the transformation parameters includes:

[0103] T(p) = R * p + t + u(p)

[0104] Where T(p) represents the transformation function corresponding to point p in the target point cloud, Rp represents the rotation matrix, t represents the translation vector, u(p) represents the non-rigid displacement field corresponding to point p in the target point cloud, and (R, t) is the rigid transformation parameter.

[0105] S160, using the transformation function to convert each point in the target point cloud to the coordinate system corresponding to the reference point cloud.

[0106] S170, based on the coordinates of the key component points in the reference point cloud, performing neighborhood search in the target point cloud located in the coordinate system corresponding to the reference point cloud, taking the nearest neighbor point of the key component points as the key part point in the target point cloud; and taking the coordinates of the key part point in the target point cloud in the coordinate system corresponding to the target point cloud as the final positioning coordinates of the key part point in the target point cloud.

[0107] After determining the above final positioning coordinates, the following steps can also be included:

[0108] Determine the index of the key part point in the target point cloud; record and return the index and final positioning coordinates of the key part point in the target point cloud.

[0109] In some embodiments, the process of obtaining the final transformation function T is realized through a deep learning registration framework that integrates a hierarchical optimization strategy and a non-rigid regularization loss. The process of obtaining the transformation function T is as follows:

[0110] 1. Initial registration and rough alignment

[0111] Constructing a point cloud pyramid: First, according to the "hierarchical optimization strategy", a multi-level resolution point cloud pyramid is constructed for the reference point cloud and the target point cloud. Starting from the coarsest level of point cloud for registration, which helps to quickly find the global transformation parameters (rotation matrix R and translation vector t), and reduce the search space in the subsequent fine registration stage.

[0112] Rough registration: In this stage, a feature descriptor-based method (such as FPFH feature) combined with RANSAC algorithm or simple ICP method is usually used to obtain a preliminary rigid transformation estimate. This preliminary transformation serves as the basis for subsequent more fine registration.

[0113] 2. Deep learning model application

[0114] Model input: For each level of point cloud data (from coarse to fine), the aligned point cloud data obtained from the previous stage will be input into the pre-trained deep learning model. The design of the deep learning model takes into account the special structure of the power tower and the possible local deformation.

[0115] Predicting rigid transformation and non-rigid displacement field:

[0116] Rigid transformation (R, t): Part of the deep learning model responsible for outputting the overall rigid transformation parameters, i.e., rotation matrix R and translation vector t. This part is usually based on the matching relationship between point clouds, using neural networks to learn the best global alignment.

[0117] Non-rigid displacement field (u(p)): Another part is to predict the local non-rigid displacement vector u(p) for each point. This is to compensate for the local deformation of the target point cloud relative to the reference point cloud due to environmental factors (such as wind load, thermal expansion and contraction, etc.). p represents a point.

[0118] 3. Application of non-rigid regularization loss

[0119] During the training process of the deep learning model, in order to ensure that the predicted non-rigid displacement field can accurately reflect the actual local deformation and not produce unreasonable distortion, non-rigid regularization loss is introduced. These loss terms include but are not limited to:

[0120] Smoothness regularization term: Encourages smooth changes in displacement between adjacent points, avoiding abrupt changes.

[0121] Isometry regularization term: Maintains local geometric structure, such as angles and distances should not change significantly.

[0122] Rigid prior term (optional): Apply stronger constraints to areas known to remain rigid.

[0123] 4. Combining results from each level

[0124] Step-by-step refinement: As each layer in the point cloud pyramid is processed, the learned transformation from the previous level is used as the initial value for the next level. This is done to speed up the convergence of the current layer and improve registration accuracy using the results of the previous layer.

[0125] Final transformation combination: After reaching the highest resolution level, the resulting transformation includes the global rigid transformation (R, t) and the local non-rigid displacement field (u(p)). These two parts together form the final transformation function T.

[0126] 5. Form of transformation function T

[0127] The final transformation function T can be represented as:

[0128] T(p) = Rp + t + u(p)

[0129] Here p is a point coordinate in the target point cloud, R and t are the global rotation and translation transformation respectively, while u(p) is the non-rigid displacement adjustment of the point.

[0130] Through the above steps, the embodiments of the present disclosure not only can handle the overall movement of complex structures such as power towers, but also can effectively capture and compensate for local deformations caused by various reasons, thereby achieving more accurate point cloud registration and key component point extraction.

[0131] In some examples, the specific implementation process of determining the transformation function T is as follows:

[0132] Input: reference point cloud P_ref and target point cloud P_target.

[0133] Layered optimization strategy starts: multi-level processing is adopted.

[0134] Deep deformation pyramid model execution:

[0135] Construct point cloud / feature pyramid.

[0136] At Level 3: Model predicts rough R, t, and applies transformation.

[0137] At Level 2: Model predicts fine R, t and preliminary u(p) based on aligned features.

[0138] At Level 1 / 0: Model outputs final R, t and final u(p).

[0139] Non-rigid regularization loss intervention:

[0140] During the training phase, for u(p) predicted by the deep learning model at each level (or final), L_smooth, L_isom, L_rigid are calculated.

[0141] These loss terms together with L_match constitute L_total.

[0142] Through backpropagation, the gradient of L_total updates all parameters of the entire "deep deformation pyramid model".

[0143] Output: The well-trained model can predict a high-quality transformation T = R*p + t+ u(p) for new point cloud pairs.

[0144] Point cloud registration is the core step of the technical solution of the present disclosure. The ultimate goal of the technical solution of the present disclosure is not only to realize the overall alignment of two point cloud sets, but also to accurately and automatically extract the three-dimensional coordinates of key component points on the power tower to be measured (target point cloud). After registration is completed, the extraction process of the key component points is the final landing point of realizing the practical value of the present disclosure. The scheme of the present disclosure realizes the automatic extraction with high precision and high reliability by innovatively combining the registration result and prior knowledge. After registration is completed, each point in the target point cloud is mapped to the coordinate system of the reference point cloud through the learned transformation function (rigid + non-rigid deformation field). The extraction process of the key component points, i.e., steps S160-S170, can be as follows:

[0145] 1) Transformation application and coordinate mapping:

[0146] Apply the final transformation function T learned by registration (including global rigid transformation and local non-rigid displacement field) to all points in the target point cloud P_target.

[0147] Calculate the transformed point cloud P_target_transformed = T(P_target). At this time, P_target_transformed is highly aligned with the reference point cloud P_reference in space.

[0148] 2) Key point positioning based on nearest neighbor search:

[0149] For each key component point KP_ref_i in the reference point cloud (whose coordinates are known and stored in the component point file), perform k-neighbor (k-NN) search or radius search (Radius Search) in the transformed target point cloud P_target_transformed.

[0150] Find one or more points closest to KP_ref_i. Usually, the nearest neighbor point is selected as the corresponding point of the key component in the target point cloud.

[0151] Record the index idx and original coordinates coord_original of the nearest neighbor point in the original target point cloud P_target.

[0152] 3) Key component point information generation:

[0153] Create an entry for each extracted key component point, including:

[0154] ID and name: inherited from the reference component point file (such as KP_001, medium small side insulator).

[0155] Extraction coordinates: coord_original (coordinates in the original target point cloud coordinate system).

[0156] Type: inherited from the reference component point file.

[0157] Associated information: records its corresponding ID in the reference point cloud, for easy tracing.

[0158] 4) Result output:

[0159] All extracted key component point information is in the same format as the input component point file (or CSV, etc.). At the same time, the extracted key points can be highlighted and visualized on the original target point cloud for manual review.

[0160] The extraction of key component points is highly dependent on and directly uses the results of registration, embodying the design idea of "registration serving extraction":

[0161] Precision transfer: the extraction accuracy directly depends on the registration accuracy. Layered optimization and non-rigid regularization loss ensure high-precision alignment of P_target_transformed and P_reference, so that the nearest neighbor search can find the truly corresponding physical points.

[0162] Semantic inheritance: the semantic information (ID, name, type) of the key points is completely inherited from the component point file of the reference point cloud, avoiding complex and error-prone semantic segmentation on the target point cloud.

[0163] Handle local missing / occlusion: even if a certain key component area in the target point cloud is missing due to occlusion, the registration algorithm will still try to "align" the corresponding area of the reference point cloud to the target point cloud. At this time, the nearest neighbor search may fail or return a point with a very long distance. The system can set a distance threshold to mark this point as "not found" or "suspected missing / damaged", which itself is also valuable diagnostic information.

[0164] Compared with the prior art, the scheme of the present disclosure has at least one of the following advantages:

[0165] Comparison dimensions Prior art (typical scheme) This disclosure (key point extraction based on registration) Automation degree Manual key point annotation on target point cloud is required, or complex component segmentation + recognition model is relied on. Fully automated, no additional segmentation or identification on target point cloud is required. Accuracy and reliability Manual annotation is highly subjective and prone to error; automatic segmentation model has limited accuracy in complex backgrounds. Accuracy is guaranteed by high-precision registration, and the result is objective and repeatable; confidence score provides reliability evaluation. Deformation processing capability Rigid registration + segmentation method cannot handle local deformation, resulting in key point misplacement. Non-rigid registration can compensate for deformation, and the extracted points can accurately reflect the true position of the component in the current deformation state. Knowledge reuse Each new tower type may require retraining of the segmentation model or reannotation. Only once for a standard tower to create a component point file (i.e., a reference point cloud with annotations), it can be used for automated extraction of all towers of the same type.

[0166] Based on the same inventive concept, the present disclosure provides a tower pole component point positioning device based on point cloud hierarchical registration. The steps performed by the components of the device are the same or similar to those of the above-mentioned method, and therefore similar parts will not be described again. As shown in Figure 3 the tower pole component point positioning device based on point cloud hierarchical registration of the present embodiment comprises:

[0167] The point cloud acquisition module 310 is configured to acquire a reference point cloud of a tower pole on which key component points are labeled in advance and a target point cloud of the tower pole on which key component points are to be positioned.

[0168] The hierarchical module 320 is configured to perform multi-level down-sampling processing on the target point cloud to obtain target point cloud data of a plurality of preset resolution levels, and perform multi-level down-sampling processing on the reference point cloud to obtain reference point cloud data of the plurality of preset resolution levels.

[0169] The matching module 330 is configured to, for the smallest resolution level, perform registration on the target point cloud data and the reference point cloud data of the smallest resolution level to obtain a transformation parameter for aligning the target point cloud data to the reference point cloud data. For each resolution level other than the smallest resolution level, in ascending order, the following operations are performed: performing feature alignment processing on the target point cloud data of the resolution level by using the transformation parameter corresponding to the previous resolution level of the resolution level, and inputting the target point cloud data after the feature alignment processing into the trained deep learning model, so that the deep learning model outputs a transformation parameter corresponding to the resolution level and used for aligning the target point cloud data to the reference point cloud data; and the loss function of the deep learning model includes a smoothness regular term, an equidistance regular term and a rigidity prior term.

[0170] The function generation module 340 is configured to generate a transformation function based on the transformation parameter.

[0171] The alignment module 350 is configured to convert each point in the target point cloud to a coordinate system corresponding to the reference point cloud by using the transformation function.

[0172] The positioning module 360 is configured to perform neighborhood search in the target point cloud in the coordinate system corresponding to the reference point cloud based on the coordinates of the key component points in the reference point cloud, take the nearest neighbor point of the key component points as the key part points in the target point cloud, and take the coordinates of the key part points in the target point cloud in the coordinate system corresponding to the target point cloud as the final positioning coordinates of the key part points in the target point cloud.

[0173] In some embodiments, the transformation parameter corresponding to the smallest resolution level includes a rigid transformation parameter, and the transformation parameter corresponding to each resolution level other than the smallest resolution level includes a rigid transformation parameter and a non-rigid displacement field.

[0174] In some embodiments, the smoothness regular term is as follows:

[0175] L_smooth = λ * Σ ||∇u||²

[0176] Wherein, L_smooth is the smoothness regular term, u is the displacement field, ∇u is the displacement field gradient, and λ is the weight coefficient.

[0177] In some embodiments, the isometry regular term described above is as follows:

[0178] L_isom = μ * Σ ||D_tgt_s - D_tgt||²

[0179] Wherein, L_isom is the isometry regular, D_tgt_s is the distance matrix of a point in the target point cloud data and its neighborhood points, D_tgt is the distance matrix of the corresponding point after transformation and its neighborhood points, and μ is the weight.

[0180] In some embodiments, the following can also be performed:

[0181] Determine the index of the key part point in the target point cloud;

[0182] Record and return the index and final positioning coordinates of the key part point in the target point cloud.

[0183] In some embodiments, before the multi-level down-sampling processing of the target point cloud and the reference point cloud, the following can also be performed:

[0184] Standardization processing is performed on the reference point cloud and the target point cloud respectively; wherein, the standardization processing includes at least one of the following: decentralization processing, scale normalization processing.

[0185] In some embodiments, before processing the target point cloud data after feature alignment processing by using the deep learning model, the following steps can also be performed:

[0186] Configure the parameters of the deep learning model;

[0187] The parameters include at least one of the following: model depth, model width, applicable device, learning rate, iteration number, training stop condition.

[0188] In some embodiments, for the minimum resolution level, the target point cloud data and the reference point cloud data of the resolution level are registered to obtain the transformation parameter for aligning the target point cloud data to the reference point cloud data, which can specifically include:

[0189] The target point cloud data and the reference point cloud data of the minimum resolution level are registered by using the method based on FPFH feature matching or by using the deep learning model to obtain the transformation parameter for aligning the target point cloud data to the reference point cloud data.

[0190] In some embodiments, the generation of the transformation function based on the transformation parameter includes:

[0191] The transformation function is generated using the following equation:

[0192] T(p) = R * p + t + u(p)

[0193] where T(p) denotes the transformation function corresponding to point p in the target point cloud, R denotes the rotation matrix, t denotes the translation vector, u(p) denotes the non-rigid displacement field corresponding to point p in the target point cloud, and (R, t) are the rigid transformation parameters.

[0194] Various implementations of the above described technology can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0195] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0196] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0197] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0198] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0199] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0200] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, without limitation herein, so long as the desired results of the technology described in the present disclosure are achieved.

[0201] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for point positioning of tower components based on hierarchical registration of point clouds, characterized in that, include: Obtain the reference point cloud of the tower with key component points pre-marked, and the target point cloud of the tower to be located for key component points; The target point cloud is subjected to multi-level downsampling processing to obtain target point cloud data at multiple preset resolution levels; the reference point cloud is subjected to multi-level downsampling processing to obtain reference point cloud data at multiple preset resolution levels. For the smallest resolution level, the target point cloud data and the reference point cloud data at that resolution level are registered to obtain the transformation parameters used to align the target point cloud data to the reference point cloud data. Following an ascending order, for each resolution level except the smallest, the following operations are performed: The target point cloud data at that resolution level is feature-aligned using the transformation parameters corresponding to the previous resolution level. The feature-aligned target point cloud data is then input into a trained deep learning model, causing the deep learning model to output transformation parameters corresponding to that resolution level for aligning the target point cloud data to the reference point cloud data. The loss function of the deep learning model includes a smoothness regularization term, an isometry regularization term, and a rigid prior term. Generate a transformation function based on the transformation parameters; Using the transformation function, each point in the target point cloud is transformed to the coordinate system corresponding to the reference point cloud; Based on the coordinates of the key component points in the reference point cloud, a neighborhood search is performed in the target point cloud located in the coordinate system corresponding to the reference point cloud, and the nearest neighbor of the key component point is taken as the key part point in the target point cloud; and the coordinates of the key part point in the target point cloud in the coordinate system corresponding to the target point cloud are taken as the final positioning coordinates of the key part point in the target point cloud.

2. The tower component point positioning method based on point cloud hierarchical registration according to claim 1, characterized in that, The transformation parameters corresponding to the smallest resolution level include rigid transformation parameters; the transformation parameters corresponding to each resolution level other than the smallest resolution level include rigid transformation parameters and non-rigid displacement fields.

3. The tower component point positioning method based on point cloud hierarchical registration according to claim 1, characterized in that, The smoothness regularization term is as follows: L_smooth = λ * Σ ||∇u||²; Where L_smooth is the smoothness regularization term, u is the displacement field, ∇u is the displacement field gradient, and λ is the weighting coefficient.

4. The tower component point positioning method based on point cloud hierarchical registration according to claim 1, characterized in that, The isometry regularization term is as follows: L_isom = μ * Σ ||D_tgt_s - D_tgt||²; Where L_isom is the isometry regularization, D_tgt_s is the distance matrix between a point in the target point cloud data and its neighboring points, D_tgt is the distance matrix between the transformed point and its neighboring points, and μ is the weight.

5. The tower component point positioning method based on point cloud hierarchical registration according to claim 1, characterized in that, Also includes: Determine the indices of key points in the target point cloud; Record and return the indexes and final location coordinates of key points in the target point cloud.

6. The tower component point positioning method based on point cloud hierarchical registration according to claim 1, characterized in that, Before performing multi-level downsampling processing on the target point cloud and the reference point cloud, the method further includes: The reference point cloud and the target point cloud are respectively subjected to standardization processing; wherein, the standardization processing includes at least one of the following: decentralization processing and scale normalization processing.

7. The tower component point positioning method based on point cloud hierarchical registration according to claim 1, characterized in that, Before processing the feature-aligned target point cloud data using the deep learning model, the following steps are also included: Configure the parameters of the deep learning model; The parameters include at least one of the following: model depth, model width, applicable device, learning rate, number of iterations, and training stopping condition.

8. The tower component point positioning method based on point cloud hierarchical registration according to claim 1, characterized in that, For the smallest resolution level, the target point cloud data and reference point cloud data at that resolution level are registered to obtain transformation parameters for aligning the target point cloud data to the reference point cloud data, including: Using the FPFH feature matching method or the deep learning model, the target point cloud data and the reference point cloud data at the smallest resolution level are registered to obtain the transformation parameters used to align the target point cloud data to the reference point cloud data.

9. The tower component point positioning method based on point cloud hierarchical registration according to claim 2, characterized in that, The generation of the transformation function based on the transformation parameters includes: The transformation function is generated using the following formula: T(p) = R*p + t + u(p); Where T(p) represents the transformation function corresponding to point p in the target point cloud, R represents the rotation matrix, t represents the translation vector, u(p) represents the non-rigid displacement field corresponding to point p in the target point cloud, and (R,t) are rigid transformation parameters.

10. A point positioning device for tower components based on point cloud hierarchical registration, characterized in that, include: The point cloud acquisition module is used to acquire the reference point cloud of the tower with key component points pre-annotated, and the target point cloud of the tower to be located for key component points. The hierarchical module is used to perform multi-level downsampling processing on the target point cloud to obtain target point cloud data at multiple preset resolution levels; The reference point cloud is subjected to multi-level downsampling processing to obtain reference point cloud data at multiple preset resolution levels; The matching module is used to register the target point cloud data and the reference point cloud data at the smallest resolution level to obtain the transformation parameters used to align the target point cloud data to the reference point cloud data. Following an ascending order, for each resolution level except the smallest, the following operations are performed: The target point cloud data at that resolution level is feature-aligned using the transformation parameters corresponding to the previous resolution level. The feature-aligned target point cloud data is then input into a trained deep learning model, causing the deep learning model to output transformation parameters corresponding to that resolution level for aligning the target point cloud data to the reference point cloud data. The loss function of the deep learning model includes a smoothness regularization term, an isometry regularization term, and a rigid prior term. The function generation module is used to generate a transformation function based on the transformation parameters; The alignment module is used to transform each point in the target point cloud to the coordinate system corresponding to the reference point cloud using the transformation function; The positioning module is used to perform a neighborhood search in the target point cloud located in the coordinate system corresponding to the reference point cloud based on the coordinates of the key component points in the reference point cloud, and to take the nearest neighbor of the key component point as the key part point in the target point cloud; and to take the coordinates of the key part point in the target point cloud in the coordinate system corresponding to the target point cloud as the final positioning coordinates of the key part point in the target point cloud.

Images