Insulator string three-dimensional intelligent segmentation method and device based on fused point cloud features

Abstract

This invention discloses a three-dimensional intelligent segmentation method and device for insulator strings based on fused point cloud features. The method includes: acquiring the original point cloud of a power transmission equipment using lidar; the power transmission equipment includes insulator strings; filtering the original point cloud to generate a filtered original point cloud; determining the normal vector of each point in the filtered original point cloud; setting the vector field generated based on the normal vector to a priori that follows a periodic Gaussian process prior distribution; reconstructing the implicit surface from the filtered original point cloud using the Poisson equation of the implicit surface defined by the prior-set vector field and the prior-set vector field; adaptively sampling the point cloud points in the implicit surface to generate a dense point cloud; generating a segmentation result using a pre-trained point cloud segmentation model; and determining the final segmentation result based on the filtered original point cloud and the segmentation result. This invention can improve the efficiency, accuracy, and robustness of point cloud segmentation for small components of insulator strings.

Description

Technical Field

[0001] This invention relates to the field of point cloud segmentation technology, and in particular to a three-dimensional intelligent segmentation method and apparatus for insulator strings based on fused point cloud features. Background Technology

[0002] This section is intended to provide background or context for the embodiments of the invention described herein. The description herein is not intended to imply that it is prior art simply because it is included in this section.

[0003] In the power sector, LiDAR-based 3D point cloud recognition technology has been widely applied to tasks such as digital construction of power scenarios and power inspection. Point cloud semantic segmentation, as one of the key technologies, aims to predict category labels for each point in the LiDAR point cloud, thereby achieving accurate identification and classification of equipment components. However, existing point cloud segmentation technologies still face many challenges in practical applications.

[0004] First, the sparsity and density inhomogeneity of lidar point clouds pose challenges to capturing the complex details of equipment components, especially for small targets in power scenarios, which are often difficult to identify and segment accurately.

[0005] Secondly, due to factors such as occlusion, surface reflectivity, and limited scanning range, the original LiDAR point cloud is often incomplete, and the lack of information severely affects the accuracy of segmentation tasks. The sparsity and structural incompleteness of the point cloud not only limit the segmentation effect but also increase the risk of misidentification and missed identification.

[0006] Therefore, overcoming these shortcomings and improving the efficiency, accuracy, and robustness of point cloud segmentation is a crucial issue that urgently needs to be addressed in current point cloud processing technology in the power sector. Summary of the Invention

[0007] This invention provides a three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features, to improve the efficiency, accuracy, and robustness of point cloud segmentation for small components of insulator strings. The method includes:

[0008] The raw point cloud of the power transmission equipment, including insulator strings, was collected using lidar.

[0009] Filter the original point cloud to generate a filtered original point cloud;

[0010] The normal vector of each point in the filtered original point cloud is determined using a preset point cloud point normal vector algorithm.

[0011] The prior of the vector field generated based on the normal vector is set to follow a periodic Gaussian process prior distribution.

[0012] Using the Poisson equation of the implicit surface defined by the vector field with prior settings and the vector field with prior settings, the implicit surface is reconstructed from the filtered original point cloud;

[0013] Adaptive sampling is performed on the point cloud points in the implicit surface to generate a dense point cloud;

[0014] The fused point cloud features are extracted using a pre-trained point cloud segmentation model, and segmentation prediction is performed using the fused point cloud features to generate segmentation results; the fused point cloud is generated by fusing the original point cloud and the dense point cloud.

[0015] Based on the filtered original point cloud and the segmentation results, the final segmentation result is determined.

[0016] This invention also provides a three-dimensional intelligent segmentation device for insulator strings based on fused point cloud features, to improve the efficiency, accuracy, and robustness of point cloud segmentation for small components of insulator strings. The device includes:

[0017] The acquisition module is used to acquire the raw point cloud of the power transmission equipment using lidar; the power transmission equipment includes insulator strings.

[0018] The filtering module is used to filter the original point cloud and generate a filtered original point cloud.

[0019] The normal vector determination module is used to determine the normal vector of each point in the filtered original point cloud using a preset point cloud point normal vector algorithm.

[0020] The prior setting module is used to set the prior of the vector field generated based on the normal vector to follow a periodic Gaussian process prior distribution;

[0021] The implicit surface reconstruction module is used to reconstruct implicit surfaces from the filtered original point cloud using the Poisson equation of the implicit surface defined by the vector field set a priori and the vector field set a priori.

[0022] The dense point cloud generation module is used to adaptively sample point cloud points in the implicit surface to generate a dense point cloud.

[0023] The segmentation result generation module is used to extract fusion point cloud features from the fused point cloud using a pre-trained point cloud segmentation model, perform segmentation prediction using the fused point cloud features, and generate segmentation results; the fused point cloud is generated by fusing the original point cloud and the dense point cloud.

[0024] The final segmentation result determination module is used to determine the final segmentation result based on the filtered original point cloud and the segmentation result.

[0025] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features.

[0026] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features.

[0027] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features.

[0028] In this embodiment of the invention, the original point cloud of a power transmission equipment, including insulator strings, is acquired using a lidar system. The original point cloud is filtered to generate a filtered original point cloud. A preset point cloud normal vector algorithm is used to determine the normal vector of each point in the filtered original point cloud. The vector field generated based on the normal vector is a priori set to follow a periodic Gaussian process prior distribution. The implicit surface is reconstructed from the filtered original point cloud using the Poisson equation of the implicit surface defined by the prior-set vector field and the prior-set vector field. Adaptive sampling is performed on the point cloud points in the implicit surface to generate a dense point cloud. A pre-trained point cloud segmentation model is used to extract fusion point cloud features from the fused point cloud, and segmentation prediction is performed using the fused point cloud features to generate a segmentation result. The fused point cloud is generated by fusing the original point cloud and the dense point cloud. The final segmentation result is determined based on the filtered original point cloud and the segmentation result. Compared with existing technologies, this invention can improve the efficiency, accuracy, and robustness of point cloud segmentation for small components of insulator strings. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0030] Figure 1 This is a flowchart illustrating the three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features in an embodiment of the present invention.

[0031] Figure 2 This is a flowchart illustrating a specific example of the three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features in this invention.

[0032] Figure 3 This is a flowchart illustrating a specific example of the three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features in this invention.

[0033] Figure 4 This is a schematic diagram of a three-dimensional intelligent segmentation device for insulator strings based on fused point cloud features in an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0036] To address the aforementioned technical problems, embodiments of the present invention provide a three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features. Figure 1 This is a flowchart illustrating the three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features in an embodiment of the present invention, as shown below. Figure 1 As shown, the method may include:

[0037] Step 101: Use lidar to collect the original point cloud of the power transmission equipment; the power transmission equipment includes insulator strings;

[0038] Step 102: Filter the original point cloud to generate the filtered original point cloud;

[0039] Step 103: Use a preset point cloud point normal vector algorithm to determine the normal vector of each point cloud point in the filtered original point cloud;

[0040] Step 104: Set the prior of the vector field generated based on the normal vector to follow a prior distribution of a periodic Gaussian process;

[0041] Step 105: Using the Poisson equation of the implicit surface defined by the vector field set a priori and the vector field set a priori, the implicit surface is reconstructed from the filtered original point cloud.

[0042] Step 106: Adaptively sample the point cloud points in the implicit surface to generate a dense point cloud;

[0043] Step 107: Use a pre-trained point cloud segmentation model to extract fused point cloud features from the fused point cloud, use the fused point cloud features to perform segmentation prediction, and generate segmentation results; the fused point cloud is generated by fusing the original point cloud and the dense point cloud.

[0044] Step 108: Determine the final segmentation result based on the filtered original point cloud and the segmentation result.

[0045] This invention, through the aforementioned steps, allows for the prior setting of a vector field based on discrete normal vectors as a periodic Gaussian process. Using the Poisson equation of the implicit surface defined by the vector field, an implicit surface is reconstructed from point cloud data. Adaptive point sampling is then performed on the implicit surface to obtain a dense point cloud rich in structural features. Compared to the original point cloud, the sampled dense point cloud contains more smoothly estimated structural information. This estimated structural information enhances the semantic differences between different targets, especially smaller targets and those far from the lidar. This invention integrates point clouds and implicit surface sampling to address the problems of sparse density, unevenness, and incompleteness in the original lidar point cloud. These dense and uniform point clouds enhance the structural and semantic information of the target, resulting in a more explicit semantic meaning for the point cloud segmentation model, effectively improving the accuracy, efficiency, and robustness of the point cloud segmentation model.

[0046] In one embodiment, step 101, which involves acquiring the raw point cloud of the power transmission equipment using a lidar system, includes: scanning the power transmission equipment with a lidar device to obtain raw point cloud data, including insulator strings. This step allows for the acquisition of high-precision point cloud data, providing a foundation for subsequent processing.

[0047] In one embodiment, step 102, filtering the original point cloud to generate a filtered original point cloud, may include: using a filtering algorithm to remove noise and outliers to generate the filtered original point cloud. For example, a statistical filter or a voxel filter can be used to remove noise points. This step can improve the quality of the point cloud data and reduce interference in subsequent processing.

[0048] Figure 2 This is a flowchart illustrating a specific example of the three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features, as described in this embodiment of the invention. Figure 2 As shown, in one embodiment, step 103, using a preset point cloud point normal vector algorithm, includes:

[0049] Step 1031: Using each point in the filtered original point cloud as the center, determine the neighborhood of each center through a preset neighborhood radius;

[0050] Step 1032: Determine the centroid of each neighborhood by averaging the position information of each point cloud point in the neighborhood.

[0051] Step 1033: Generate a covariance matrix based on the centroid of each neighborhood and the position information of each point cloud point in each neighborhood; the covariance matrix is ​​used to describe the distribution direction of each point cloud point relative to the centroid in each neighborhood.

[0052] Step 1034: Calculate the eigenvalues ​​and corresponding eigenvectors of the covariance matrix;

[0053] Step 1035: Compare the magnitudes of the eigenvalues ​​to determine the smallest eigenvalue;

[0054] Step 1036: Determine the eigenvector corresponding to the minimum eigenvalue as the normal vector of the point cloud point.

[0055] In this embodiment, the process of calculating the normal vector of each point cloud point can be as follows: Taking each point cloud point in the filtered original point cloud as the center, the neighborhood of each point cloud point in the point cloud is obtained through a preset neighborhood radius. For each neighborhood, its centroid is calculated by averaging the coordinates of the points in the neighborhood. Based on the centroid and the position of each point in the neighborhood, a covariance matrix describing the distribution direction of the neighborhood points relative to the centroid is calculated. The eigenvalues ​​and eigenvectors of the covariance matrix are calculated. The magnitudes of the eigenvalues ​​are compared, and the eigenvector corresponding to the smallest eigenvalue is obtained, which is the eigenvector of each point cloud point p. i =(x i ,y i ,z i The normal vector n of ) i .

[0056] Step 103 allows for the accurate calculation of the normal vector for each point cloud point, providing crucial information for subsequent implicit surface reconstruction.

[0057] In one embodiment, for steps 104-105, the vector field based on the discrete normal vector is a priori set as a periodic Gaussian process, utilizing the Poisson equation of the implicit surface defined by the vector field v. Reconstruct the implicit surface f(p) = 0 from point cloud data, where f is a function defined on a d-dimensional torus.

[0058] In one embodiment, setting the prior of the vector field generated based on the normal vector to follow a periodic Gaussian process prior distribution may include: setting the prior of the vector field generated based on the normal vector to follow a periodic Gaussian process prior distribution according to the following formula:

[0059] GP(0,k γ );

[0060]

[0061] Where, k γ For a stationary periodic kernel; k λ (p i p j (k) is a stationary periodic core γ The parsing form of p; i p jFor two distinct point cloud points in the original point cloud after filtering; |p i -p j |for p i p j The distance between them; κ is the hyperparameter of the stationary periodic kernel; λ is a half-integer.

[0062] In practice, the process of reconstructing the implicit surface first includes:

[0063] Assume a prior periodic Gaussian process vector field v: The vector field v generated from the normal vector follows the prior distribution GP(0,k) of a periodic Gaussian process. γ ), k γ It is a stationary periodic kernel, and its analytical form is: Stationary periodic kernel calculation p i ,p j The covariance between them, κ is the hyperparameter of the stationary periodic kernel, which controls the length scale of the kernel function.

[0064] Step 104 provides reasonable prior assumptions for the generation of the vector field, improving the accuracy of implicit surface reconstruction.

[0065] In one embodiment, step 105, reconstructing the implicit surface from the filtered original point cloud using the Poisson equation of the implicit surface defined by the prior vector field and the prior vector field, may include: expressing the prior vector field in Fourier series form using the spectral measure of a stationary periodic kernel, generating the component expansion of the prior vector field; generating the component expansion of the implicit surface using the Fourier domain-based Poisson equation of the implicit surface defined by the prior vector field and the Fourier series form of the vector field; determining the joint sampling result of the prior distribution based on the component expansion of the prior vector field and the component expansion of the implicit surface; obtaining the posterior distribution using the prior vector field through path condition sampling; and solving for the implicit surface of the filtered original point cloud using the relationship between the joint sampling result of the prior distribution and the posterior distribution.

[0066] In this embodiment, the a priori vector field is represented as a Fourier series using the spectral measure of a stationary periodic kernel, generating the component expansion of the a priori vector field. This can include: the generated component expansion of the a priori vector field is:

[0067]

[0068] Where, ω z,m,1 ω z,m,2 Let v be an independent and identically distributed Gaussian random variable; z (p i ) is the vector field v(p) iThe z-th component of ); p i These are the point cloud points in the original point cloud after filtering. is the spectral measure of the stationary periodic kernel, used to convert the stationary periodic kernel in the spatial domain into a frequency domain weight; m is the Fourier frequency index; κ is the hyperparameter of the stationary periodic kernel; <m,p i > is the dot product of the Fourier frequency index and the point cloud points; Z d Let m be the z-th component of the Fourier frequency index m with dimension d; d is the dimension; \0 indicates non-zero.

[0069] In this embodiment, the spectral measure of the corresponding stationary periodic kernel is calculated using the spectral decomposition property of the stationary periodic kernel through Fourier transform, as shown above. As shown in the formula, the spectral measure of the stationary periodic kernel is represented by the Fourier transform result of the stationary periodic kernel, which is used to convert the kernel in the spatial domain into a frequency domain weight.

[0070] The periodic Gaussian process can be expressed as a Fourier series using spectral measures; where any component of the vector field v is v0. z The expansion is as shown above. z (p i As shown in the formula, ω z,m,1 ω z,m,2 For the independent and identically distributed components to follow a Gaussian distribution, z is the z-th component of the vector field v. According to the spectral representation theorem for Gaussian processes, a stationary Gaussian process can be represented as the sum of randomly weighted sine and cosine functions.

[0071] In one embodiment, for step 105, generating the component expansion of the implicit surface using the Poisson equation based on the Fourier domain and the vector field in Fourier series form of the implicit surface defined by the vector field set a priori may include: the generated component expansion of the implicit surface is:

[0072]

[0073] Among them, f z (p i f(p) represents the z-th component of the implicit surface; i ) is an implicit surface; ω z,m,1 ω z,m,2 Let m be an independent and identically distributed Gaussian random variable; m is the Fourier frequency index; m z Z is the z-th component of the Fourier frequency index m; d p is the z-th component of the Fourier frequency index m of dimension d; d is the dimension; p i These are the point cloud points in the original point cloud after filtering. For spectral measures of stationary periodic nuclei; \0 indicates non-zero; <m,p i > is the dot product of the Fourier frequency index and the point cloud points.

[0074] Based on the Poisson equation in the Fourier domain and the expansion of the vector field, the corresponding implicit surface expansion representation is obtained, where the components f of the implicit surface are... z (p i The expansion of ) is shown in the formula above.

[0075] In one embodiment, the implicit surface of the filtered original point cloud is obtained by utilizing the relationship between the joint sampling results of the prior distribution and the posterior distribution. This can include obtaining the implicit surface of the filtered original point cloud according to the following formula:

[0076] f|v=f p +K fv (K vv +∑) -1 (yv p -ε);

[0077] Where f|v is the posterior conditional distribution between the vector field and the implicit surface; f p and v p The joint sampling result is the prior distribution; y is the normal vector; ε is the observation noise of the vector field; K fv K is the cross covariance of the vector field and the implicit surface; vv Let be the autocovariance of the vector field; ∑ be the noise covariance of the vector field.

[0078] The posterior distribution is obtained by using the normal vectors of discrete point cloud points through path conditional sampling. The implicit surface is solved by utilizing the relationship between the joint sampling results of the posterior and prior distributions, where y is the observed normal vector. This embodiment of the invention estimates the surface of the structure represented by the original point cloud by reconstructing the implicit surface, supplementing rich structural features and enhancing the semantic differences between different device components.

[0079] Step 105 allows for the accurate reconstruction of the implicit surface, supplementing the structural information of the original point cloud and improving the segmentation effect.

[0080] In one embodiment, step 106, adaptively sampling the point cloud data in the implicit surface to generate a dense point cloud, may include: meshing the implicit surface; calculating the area of ​​the implicit surface using the meshed implicit surface; repeating the following steps until the difference between the actual number of sampling points and a preset value is less than a preset threshold, and determining the actual sampling points as a dense point cloud: determining the number of actual sampling points obtained by adaptively sampling the point cloud data in the implicit surface according to the sampling interval radius based on the area; the sampling interval radius is initialized during the initial sampling; iteratively updating the sampling interval radius based on the difference between the actual number of sampling points and the preset value.

[0081] In one embodiment, determining the number of actual sampling points obtained by adaptively sampling the point cloud data in the implicit surface according to the sampling interval radius, based on the area, may include: determining the number of actual sampling points obtained by adaptively sampling the point cloud data in the implicit surface according to the sampling interval radius using the following formula:

[0082]

[0083] Where α is the attenuation factor; b is the actual number of sampling points obtained by adaptively sampling the point cloud data in the implicit surface according to the sampling interval radius r; r is the sampling interval radius; and S is the area of ​​the implicit surface.

[0084] In one embodiment, iteratively updating the sampling interval radius based on the difference between the actual number of sampling points and a preset value may include: iteratively updating the sampling interval radius according to the following formula:

[0085]

[0086] Where th is the preset percentage threshold; ...

[0087] Figure 3 This is a flowchart illustrating a specific example of the three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features, as described in this embodiment of the invention. Figure 3 As shown, in this embodiment, adaptive sampling of point cloud data in the implicit surface to generate a dense point cloud may include:

[0088] The implicit surface is meshed, and the area S of the implicit surface is calculated by accumulating the mesh surfaces.

[0089] Set the number of points B in the dense point cloud to be sampled; initialize the sampling interval radius;

[0090] Iterative execution:

[0091] The actual number of sampling points in the dense point cloud is sampled based on the area of ​​the structure formed by the implicit surface and the current sampling interval radius r.

[0092] Update the sampling interval radius based on the difference between the actual number of sampling points b and the set number of sampling points B;

[0093] Determine whether the actual number of sampling points and the set number of points B meet the set percentage threshold;

[0094] If so, generate a dense point cloud;

[0095] If not, return to the iteration.

[0096] By fine-tuning the sampling interval radius, the number of sampling points can be further optimized while maintaining a uniform point distribution.

[0097] Step 106 allows for the dynamic adjustment of the number of sampling points, ensuring a uniform distribution and meeting preset conditions. The sampling interval radius can be gradually adjusted to bring the actual number of sampling points close to the preset value, ensuring a uniform distribution and meeting requirements. This generates a dense point cloud rich in structural features, enhancing semantic differences between different targets and improving segmentation accuracy. Compared to the original point cloud, the sampled dense point cloud contains more smoothly estimated structural information. This estimated structural information enhances the semantic differences between different targets, especially smaller targets and those farther from the lidar. This compensates for the uneven density and incompleteness of the original lidar point cloud.

[0098] In one embodiment, for step 107, a pre-trained point cloud segmentation model is used to extract fusion point cloud features from the fused point cloud, and the fused point cloud features are used for segmentation prediction to generate a segmentation result. The fused point cloud is generated by fusing the original point cloud and the dense point cloud. The point cloud segmentation model extracts fusion point cloud features from the fused point cloud of the original point cloud and the dense point cloud, and the point cloud segmentation model uses the fused point cloud features for segmentation prediction to improve the original segmentation accuracy of the insulator string components.

[0099] In this embodiment, the pre-trained point cloud segmentation model can be a PointNet model or a PointNet++ model. Step 107 can fully utilize the structural information provided by the dense point cloud to improve the accuracy of the segmentation model.

[0100] In one embodiment, step 108, determining the final segmentation result based on the filtered original point cloud and the segmentation result, may include: taking the intersection of the filtered original point cloud and the segmentation result; and determining the result of the intersection as the final segmentation result. After the point cloud segmentation model outputs the segmentation result of the fused point cloud, the intersection of the original point cloud and the segmentation result is taken to obtain the final point cloud segmentation result. Step 108 ensures that the final segmentation result only contains the valid part of the original point cloud, improving the reliability of the segmentation result.

[0101] This invention provides a 3D intelligent segmentation method for insulator strings based on fused point cloud features. It utilizes a lidar system to acquire the original point cloud of power transmission equipment and, through steps such as filtering, normal vector calculation, implicit surface reconstruction, dense point cloud generation, and fused point cloud feature extraction, effectively solves the problems of sparsity, uneven density, and incomplete information inherent in traditional point cloud segmentation methods. Compared with existing technologies, this invention not only significantly improves the segmentation accuracy of small components of insulator strings but also enhances the semantic differences between different targets, especially for small targets and those far from the lidar. The dense point cloud generated through adaptive sampling supplements rich structural features, making the semantics of the point cloud segmentation model more explicit, thereby improving the overall segmentation effect.

[0102] This invention addresses the issues of sparse and incomplete point clouds caused by the integration of implicit surface sampling into dense point clouds. These dense point clouds are both concentrated and uniform, enhancing the structural and semantic information of the target. For point cloud segmentation models, this results in more explicit semantics, effectively improving the accuracy of the segmentation model. Results on two large-scale benchmark point cloud datasets, SemanticKITTI and nuScenes, which include point cloud data from power transmission equipment containing insulator strings, validate the effectiveness of this invention.

[0103] This invention also provides a three-dimensional intelligent segmentation device for insulator strings based on fused point cloud features, as described in the following embodiments. Since the principle behind this device is similar to the three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features, the implementation of this device can refer to the implementation of the three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features; repeated details will not be elaborated further.

[0104] Figure 4 This is a schematic diagram of a three-dimensional intelligent segmentation device for insulator strings based on fused point cloud features, as described in an embodiment of the present invention. Figure 4 As shown, the device includes:

[0105] Acquisition module 401 is used to acquire the raw point cloud of power transmission equipment using lidar; the power transmission equipment includes insulator strings.

[0106] The filtering module 402 is used to filter the original point cloud and generate a filtered original point cloud.

[0107] The normal vector determination module 403 is used to determine the normal vector of each point in the filtered original point cloud using a preset point cloud point normal vector algorithm.

[0108] The prior setting module 404 is used to set the prior of the vector field generated based on the normal vector to follow a periodic Gaussian process prior distribution.

[0109] Implicit surface reconstruction module 405 is used to reconstruct implicit surfaces from the filtered original point cloud using the Poisson equation of the implicit surface defined by the vector field set a priori and the vector field set a priori.

[0110] The dense point cloud generation module 406 is used to adaptively sample point cloud points in the implicit surface to generate a dense point cloud.

[0111] The segmentation result generation module 407 is used to extract fusion point cloud features from the fused point cloud using a pre-trained point cloud segmentation model, perform segmentation prediction using the fused point cloud features, and generate segmentation results; the fused point cloud is generated by fusing the original point cloud and the dense point cloud.

[0112] The final segmentation result determination module 408 is used to determine the final segmentation result based on the filtered original point cloud and the segmentation result.

[0113] In one embodiment, the preset point cloud point normal vector algorithm includes:

[0114] Using each point in the filtered original point cloud as the center, the neighborhood of each center is determined by a preset neighborhood radius;

[0115] The centroid of each neighborhood is determined by averaging the position information of each point cloud point in the neighborhood.

[0116] A covariance matrix is ​​generated based on the centroid of each neighborhood and the position information of each point cloud point in each neighborhood; the covariance matrix is ​​used to describe the distribution direction of each point cloud point relative to the centroid in each neighborhood.

[0117] Calculate the eigenvalues ​​and corresponding eigenvectors of the covariance matrix;

[0118] Compare the eigenvalues ​​to determine the smallest eigenvalue;

[0119] The eigenvector corresponding to the smallest eigenvalue is determined as the normal vector of the point cloud point.

[0120] In one embodiment, the prior setting module 404 is specifically used for:

[0121] The prior assumption is that the vector field generated based on the normal vector follows a periodic Gaussian process prior distribution as follows:

[0122] GP(0,k γ );

[0123]

[0124] Where, k γ For a stationary periodic kernel; k λ (p i pj (k) is a stationary periodic core γ The parsing form of p; i p j For two distinct point cloud points in the original point cloud after filtering; |p i -p j |for p i p j The distance between them; κ is the hyperparameter of the stationary periodic kernel; λ is a half-integer.

[0125] In one embodiment, the implicit surface reconstruction module 405 is specifically used for:

[0126] The vector field with prior settings is expressed as a Fourier series using the spectral measure of a stationary periodic kernel, generating the component expansion of the vector field with prior settings.

[0127] Using the Poisson equation based on the Fourier domain and the vector field in Fourier series form, which defines the implicit surface by the vector field defined by the prior setting, the component expansion of the implicit surface is generated.

[0128] Based on the component expansions of the vector field and the implicit surface after prior settings, the joint sampling results of the prior distribution are determined.

[0129] The posterior distribution is obtained by using a vector field with prior settings through path condition sampling;

[0130] By utilizing the relationship between the joint sampling results of the prior distribution and the posterior distribution, the implicit surface of the filtered original point cloud is obtained.

[0131] In one embodiment, the implicit surface reconstruction module 405 is specifically used for:

[0132] The component expansion of the generated vector field after prior settings is as follows:

[0133]

[0134] Where, ω z,m,1 ω z,m,2 Let v be an independent and identically distributed Gaussian random variable; z (p i ) is the vector field v(p) i The z-th component of ); p i These are the point cloud points in the original point cloud after filtering. is the spectral measure of the stationary periodic kernel, used to convert the stationary periodic kernel in the spatial domain into a frequency domain weight; m is the Fourier frequency index; κ is the hyperparameter of the stationary periodic kernel; <m,p i > is the dot product of the Fourier frequency index and the point cloud points; Z dLet m be the z-th component of the Fourier frequency index m with dimension d; d is the dimension; \0 indicates non-zero.

[0135] In one embodiment, the implicit surface reconstruction module 405 is specifically used for:

[0136] The component expansion of the generated implicit surface is:

[0137]

[0138] Among them, f z (p i f(p) represents the z-th component of the implicit surface; i ) is an implicit surface; ω z,m,1 ω z,m,2 Let m be an independent and identically distributed Gaussian random variable; m is the Fourier frequency index; m z Z is the z-th component of the Fourier frequency index m; d p is the z-th component of the Fourier frequency index m of dimension d; d is the dimension; p i These are the point cloud points in the original point cloud after filtering. For spectral measures of stationary periodic nuclei; \0 indicates non-zero; <m,p i > is the dot product of the Fourier frequency index and the point cloud points.

[0139] In one embodiment, the implicit surface reconstruction module 405 is specifically used for:

[0140] The implicit surface of the filtered original point cloud can be obtained by solving the following formula:

[0141] f|v=f p +K fv (K vv +∑) -1 (yv p -ε);

[0142] Where f|v is the posterior conditional distribution between the vector field and the implicit surface; f p and v p The joint sampling result is the prior distribution; y is the normal vector; ε is the observation noise of the vector field; K fv K is the cross covariance of the vector field and the implicit surface; vv Let be the autocovariance of the vector field; ∑ be the noise covariance of the vector field.

[0143] In one embodiment, the dense point cloud generation module 406 is specifically used for:

[0144] Mesh the implicit surface;

[0145] Calculate the area of ​​the implicit surface using the meshed implicit surface;

[0146] Repeat the following steps until the difference between the actual number of sampling points and the preset value is less than the preset threshold, then define the actual sampling points as a dense point cloud:

[0147] Based on the area, determine the actual number of sampling points obtained by adaptively sampling the point cloud data in the implicit surface according to the sampling interval radius; the sampling interval radius is the initial data during the initial sampling.

[0148] The sampling interval radius is iteratively updated based on the difference between the actual number of sampling points and the preset value.

[0149] In one embodiment, the dense point cloud generation module 406 is specifically used for:

[0150] The actual number of sampling points obtained by adaptively sampling the point cloud data in the implicit surface based on the sampling interval radius is determined according to the following formula:

[0151]

[0152] Where α is the attenuation factor; b is the actual number of sampling points obtained by adaptively sampling the point cloud data in the implicit surface according to the sampling interval radius r; r is the sampling interval radius; and S is the area of ​​the implicit surface.

[0153] In one embodiment, the dense point cloud generation module 406 is specifically used for:

[0154] The sampling interval radius is updated iteratively according to the following formula:

[0155]

[0156] Where th is the preset percentage threshold; ...

[0157] In one embodiment, the final segmentation result determination module 408 is specifically used for:

[0158] Take the intersection of the filtered original point cloud and the segmentation result;

[0159] The result of taking the intersection is determined as the final segmentation result.

[0160] Figure 5 This is a schematic diagram of a computer device in an embodiment of the present invention, such as... Figure 5As shown, this embodiment of the invention also provides a computer device 500, including a processor 510, a memory 520, and a computer program 530 stored in the memory 520 and executable on the processor 510. When the processor 510 executes the computer program 530, it implements the above-mentioned three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features.

[0161] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features.

[0162] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features.

[0163] In this embodiment of the invention, the original point cloud of a power transmission equipment, including insulator strings, is acquired using a lidar system. The original point cloud is filtered to generate a filtered original point cloud. A preset point cloud normal vector algorithm is used to determine the normal vector of each point in the filtered original point cloud. The vector field generated based on the normal vector is a priori set to follow a periodic Gaussian process prior distribution. The implicit surface is reconstructed from the filtered original point cloud using the Poisson equation of the implicit surface defined by the prior-set vector field and the prior-set vector field. Adaptive sampling is performed on the point cloud points in the implicit surface to generate a dense point cloud. A pre-trained point cloud segmentation model is used to extract fusion point cloud features from the fused point cloud, and segmentation prediction is performed using the fused point cloud features to generate a segmentation result. The fused point cloud is generated by fusing the original point cloud and the dense point cloud. The final segmentation result is determined based on the filtered original point cloud and the segmentation result. Compared with existing technologies, this invention can improve the efficiency, accuracy, and robustness of point cloud segmentation for small components of insulator strings.

[0164] The beneficial effects of the embodiments of the present invention are as follows:

[0165] This invention addresses the issues of sparse and incomplete point clouds caused by the integration of implicit surface sampling into dense point clouds. These dense point clouds are both concentrated and uniform, enhancing the structural and semantic information of the target. For point cloud segmentation models, this results in more explicit semantics, effectively improving the accuracy of the segmentation model. Results on two large-scale benchmark point cloud datasets, SemanticKITTI and nuScenes, which include point cloud data from power transmission equipment containing insulator strings, validate the effectiveness of this invention.

[0166] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0168] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0169] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0170] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features, characterized in that, include: The raw point cloud of the power transmission equipment is collected using lidar; the power transmission equipment includes insulator strings. Filter the original point cloud to generate a filtered original point cloud; The normal vector of each point in the filtered original point cloud is determined using a preset point cloud point normal vector algorithm. The prior of the vector field generated based on the normal vector is set to follow a periodic Gaussian process prior distribution. Using the Poisson equation of the implicit surface defined by the vector field with prior settings and the vector field with prior settings, the implicit surface is reconstructed from the filtered original point cloud; Adaptive sampling is performed on the point cloud points in the implicit surface to generate a dense point cloud; The fused point cloud features are extracted using a pre-trained point cloud segmentation model, and segmentation prediction is performed using the fused point cloud features to generate segmentation results; the fused point cloud is generated by fusing the original point cloud and the dense point cloud. Based on the filtered original point cloud and the segmentation results, the final segmentation result is determined; Using the Poisson equation of the implicit surface defined by the prior vector field and the prior vector field, the implicit surface is reconstructed from the filtered original point cloud, including: The vector field with prior settings is expressed as a Fourier series using the spectral measure of a stationary periodic kernel, generating the component expansion of the vector field with prior settings. Using the Poisson equation based on the Fourier domain and the vector field in Fourier series form, which defines the implicit surface by the vector field defined by the prior setting, the component expansion of the implicit surface is generated. Based on the component expansions of the vector field and the implicit surface after prior settings, the joint sampling results of the prior distribution are determined. The posterior distribution is obtained by using a vector field with prior settings through path condition sampling; By utilizing the relationship between the joint sampling results of the prior distribution and the posterior distribution, the implicit surface of the filtered original point cloud is obtained.

2. The method as described in claim 1, characterized in that, The preset point cloud point normal vector algorithm includes: Using each point in the filtered original point cloud as the center, the neighborhood of each center is determined by a preset neighborhood radius; The centroid of each neighborhood is determined by averaging the position information of each point cloud point in the neighborhood. A covariance matrix is ​​generated based on the centroid of each neighborhood and the position information of each point cloud point in each neighborhood; the covariance matrix is ​​used to describe the distribution direction of each point cloud point in each neighborhood relative to the centroid. Calculate the eigenvalues ​​and corresponding eigenvectors of the covariance matrix; Compare the eigenvalues ​​to determine the smallest eigenvalue; The eigenvector corresponding to the smallest eigenvalue is determined as the normal vector of the point cloud point.

3. The method as described in claim 1, characterized in that, Setting the prior of the vector field generated based on the normal vector to follow a periodic Gaussian process prior distribution includes: The prior assumption is that the vector field generated based on the normal vector follows a periodic Gaussian process prior distribution as follows: ； ； in, It is a stationary periodic core; For a stable periodic core The parsing form; These are two different point cloud points in the original point cloud after filtering. for The distance between them; λ is the hyperparameter of the stationary periodic kernel; λ is a half-integer.

4. The method as described in claim 3, characterized in that, The a priori vector field is expressed in Fourier series form using the spectral measure of a stationary periodic kernel, generating the component expansions of the a priori vector field, including: The component expansion of the generated vector field after prior settings is as follows: ； ； in, They are independent and identically distributed Gaussian random variables; For vector fields The z-th component; These are the point cloud points in the original point cloud after filtering. is the spectral measure of the stationary periodic kernel, used to convert the stationary periodic kernel in the spatial domain into a frequency domain weight; m is the Fourier frequency index; Hyperparameters for stationary periodic kernels; The dot product of the Fourier frequency index and the point cloud points; Let m be the z-th component of the Fourier frequency index m with dimension d; d is the dimension; \0 indicates non-zero.

5. The method as described in claim 4, characterized in that, Using the Poisson equation in the Fourier domain and the vector field in Fourier series form, which defines the implicit surface by a priori vector field, the component expansions of the implicit surface are generated, including: The component expansion of the generated implicit surface is: ； in, Let z be the z-th component of the implicit surface; It is an implicit surface; is an independent and identically distributed Gaussian random variable; m is the Fourier frequency index; The z-th component of the Fourier frequency index m; Let m be the z-th component of the Fourier frequency index m with dimension d; where d is the dimension. These are the point cloud points in the original point cloud after filtering. For spectral measures of stationary periodic nuclei; \0 indicates non-zero; This is the dot product of the Fourier frequency index and the point cloud points.

6. The method according to claim 5, characterized in that, By utilizing the joint sampling results of the prior distributions and the relationship between the posterior distributions, the implicit surface of the filtered original point cloud is obtained, including: The implicit surface of the filtered original point cloud can be obtained by solving the following formula: ； in, Let be the posterior conditional distribution between the vector field and the implicit surface; and The result is the joint sampling of the prior distribution; y is the normal vector; For the observation noise of the vector field; Let be the cross-covariance between the vector field and the implicit surface; Let be the autocovariance of the vector field; Let be the noise covariance of the vector field.

7. The method according to claim 1, characterized in that, Adaptive sampling of point cloud data in the implicit surface to generate a dense point cloud includes: Mesh the implicit surface; Calculate the area of ​​the implicit surface using the meshed implicit surface; Repeat the following steps until the difference between the actual number of sampling points and the preset value is less than the preset threshold, then define the actual sampling points as a dense point cloud: Based on the area, determine the actual number of sampling points obtained by adaptively sampling the point cloud data in the implicit surface according to the sampling interval radius; the sampling interval radius is the initial data during the initial sampling. The sampling interval radius is iteratively updated based on the difference between the actual number of sampling points and the preset value.

8. The method according to claim 7, characterized in that, Based on the area, determine the number of actual sampling points obtained by adaptively sampling the point cloud data in the implicit surface according to the sampling interval radius, including: The actual number of sampling points obtained by adaptively sampling the point cloud data in the implicit surface based on the sampling interval radius is determined according to the following formula: ； in, It is the attenuation factor; S represents the actual number of sampling points obtained by adaptively sampling the point cloud data in the implicit surface according to the sampling interval radius r; r is the sampling interval radius; and S is the area of ​​the implicit surface.

9. The method according to claim 7, characterized in that, The sampling interval radius is iteratively updated based on the difference between the actual number of sampling points and the preset value, including: The sampling interval radius is iteratively updated according to the following formula: ； Where th is the preset percentage threshold; , denoted by two preset scaling factors; r is the sampling interval radius; r' is the sampling interval radius after iterative update; b is the actual number of sampling points obtained by adaptively sampling the point cloud data in the implicit surface according to the sampling interval radius; and B is a preset value.

10. The method as described in claim 1, characterized in that, Based on the filtered original point cloud and the segmentation results, the final segmentation result is determined, including: Take the intersection of the filtered original point cloud and the segmentation result; The result of taking the intersection is determined as the final segmentation result.

11. A three-dimensional intelligent segmentation device for insulator strings based on fused point cloud features, characterized in that, include: A data acquisition module is used to acquire raw point clouds of power transmission equipment using lidar; the power transmission equipment includes insulator strings. The filtering module is used to filter the original point cloud and generate a filtered original point cloud. The normal vector determination module is used to determine the normal vector of each point in the filtered original point cloud using a preset point cloud point normal vector algorithm. The prior setting module is used to set the prior of the vector field generated based on the normal vector to follow a periodic Gaussian process prior distribution. The implicit surface reconstruction module is used to reconstruct implicit surfaces from the filtered original point cloud using the Poisson equation of the implicit surface defined by the vector field set a priori and the vector field set a priori. The dense point cloud generation module is used to adaptively sample the point cloud points in the implicit surface to generate a dense point cloud. The segmentation result generation module is used to extract fused point cloud features from the fused point cloud using a pre-trained point cloud segmentation model, perform segmentation prediction using the fused point cloud features, and generate segmentation results; the fused point cloud is generated by fusing the original point cloud and the dense point cloud. The final segmentation result determination module is used to determine the final segmentation result based on the filtered original point cloud and the segmentation result. The implicit surface reconstruction module is specifically used for: The vector field with prior settings is expressed as a Fourier series using the spectral measure of a stationary periodic kernel, generating the component expansion of the vector field with prior settings. Using the Poisson equation based on the Fourier domain and the vector field in Fourier series form, which defines the implicit surface by the vector field defined by the prior setting, the component expansion of the implicit surface is generated. Based on the component expansions of the vector field and the implicit surface after prior settings, the joint sampling results of the prior distribution are determined. The posterior distribution is obtained by using a vector field with prior settings through path condition sampling; By utilizing the relationship between the joint sampling results of the prior distribution and the posterior distribution, the implicit surface of the filtered original point cloud is obtained.

12. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features as described in any one of claims 1-10.

14. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the three-dimensional intelligent segmentation method for insulator strings based on fused point cloud features as described in any one of claims 1-10.

Images