A method, system and device for fitting geometric primitives of three-dimensional point cloud data

The feature extraction network and weighted least squares method constructed by PointConv solve the problems of manual parameter adjustment and noise sensitivity in primitive fitting of three-dimensional point cloud data, and realize efficient and accurate geometric primitive parameter prediction.

CN118537564BActive Publication Date: 2024-11-08SOUTH CENTRAL UNIVERSITY FOR NATIONALITIES
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411000570.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-08
Estimated Expiration
2044-07-25

AI Technical Summary

Technical Problem

The prior art has problems such as manual parameter adjustment dependence, poor large-scale data processing and noise sensitivity in the fitting of three-dimensional point cloud data primitives. The PointNet++-based method has insufficient accuracy.

Method used

The geometric primitive fitting method of three-dimensional point cloud data based on PointConv, including preprocessing, feature extraction, segmentation and classification, build a feature extraction network through PointConv, and use the weighted least squares algorithm to improve the prediction accuracy of primitive parameters.

Benefits of technology

Effectively remove noise interference, improve the feature recognition ability of large-scale point cloud data, enhance the accuracy and efficiency of point cloud data processing, and improve the accuracy of geometric primitive parameter prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118537564B_ABST
    Figure CN118537564B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, system and device for fitting geometric primitives of three-dimensional point cloud data. First, the three-dimensional point cloud data is preprocessed, including noise reduction, smoothing and downsampling processing; then a three-dimensional point cloud feature extraction network is used to extract three-dimensional point cloud features; then the three-dimensional point cloud data is segmented and classified; first, the three-dimensional point cloud data is segmented into several subsets, and then each subset is classified to determine the most matching geometric primitive type; finally, specific mathematical parameters are predicted for the matching geometric primitives. The present invention realizes high-precision point cloud segmentation and classification, effectively improves the feature recognition and extraction capabilities for large-scale point cloud data, and further enhances the accuracy and efficiency of point cloud data processing and analysis. The present invention introduces a weight mechanism on the basis of the traditional least squares method, which effectively improves the accuracy of the prediction of geometric primitives, such as planes, spheres and other geometric body parameters.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer graphics, and relates to a method, system and device for fitting geometric primitives of three-dimensional point cloud data, and in particular to a method, system and device for fitting geometric primitives based on PointConv three-dimensional point cloud feature extraction and least squares method. Background Art

[0002] The significant improvement of 3D scanning technology and the enhancement of data storage capacity play a vital role in the field of computer vision, providing solid technical support for the processing and analysis of 3D geometric data. However, the data obtained by scanning devices such as LiDAR often appear in the form of primary point clouds, which lacks the intuitive expression of object structure and semantic information, thus limiting the in-depth understanding and accurate reproduction of object structure and semantics. Especially when it comes to complex tasks such as advanced editing and model fitting, it is often difficult to achieve the desired processing effect due to the lack of clearly defined geometric primitives and corresponding parameters. Therefore, researchers have been committed to exploring how to transform such low-level data into a structured and highly semantic representation. This conversion process plays an important role in the field of computer vision and is a key link in achieving accurate digitization of physical objects. Taking computer-aided design (CAD) as an example, the constructive solid geometry (CSG) it relies on is a concrete manifestation of this conversion demand. By constructing a structured model with clear geometric primitives and parameters, a more solid foundation is provided for the application of computer vision in fields such as object recognition and scene understanding.

[0003] By accurately mapping 3D scan data to geometric primitives, the ability of models in computer vision systems to express complex design intent can be greatly improved, while optimizing and enhancing the accurate reconstruction of actual object shapes in reverse engineering applications. Currently, point cloud primitive fitting methods are mainly divided into two categories: traditional algorithms based on random sampling consensus (RANSAC) and methods based on deep learning.

[0004] In the paper "Efficient RANSAC for Point-Cloud Shape Detection", R. Schnabel et al. proposed an algorithm based on random sampling and detection of planes, spheres, cylinders, cones, and tori. The algorithm detects basic shapes in point clouds without human intervention by automatically decomposing point clouds into a concise, intrinsically shaped mixed structure and a set of residual points. It can handle large-scale point cloud data and adapt to different shapes and sizes. It can also remain robust in the presence of many outliers and high noise in the data. However, the current algorithm can only detect a limited number of basic shapes and cannot cope with other complex shapes. Due to the non-full coverage detection of point clouds, it cannot guarantee that shape agents can be found for every part of the surface, which may cause some characteristics of the data to be ignored.

[0005] In the paper "GlobFit: consistently fitting primitives by discovering global relations", Li et al. proposed an algorithm based on RANSAC to simultaneously recover a set of locally fitted primitives and their global relationships from noisy and incomplete point clouds by directly coupling local and global data. Specifically, a RANSAC-based method is used to locally fit the input point set to identify primitives (such as planes, spheres, etc.), and then the local fitting results are extended to the global context by considering the global relations between these primitives (for example, their relative position, orientation, and equality), and finally gradually derived and enforced through an iterative learning process. This method effectively overcomes the challenge of recovering the structure of artificial objects from noisy and incomplete data, especially for those objects composed of relatively simple repetitive primitives. However, it does not work well for irregular complex objects and cannot solve the problem of consistent segmentation with noisy input.

[0006] In the article "Primitive Fitting Based on the Efficient multiBaySAC Algorithm", Kang et al. proposed a multi-BaySAC (Bayesian sample consistency) algorithm based on conditional sampling for matching multiple primitives in unorganized three-dimensional point clouds. First, a statistical test algorithm for candidate model parameters is used to detect potential multiple primitives, and then each data point is assigned to multiple prior inlier probabilities of the initial multiple primitives, and a parallel strategy rather than a sequential strategy is used to optimize the detected primitives. Although it shows good performance in primitive fitting, this method is too dependent on accurate prior probability estimation and statistical testing of model parameters, and requires careful manual adjustment of algorithm parameters to obtain optimal performance. In addition, the performance is highly dependent on specific application scenarios and data characteristics, so it is not universal and easy to use.

[0007] In the paper "Matching with PROSAC - progressive sample consensus", Chum et al. proposed a progressive sample consensus algorithm that is an improvement on RANSAC. The algorithm uses a linear ordering defined by a similarity function used when establishing temporary correspondences. Unlike the RANSAC algorithm, which treats all correspondences equally and uniformly randomly extracts samples from the entire set, this algorithm preferentially extracts samples from point sets with high-similarity correspondences. Although the matching speed and adaptability of this method are better than the RANSAC algorithm, the progressive sample consensus algorithm is highly dependent on the accuracy of the similarity function. If the similarity assessment is inaccurate, it may lead to a decrease in efficiency and effectiveness, and requires manual setting of specific parameters and optimization. Although these traditional methods have achieved remarkable results in their application fields, their performance is often limited by the complicated parameter tuning work.

[0008] In the primitive deep learning primitive fitting method, Smirnov et al. proposed a framework for predicting parametric shape primitives through deep learning in the article "Deep parametric shape predictions using distance fields". It converts between pixel grids and input data through distance fields, and abstracts by using surface primitives instead of curves, thereby outputting each input parametric primitive. It can effectively handle two-dimensional and three-dimensional tasks, such as font vectorization and surface abstraction, showing its wide application potential. However, this method is highly dependent on a large amount of high-quality training data and can only predict surface and cube primitives.

[0009] In the paper "Modeling 3D shapes by reinforcement learning", Lin et al. proposed a method to edit primitives to adapt to target shapes based on reinforcement learning. In the paper "Learning 3D shape parsing beyond cuboids", Paschalidou et al. proposed a solution based on superquadric geometric primitives. In the paper "Learning generative models of shape handles", Gadelha et al. proposed a generative model that builds a compact 3D representation by predicting the parameters of cuboid and spherical meshes; in the paper "CvxNet: Learnable convex decomposition", Deng et al. studied a 3D representation learning network that recursively subdivides the space to obtain a set of convex shapes; in the paper "Local deep implicit functions for 3d shape", Genova et al. used implicit functions to express local shape elements. However, the types of primitives used in these studies are relatively limited, and most of the work focuses on abstracting the input shape with rough primitive fitting.

[0010] In "Supervised Fitting of Geometric Primitives to 3D PointClouds", Li et al. proposed a SPFN network based on PointNet++ point cloud feature extraction, which can automatically calculate the relevant information from each point to the primitive and its type from the point cloud data, and then infer the primitive parameters that minimize the fitting error, significantly reducing the reliance on manual parameter fine-tuning and pre-processing steps. Thanks to the powerful performance of deep networks, SPFN has the ability to extract features from complex data and handle complex scenes that traditional algorithms have difficulty coping with to more accurately fit a variety of primitives including planes, spheres, cylinders, and cones. Sharma et al. further extended this method to adapt to B-spline surface splicing in the paper "ParSeNet A ParametricSurface Fitting Network for 3D Point Clouds"; Saporta et al. made unsupervised recursive structural transformation to this framework in "Unsupervised recursive deep fitting of 3D primitives to points"; and Tuan et al. proposed a cascaded structural network that combines local and global characteristics for SPFN in the paper "CPFN: CascadedPrimitive Fitting Networks for High-Resolution Point Clouds", so that primitives of various sizes can be matched to high-resolution point clouds. Although these methods have their own effectiveness, most of them use the existing network architecture PointNet++ to encode the input point cloud, which is limited by the scale of the point cloud and the ability to extract point cloud features. Summary of the invention

[0011] The present invention aims at the defects of the prior art in primitive fitting, such as the traditional primitive fitting algorithm is too dependent on manual parameter adjustment, it is difficult to effectively process large-scale point cloud data, and it is too sensitive to noise; at the same time, based on PointNet++ and its variant neural networks, there is a problem of insufficient accuracy when performing primitive fitting. A geometric primitive fitting method, system and device are proposed to preprocess point cloud data, extract features, segment, classify, and predict primitive parameters, thereby realizing geometric primitive fitting.

[0012] The technical solution adopted by the method of the present invention is: a three-dimensional point cloud data geometric primitive fitting method based on PointConv, comprising the following steps:

[0013] Step 1: Preprocess the 3D point cloud data, including noise reduction, smoothing and downsampling;

[0014] Step 2: Use a 3D point cloud feature extraction network to extract 3D point cloud features;

[0015] The three-dimensional point cloud feature extraction network includes a feature encoding module and a feature decoding module;

[0016] The feature encoding module includes three first combination layers connected in sequence and a transformation network, wherein the first combination layer is composed of a sampling layer, a grouping layer, and a PointConv convolution kernel; the PointConv convolution kernel includes three convolution layers connected in sequence, a maximum pooling layer, three fully connected layers, and a PointConv layer; a batch normalization layer and a ReLU activation layer are arranged after the first convolution layer, the second convolution layer, the first fully connected layer, and the second fully connected layer; then a fully connected layer outputs a transformation matrix for rigidly transforming and aligning features for each point in the three-dimensional point cloud data; the transformation network includes a feature transformation layer and a one-dimensional convolution layer, wherein the feature transformation layer calculates a density scale, then multiplies the local features by the density scale to obtain weighted local features, and then further processes these features through a one-dimensional convolution layer, and finally outputs an encoded feature matrix;

[0017] The feature decoding module includes three sequentially connected second combination layers, which recover the detailed features of the point cloud from the feature representation compressed in the encoding process; the second combination layer is composed of an interpolation operation layer and a PoinConv layer, and each layer of the second combination layer receives the previous layer data and the skip link data of the corresponding first combination layer in the feature encoding module, and outputs local features and global features;

[0018] Step 3: Segment and classify the 3D point cloud data;

[0019] First, the 3D point cloud data is segmented into several subsets, and then each subset is classified to determine the most matching geometric primitive type;

[0020] Step 4: For the geometric primitives matched in step 3, predict specific mathematical parameters.

[0021] Preferably, in step 1, the noise reduction process is to perform statistical filtering on the three-dimensional point cloud data; k The average distance of the nearest neighbor points is calculated, and the global standard deviation is obtained based on the average distance of all points. The threshold is set as the average of the average distance of each point to its neighbor points plus a number of standard deviations, outliers with an average distance greater than the threshold are removed, and the remaining points are retained as the filtered point cloud data.

[0022] Preferably, the smoothing process in step 1 is to perform Gaussian filtering on the three-dimensional point cloud data. For each point P in the point cloud obtained after the noise reduction process, its neighborhood is defined, the distance from each point in the neighborhood to the center point is calculated, and the weight is calculated based on the distance using a Gaussian function. Using the calculated weight, a new position is calculated for each point, and the new position is the weighted average position of all points in the neighborhood.

[0023] Preferably, in step 1, the smoothing process is performed for each point , m is the size of the 3D point cloud dataset, Each point in the neighborhood ,calculate and The Euclidean distance between , using Gaussian function to calculate weights ;in, is the standard deviation of the Gaussian function, which controls the speed at which the weight changes with distance. Calculate weighted position ; Finally calculate the new position after smoothing .

[0024] Preferably, in step 1, the downsampling process is to downsample the three-dimensional point cloud data; for the point cloud obtained after smoothing, a greedy algorithm is used for sampling, and points in the point cloud are selected in an iterative manner to cover the entire point cloud to the greatest extent; in each iteration, the point farthest from the currently selected point set is selected; the initial point is randomly selected, and then the algorithm is performed in an iterative manner until a preset number of points are collected.

[0025] Preferably, in step 2, the interpolation layer and the interpolation operation satisfy the formula ,in, q For any query point, Indicates i The weight of the nearest neighbor point. The larger the weight, the greater the influence of the neighbor point on the estimated value. , express q With i The distance to the nearest neighbor, p is a constant greater than 0, used to control how distance affects the weight; Indicates i nearest neighbor points; ε Represents a very small positive number, which is used to prevent the sum of the weights from being 0 and causing the denominator to be zero; k is the number of nearest neighbors.

[0026] Preferably, in step 3, a three-dimensional point cloud data segmentation network is used to perform a detailed analysis of the point cloud, wherein each point is regarded as an independent unit and is assigned a corresponding category label; in the high-dimensional feature space extracted by the feature extraction network, a neighbor aggregation operation is performed within the spatial range to enhance the descriptiveness of each point feature; then, the spatial information lost in the downsampling process is restored through interpolation and feature propagation; finally, the enhanced features are passed through one or more fully connected layers, combined with activation functions and classifiers for point-by-point classification to obtain multiple point cloud fragments;

[0027] The three-dimensional point cloud data segmentation network is a deep neural network, and its segmentation network relaxes the intersection-and-union ratio Defined as:

[0028] ;

[0029] in, is the preset truth label matrix, is the prediction matrix, N is the total number of points; , representing different geometric primitives, K = 0 is a plane, K =1 for a sphere, K =2 is a cylinder, K =3 is a cone.

[0030] Preferably, in step 3, during the classification process, a three-dimensional point cloud data classification network is used to convert the features of all points in the point cloud fragment into a single high-dimensional vector through a global pooling operation. All high-dimensional vectors together form a global high-dimensional vector, thereby capturing global context information; the global high-dimensional vector is then input into a series of fully connected layers to capture the complex relationships and patterns between different features; finally, the Softmax activation function is used to map the output of the fully connected layer to the category probability distribution, thereby completing the global classification of the point cloud fragment.

[0031] The technical solution adopted by the system of the present invention is: a three-dimensional point cloud data geometric primitive fitting system based on PointConv, including the following modules:

[0032] A 3D point cloud data preprocessing module is used to preprocess the 3D point cloud data, including noise reduction, smoothing and downsampling processing;

[0033] A three-dimensional point cloud feature extraction module is used to extract three-dimensional point cloud features using a three-dimensional point cloud feature extraction network;

[0034] The three-dimensional point cloud feature extraction network includes a feature encoding module and a feature decoding module;

[0035] The feature encoding module includes three first combination layers connected in sequence and a transformation network, wherein the first combination layer is composed of a sampling layer, a grouping layer, and a PointConv convolution kernel; the PointConv convolution kernel includes three convolution layers connected in sequence, a maximum pooling layer, three fully connected layers, and a PointConv layer; a batch normalization layer and a ReLU activation layer are arranged after the first convolution layer, the second convolution layer, the first fully connected layer, and the second fully connected layer; then a fully connected layer outputs a transformation matrix for rigidly transforming and aligning features for each point in the point cloud; the transformation network includes a feature transformation layer and a one-dimensional convolution layer, wherein the feature transformation layer calculates a density scale, then multiplies the local features by the density scale to obtain weighted local features, and then further processes these features through a one-dimensional convolution layer with a final number of convolution kernels of 512, and finally outputs an encoded feature matrix.

[0036] The feature decoding module includes three second combination layers connected in sequence, which recover the detailed features of the point cloud from the feature representation compressed in the encoding process; the second combination layer is composed of an interpolation operation layer and a PoinConv layer, and the second combination layer receives the previous layer data and the three first combination layer jump link data in the feature encoding module respectively, and outputs local features and global features;

[0037] A 3D point cloud data segmentation and classification module is used to segment and classify 3D point cloud data;

[0038] First, the 3D point cloud data is segmented into several subsets, and then each subset is classified to determine the most matching geometric primitive type;

[0039] The geometric primitive mathematical parameter prediction module is used to predict specific mathematical parameters for the matched geometric primitives.

[0040] The technical solution adopted by the device of the present invention is: a three-dimensional point cloud data geometric primitive fitting device based on PointConv, comprising:

[0041] one or more processors;

[0042] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the three-dimensional point cloud data geometric primitive fitting method based on PointConv.

[0043] Compared with the prior art, the beneficial effects of the present invention include:

[0044] (1) The present invention proposes a 3D point cloud preprocessing method, which effectively removes various noise interferences and abnormal data points in the original point cloud data through systematic noise reduction, smoothing and precise downsampling technology. Not only can the noise points be accurately identified and removed, but the integrity and consistency of the data can also be maintained. After smoothing, the jagged phenomenon and irregular edges are effectively eliminated, making the point cloud smoother and more natural. Such processing makes the point cloud data closer to the actual situation of the physical object and lays a solid foundation for the subsequent processing steps.

[0045] (2) The present invention constructs a new feature extraction network based on PointConv to achieve high-precision point cloud segmentation and classification, effectively improving the feature recognition and extraction capabilities for large-scale point cloud data, and further enhancing the accuracy and efficiency of point cloud data processing and analysis.

[0046] (3) The present invention adopts a weighted least squares algorithm and introduces a weight mechanism on the basis of the traditional least squares method, which effectively improves the accuracy of the prediction of geometric primitives, such as planes, spheres and other geometric body parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The technical solution of the present invention is further described below using embodiments and specific implementation methods. In addition, in the process of describing the technical solution, some drawings are also used. For those skilled in the art, other drawings and the intention of the present invention can also be obtained based on these drawings without paying creative work.

[0048] Figure 1 This is a schematic diagram of the method principle of an embodiment of the present invention;

[0049] Figure 2 is a preprocessing flow chart of an embodiment of the present invention;

[0050] Figure 3 This is a point cloud effect diagram after preprocessing in an embodiment of the present invention;

[0051] Figure 4 It is a structural diagram of a feature extraction module according to an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of geometric primitives according to an embodiment of the present invention;

[0053] Figure 6 This is a point cloud effect diagram after segmentation and classification processing according to an embodiment of the present invention;

[0054] Figure 7 This is a diagram showing the effect of the embodiment of the present invention on point cloud segmentation and classification compared with other geometric primitive fitting methods;

[0055] Figure 8 This is a point cloud primitive fitting effect diagram of an embodiment of the present invention;

[0056] Fig. 9 This is a diagram showing the effect of fitting plane primitive parameters according to an embodiment of the present invention;

[0057] Fig.10 This is a diagram showing the effect of fitting spherical primitive parameters according to an embodiment of the present invention;

[0058] Fig.11 This is a diagram showing the effect of fitting cylinder parameters according to an embodiment of the present invention;

[0059] Fig.12 This is a diagram showing the effect of fitting cone parameters according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0061] Please see Figure 1 The present embodiment provides a method for fitting geometric primitives of three-dimensional point cloud data based on PointConv, comprising the following steps:

[0062] Step 1: Preprocess the 3D point cloud data, including noise reduction, smoothing and downsampling;

[0063] In one embodiment, see Figure 2 The preprocessing first reduces isolated points and outliers in the three-dimensional point cloud dataset. Perform statistical filtering. n is the size of the 3D point dataset. Select a dimension in the 3D space, for example, x dimension, and then sort all the points according to the value of this dimension. Find the median point after sorting, and this point will be used as kd The root node of the tree makes the coordinates of the points on its left smaller than it in the current dimension, and the coordinates of the points on its right greater than or equal to it, dividing the point set into two parts. The point set on the left side of the median point constitutes the left subtree, and the point set on the right side constitutes the right subtree. Then, select the next dimension (for example, the current dimension is x , the next dimension will be y ). Repeat the above steps for the left and right sub-point sets respectively, find the median point of each sub-point set in the new dimension, and use the median point as the internal node of the tree. Recursively repeat this operation, following the strategy of rotating dimensions each time, until there is only one point left in the point set or no point. After completing the above steps, you will get a kd tree, and then use this tree to efficiently find each point ofK Using depth-first search with heuristic pruning, we start from the root node in the tree and recursively search to find the nearest neighbor. We calculate the distance between each point and its nearest neighbor. K The average distance of the nearest neighbors Next, we calculate the average distance of the entire point cloud. and standard deviation . This average is the average distance of all points The average of , and the standard deviation is the standard deviation of these average distances, describing the variability of the distances between a point and its neighbors. Points, get a new point set .

[0064] Then, the 3D point cloud is smoothed to reduce the noise caused by sensor errors, environmental interference or other factors. , m is the size of the 3D point cloud dataset, Each point in the neighborhood ,calculate and The Euclidean distance between , through the Euclidean distance Sure right The influence of . The weight is calculated using the Gaussian function ;in, is the standard deviation of the Gaussian function, which controls the speed at which the weight changes with distance. Calculate weighted position ; Finally calculate the new position after smoothing . In this way, click It is updated to be a weighted average position based on its neighboring points.

[0065] Finally, the 3D point cloud is downsampled because the number of data points in the high-resolution point cloud is large. Direct processing not only increases the computational complexity but also may introduce redundant information. S is an empty set, and the point cloud set is obtained after the previous steps , select any point as a starting point and add to S . Initialize a distance list D , which contains all points relative to the set S The maximum distance between the midpoints. Initially, D Each value in is equal to The distance to the corresponding point. Loop through the following steps until the point set S to the desired size;

[0066] The specific steps include:

[0067] (1) From the remaining point cloud (i.e., not in S Find the distance set S The farthest point in the , which can be done by looking up D The maximum value in is achieved quickly.

[0068] (2) Find the farthest point Add to Point Set S middle.

[0069] (3) Update distance list D For each distance value in the list , calculation point With new join point The distance between .if , then use renew .

[0070] This guarantees D Always save every unselected point to S The distance to the closest point.

[0071] After the cycle is completed, the final preprocessed three-dimensional point cloud data is obtained.

[0072] Please see Figure 3 , which is the effect diagram after preprocessing the point cloud data. It can be seen that the quality of the point cloud is effectively improved.

[0073] Step 2: Use a 3D point cloud feature extraction network to extract 3D point cloud features;

[0074] In one embodiment, see Figure 4 , the three-dimensional point cloud feature extraction network includes a feature encoding module and a feature decoding module;

[0075] The feature encoding module includes three first combination layers connected in sequence and a transformation network connected in sequence, wherein the first combination layer is composed of a sampling layer, a grouping layer, and a PointConv convolution kernel; the PointConv convolution kernel includes three convolution layers connected in sequence, a maximum pooling layer, three fully connected layers, and a PointConv layer; a batch normalization layer and a ReLU activation layer are arranged after the first convolution layer, the second convolution layer, the first fully connected layer, and the second fully connected layer; then a fully connected layer outputs a transformation matrix for rigidly transforming and aligning features for each point in the point cloud; the transformation network includes a feature transformation layer and a one-dimensional convolution layer, wherein the feature transformation layer calculates a density scale, then multiplies the local features by the density scale to obtain weighted local features, and then further processes these features through a one-dimensional convolution layer with a final number of convolution kernels of 512, and finally outputs an encoded feature matrix.

[0076] The feature decoding module includes three second combination layers connected in sequence, which recover the detailed features of the point cloud from the feature representation compressed in the encoding process; the second combination layer is composed of an interpolation operation layer and a PoinConv layer, and the second combination layer receives the previous layer data and the three first combination layer jump link data in the feature encoding module respectively, and outputs local features and global features;

[0077] In the feature encoding module, point cloud data After three combination layers, three feature matrices are output respectively, and the feature matrix sizes are ,in Indicates the number of points, Represents the feature dimension. The sampling layer and grouping adopt a hierarchical strategy to iteratively sample and group the point cloud, select key points by far-point sampling, and then use the spherical neighborhood to determine the local neighborhood group of each key point, and perform independent feature learning on these local area groups. The PointConv convolution kernel includes three convolution layers, a maximum pooling layer, three fully connected layers and a PointConv layer. The first layer is a one-dimensional convolution layer with 64 convolution kernels and a step size of 1. The features of each point are determined by its x , y , zThe coordinates are converted into a 64-dimensional feature vector. A batch normalization layer and a ReLU activation layer are added after the first convolution layer. The second layer is a one-dimensional convolution layer with 128 convolution kernels and a step size of 1. It further processes the feature vector. A batch normalization layer and a ReLU activation layer are also added after this layer. The third layer is a one-dimensional convolution layer with 1024 convolution kernels and a step size of 1. After that, this layer is connected to a global maximum pooling layer to obtain the global features of the entire input point cloud. After that, the global features output by the maximum pooling layer are input to the fully connected layer. The output dimension of the first fully connected layer is 512, followed by a batch normalization layer and a ReLU activation layer, followed by a second fully connected layer with an output dimension of 256, followed by a batch normalization layer and a ReLU activation layer, followed by the output transformation matrix of a fully connected layer, which is used to perform rigid transformation and feature alignment on each point in the point cloud, and finally connected to the PointConv layer, whose weight function is dynamically calculated by the multilayer perceptron based on the relative relationship between points.

[0078] The feature decoding module includes three combined layers to recover the detailed features of the point cloud from the feature representation compressed during the encoding process. The combined layer consists of an interpolation layer and a PoinConv layer. Each layer of the combined layer receives the data of the previous layer and the data of the three combined layers in the feature encoding module, and outputs local features and global features. The interpolation satisfies the formula , q For any query point, Indicates i The weight of the nearest neighbor point. The larger the weight, the greater the influence of the neighbor point on the estimated value. , express q With i The distance to the nearest neighbor, p is a constant greater than 0, used to control how distance affects the weight; Indicates i nearest neighbor points; ε Represents a very small positive number, which is used to prevent the sum of the weights from being 0 and causing the denominator to be zero; k is the number of nearest neighbors. p Set to 2, k Set to 3.

[0079] The skip link transfers information from the early layers to the deep layers to restore local details and enhance the accuracy of global semantics. A fully connected layer and a ReLU layer are used to update the feature vector of each point, ensuring effective feature transfer and nonlinear enhancement.

[0080] Step 3: Segment and classify the 3D point cloud data;

[0081] First, the 3D point cloud data is segmented into several subsets, and then each subset is classified to determine the most matching geometric primitive type.

[0082] In one embodiment, a 3D point cloud data segmentation network is used to segment the 3D point cloud data. The 3D point cloud data is subdivided into several subsets, and then each segmented subset is classified to determine the most matching geometric primitive type. The RIoU (Relaxed Intersection over Union) of the 3D point cloud data segmentation network is defined as follows:

[0083] ;

[0084] in, is the preset ground truth matrix, is the prediction matrix, N is the total number of points; , representing different geometric primitives, K = 0 is a plane, K =1 for a sphere, K =2 is a cylinder, K =3 is a cone.

[0085] In the segmentation task, the point cloud is carefully analyzed, where each point is regarded as an independent unit and assigned a corresponding category label. In the high-dimensional feature space extracted by the feature extraction network, a neighbor aggregation operation is performed within the spatial range to enhance the descriptiveness of each point feature. Then, the spatial information lost in the downsampling process is restored through interpolation and feature propagation. Finally, the enhanced features are passed through one or more fully connected layers, combined with activation functions and classifiers (such as Softmax) for point-by-point classification, and multiple point cloud fragments are obtained, each of which is a subset.

[0086] In one embodiment, during the classification process, a three-dimensional point cloud data classification network is used to classify the three-dimensional point cloud data. In the classification task, the three-dimensional point cloud data classification network performs a holistic analysis of the point cloud data in each subset of the input, and outputs a prediction of the primitive type represented by the data. This means that instead of focusing on a single point, the classification task focuses on the entire set of point cloud fragments, and the features of all points in the point cloud fragment are converted into a single high-dimensional vector through a global pooling operation, thereby capturing global contextual information. The global high-dimensional vector is then input into a series of fully connected layers to capture the complex relationships and patterns between different features. Finally, the Softmax activation function is used to map the output of the fully connected layer to the category probability distribution, and the global classification of the point cloud fragment is completed.

[0087] Step 4: For the geometric primitives matched in step 3, predict specific mathematical parameters.

[0088] (1) Plane

[0089] The fitted plane can be expressed by the following equation:

[0090] ;

[0091] in, A, B, C The three coefficients form the normal vector of the plane, x,y,z are the coordinates of the points, D is a constant coefficient.

[0092] Perform the calculation process of minimizing the weighted sum of squares on it:

[0093] ;

[0094] Among them, the normal vector of the fitting plane , is the weight, and N is the number of points. S about A. B , C.D Find the partial derivatives respectively and set them to 0, and get the following system of equations:

[0095] ;

[0096] Convert the above system of equations into matrix form:

[0097] ;

[0098] Using singular value decomposition M This breaks down into: ,in, U is a unit orthogonal matrix with columns M The left singular vectors of . is a diagonal matrix with non-zero entries on the diagonal M The singular values ​​of . V The columns are M The right singular vectors of , whose columns form an orthogonal basis. The smallest singular value in V The columns in are non-zero solutions to the homogeneous linear system, which describe the coefficients of the best plane that fits the data points. A , B and C .

[0099] (2) Sphere

[0100] The fitted sphere can be expressed by the following equation:

[0101] ;

[0102] in, are the coordinates of the center of the sphere, is the radius of the sphere.

[0103] Given a set of points and the corresponding weights , find the optimal sphere by minimizing the sum of the squares of the weighted distances, that is:

[0104] ;

[0105] make ,have to ,Will Substituting into the above equation we get c The equation is:

[0106] ;

[0107] Let the centroid of all points be: , where the matrix X No. i Behavior ,vector y No. i The elements are: .in, Indicates i The coordinate matrix of the points. Construct the coefficient matrix of the normal equation , and the constant term Solve the following normal equations using Cholesky decomposition to find the center of the sphere c :

[0108] ;

[0109] According to the center of the ball c The coordinates of the sphere can be used to find the square of the radius of the sphere, and then the radius value:

[0110] ;

[0111] (3) Cylinder

[0112] The parameterized equation of the fitted cylinder is expressed as:

[0113] ;

[0114] Among them, the vector , represents the unit vector in the direction of the cylinder axis, coordinate are the coordinates of the geometric center point of the cylinder, ris the radius of the cylinder. For a point in space and the corresponding weights , calculate the square of its distance to the surface of the cylinder to be fitted:

[0115] ;

[0116] in, , indicating a point Relative to the center of the cylinder c The position vector of .

[0117] To solve the above distance, we first need to determine the unit vector of the cylinder axis direction, which can be achieved by finding a common direction in the set of normal lines of the data points. The normal line at the data point is perpendicular to the cylinder axis vector. Therefore, we need to find a vector a , so that the dot product of the normal of all points and this vector is as small as possible.

[0118] ;

[0119] in, N is the normal vector of the data point. Once the unit vector in the direction of the cylinder axis is determined a , we can find the geometric center point of the cylinder c and radius r . Represents the weight matrix. Assume that all data points can form a circle after being projected onto the plane where the cylinder axis is located. Fitting a circle through these projected points is similar to the sphere fitting process. Its goal is to find the best center and radius to approximate the data points.

[0120] (4) Cone

[0121] The cone can be expressed parametrically as:

[0122] ;

[0123] in, a is a unit vector, indicating the distance from the vertex c The axial direction vector of the cone is indicated by , is the half angle of the cone taper. For a point in space and the corresponding weights , calculate its distance to the cone surface:

[0124] ;

[0125] in, , indicating a point Relative to the top of the cone c The position vector of , when solving the computational problem of approximating the cone vertex and axis, we first independently estimate a and c Then estimate the half angle θ Since the intersection of all tangent planes on the cone surface is the cone vertex c , so the predicted point normal vector can be N , the multi-plane intersection problem is expressed as a least squares method:

[0126] ;

[0127] in, .

[0128] Finally, use the cone top c and vector a , calculate the half angle θ :

[0129] ;

[0130] This embodiment also provides a three-dimensional point cloud data geometric primitive fitting system based on PointConv, including the following modules:

[0131] A 3D point cloud data preprocessing module is used to preprocess the 3D point cloud data, including noise reduction, smoothing and downsampling processing;

[0132] A three-dimensional point cloud feature extraction module is used to extract three-dimensional point cloud features using a three-dimensional point cloud feature extraction network;

[0133] The three-dimensional point cloud feature extraction network includes a feature encoding module and a feature decoding module;

[0134] The feature encoding module includes three first combination layers connected in sequence and a transformation network connected in sequence, wherein the first combination layer is composed of a sampling layer, a grouping layer, and a PointConv convolution kernel; the PointConv convolution kernel includes three convolution layers connected in sequence, a maximum pooling layer, three fully connected layers, and a PointConv layer; a batch normalization layer and a ReLU activation layer are arranged after the first convolution layer, the second convolution layer, the first fully connected layer, and the second fully connected layer; then a fully connected layer outputs a transformation matrix for rigidly transforming and aligning features for each point in the point cloud; the transformation network includes a feature transformation network and a one-dimensional convolution layer, wherein the feature transformation network calculates a density scale, then multiplies the local features by the density scale to obtain weighted local features, and then further processes these features through a one-dimensional convolution layer with a final number of convolution kernels of 512, and finally outputs an encoded feature matrix.

[0135] The feature decoding module includes three second combination layers connected in sequence, which recover the detailed features of the point cloud from the feature representation compressed in the encoding process; the second combination layer is composed of an interpolation operation layer and a PoinConv layer, and the second combination layer receives the previous layer data and the three first combination layer jump link data in the feature encoding module respectively, and outputs local features and global features;

[0136] A 3D point cloud data segmentation and classification module is used to segment and classify 3D point cloud data;

[0137] First, the 3D point cloud data is segmented into several subsets, and then each subset is classified to determine the most matching geometric primitive type;

[0138] The geometric primitive mathematical parameter prediction module is used to predict specific mathematical parameters for the matched geometric primitives.

[0139] This embodiment also provides a three-dimensional point cloud data geometric primitive fitting device based on PointConv, including:

[0140] one or more processors;

[0141] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the three-dimensional point cloud data geometric primitive fitting method based on PointConv.

[0142] The invention is further described below through specific experiments.

[0143] Please see Figure 5 , are the geometric primitives used in this experiment, including plane, sphere, cylinder and cone. The experimental results can be found in Figure 6-Figure 12 , Figure 6 This is the effect diagram of point cloud processing. Figure 7 This is a comparison chart with other methods. It can be seen that the segmentation of point cloud is improved by 31.49% compared with RANSAC, 11.64% compared with SPFN, and 9.15% compared with CPFN. In terms of classification accuracy, it is improved by 32.59% compared with RANSAC, 6.89% compared with SPFN, and 2.06% compared with CPFN. Figure 8 The effect of processing point clouds and reproducing the geometry by using the geometric parameters predicted by the network, Fig. 9 , 10, 11, and 12 are the prediction tests of each geometric primitive parameter. Each primitive uses 1000 point cloud data to test the average absolute error of the primitive parameter prediction under the conditions of no noise, 5% noise, 10% noise, and 15% noise. When the point cloud data is noise-free, the error of each geometric primitive parameter is calculated to be almost 0. As can be seen from the figure, the curve of each geometric primitive parameter prediction is almost close to the true value (ground-truth), which reflects the accuracy of the primitive parameter prediction.

[0144] It should be understood that the embodiments described above are part of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. Such combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0145] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A method for fitting geometric primitives of 3D point cloud data. It is characterized in that The following steps are involved: Step 1: Preprocess the 3D point cloud data, including noise reduction, smoothing and downsampling; Step 2: Use a 3D point cloud feature extraction network to extract 3D point cloud features; The three-dimensional point cloud feature extraction network includes a feature encoding module and a feature decoding module; The feature encoding module includes three first combination layers connected in sequence and a transformation network, wherein the first combination layer is composed of a sampling layer, a grouping layer, and a PointConv convolution kernel; the PointConv convolution kernel includes three convolution layers connected in sequence, a maximum pooling layer, three fully connected layers, and a PointConv layer; a batch normalization layer and a ReLU activation layer are arranged after the first convolution layer, the second convolution layer, the first fully connected layer, and the second fully connected layer; then a fully connected layer outputs a transformation matrix for rigidly transforming and aligning features for each point in the three-dimensional point cloud data; the transformation network includes a feature transformation layer and a one-dimensional convolution layer, wherein the feature transformation layer calculates a density scale, then multiplies the local features by the density scale to obtain weighted local features, and then further processes these features through a one-dimensional convolution layer, and finally outputs an encoded feature matrix; The feature decoding module includes three second combination layers connected in sequence, which recover the detailed features of the point cloud from the feature representation compressed in the encoding process; the second combination layer is composed of an interpolation operation layer and a PoinConv layer, and each layer of the second combination layer receives the data of the previous layer and the data of the corresponding first combination layer jump link in the feature encoding module, and outputs local features and global features; Step 3: Segment and classify the 3D point cloud data; First, the 3D point cloud data is segmented into several subsets, and then each subset is classified to determine the most matching geometric primitive type; Among them, during the segmentation process, a 3D point cloud data segmentation network is used to perform a detailed analysis of the point cloud. Each point in the 3D point cloud data is regarded as an independent unit and is assigned a corresponding category label. In the high-dimensional feature space extracted by the feature extraction network, a neighbor aggregation operation is performed within the spatial range to enhance the descriptiveness of each point feature. Then, the spatial information lost in the downsampling process is restored through interpolation and feature propagation. Finally, the enhanced features are passed through one or more fully connected layers, combined with activation functions and classifiers for point-by-point classification, to obtain multiple point cloud fragments, each of which is a subset. The three-dimensional point cloud data segmentation network is a deep neural network, and its segmentation network relaxes the intersection-and-union ratio Defined as: ; in, is the preset truth label matrix, is the prediction matrix, N is the total number of points; , representing different geometric primitives, K = 0 is a plane, K =1 for a sphere, K =2 is a cylinder, K =3 is a cone; Among them, during the classification process, a three-dimensional point cloud data classification network is used to convert the features of all points in the point cloud fragment into a single high-dimensional vector through a global pooling operation. All high-dimensional vectors together form a global high-dimensional vector, thereby capturing global context information; then the global high-dimensional vector is input into a series of fully connected layers to capture the complex relationships and patterns between different features; finally, the Softmax activation function is used to map the output of the fully connected layer to the category probability distribution, completing the global classification of the point cloud fragment; Step 4: For the geometric primitives matched in step 3, predict specific mathematical parameters.

2. The method for fitting geometric primitives of three-dimensional point cloud data according to claim 1, Features: In step 1, the noise reduction process is to perform statistical filtering on the three-dimensional point cloud data; for each point in the three-dimensional point cloud, its k The average distance of the nearest neighbor points, and the global standard deviation is obtained based on the average distance of all points; The threshold is set as the average of the average distance from each point to its neighboring points plus a number of standard deviations, outliers whose average distance is greater than the threshold are removed, and the remaining points are retained as the filtered point cloud data.

3. The method for fitting geometric primitives of three-dimensional point cloud data according to claim 1, Features: In step 1, the smoothing process is to perform Gaussian filtering on the three-dimensional point cloud data. For each point P in the point cloud obtained after the noise reduction process, its neighborhood is defined, the distance from each point in the neighborhood to the center point is calculated, and the weight is calculated using the Gaussian function based on this distance. Using the calculated weight, a new position is calculated for each point, and the new position is the weighted average position of all points in the neighborhood.

4. The method for fitting geometric primitives of three-dimensional point cloud data according to claim 1, Features: In step 1, the smoothing process is performed for each point , m is the size of the 3D point cloud dataset, Each point in the neighborhood ,calculate and The Euclidean distance between , using Gaussian function to calculate weights ;in, is the standard deviation of the Gaussian function, which controls the speed at which the weight changes with distance. Calculate weighted position ; Finally calculate the new position after smoothing .

5. The method for fitting geometric primitives of three-dimensional point cloud data according to claim 1, Features: In step 1, the downsampling process is to downsample the three-dimensional point cloud data; for the point cloud obtained after the smoothing process, a greedy algorithm is used for sampling, and points in the point cloud are selected in an iterative manner to cover the entire point cloud to the greatest extent; In each iteration, select the point farthest from the currently selected set of points; The initial points are randomly selected, and then the algorithm proceeds in an iterative manner until a preset number of points are collected.

6. The method for fitting geometric primitives of three-dimensional point cloud data according to claim 1, Features: In step 2, the interpolation operation layer, the interpolation operation satisfies the formula ,in, q For any query point, Indicates i The weight of the nearest neighbor point. The larger the weight, the greater the influence of the neighbor point on the estimated value. , express q With i The distance to the nearest neighbor, p is a constant greater than 0, used to control how distance affects the weight; Indicates i nearest neighbor points; ε Represents a very small positive number, which is used to prevent the sum of the weights from being 0 and causing the denominator to be zero; k is the number of nearest neighbors.

7. A 3D point cloud data geometric primitive fitting system, It is characterized in that Includes the following modules: A 3D point cloud data preprocessing module is used to preprocess the 3D point cloud data, including noise reduction, smoothing and downsampling processing; A three-dimensional point cloud feature extraction module is used to extract three-dimensional point cloud features using a three-dimensional point cloud feature extraction network; The three-dimensional point cloud feature extraction network includes a feature encoding module and a feature decoding module; The feature encoding module includes three first combination layers connected in sequence and a transformation network, wherein the first combination layer is composed of a sampling layer, a grouping layer, and a PointConv convolution kernel; the PointConv convolution kernel includes three convolution layers connected in sequence, a maximum pooling layer, three fully connected layers, and a PointConv layer; a batch normalization layer and a ReLU activation layer are arranged after the first convolution layer, the second convolution layer, the first fully connected layer, and the second fully connected layer; then a fully connected layer outputs a transformation matrix for rigidly transforming and aligning features for each point in the point cloud; the transformation network includes a feature transformation layer and a one-dimensional convolution layer, wherein the feature transformation layer calculates a density scale, then multiplies the local features by the density scale to obtain weighted local features, and then further processes these features through a one-dimensional convolution layer, and finally outputs an encoded feature matrix; The feature decoding module includes three second combination layers connected in sequence, which recover the detailed features of the point cloud from the feature representation compressed in the encoding process; the second combination layer is composed of an interpolation operation layer and a PoinConv layer, and the second combination layer receives the previous layer data and the three first combination layer jump link data in the feature encoding module respectively, and outputs local features and global features; A 3D point cloud data segmentation and classification module is used to segment and classify 3D point cloud data; First, the 3D point cloud data is segmented into several subsets, and then each subset is classified to determine the most matching geometric primitive type; Among them, during the segmentation process, a 3D point cloud data segmentation network is used to perform a detailed analysis of the point cloud. Each point in the 3D point cloud data is regarded as an independent unit and is assigned a corresponding category label. In the high-dimensional feature space extracted by the feature extraction network, a neighbor aggregation operation is performed within the spatial range to enhance the descriptiveness of each point feature. Then, the spatial information lost in the downsampling process is restored through interpolation and feature propagation. Finally, the enhanced features are passed through one or more fully connected layers, combined with activation functions and classifiers for point-by-point classification, to obtain multiple point cloud fragments, each of which is a subset. The three-dimensional point cloud data segmentation network is a deep neural network, and its segmentation network relaxes the intersection-and-union ratio Defined as: ; in, is the preset truth label matrix, is the prediction matrix, N is the total number of points; , representing different geometric primitives, K = 0 is a plane, K =1 for a sphere, K =2 is a cylinder, K =3 is a cone; Among them, during the classification process, a three-dimensional point cloud data classification network is used to convert the features of all points in the point cloud fragment into a single high-dimensional vector through a global pooling operation. All high-dimensional vectors together form a global high-dimensional vector, thereby capturing global context information; then the global high-dimensional vector is input into a series of fully connected layers to capture the complex relationships and patterns between different features; finally, the Softmax activation function is used to map the output of the fully connected layer to the category probability distribution, completing the global classification of the point cloud fragment; The geometric primitive mathematical parameter prediction module is used to predict specific mathematical parameters for the matched geometric primitives.

8. A three-dimensional point cloud data geometric primitive fitting device, It is characterized in that include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the three-dimensional point cloud data geometric primitive fitting method as described in any one of claims 1 to 6.