A parametric edge curve extraction method for point cloud objects

Abstract

The present invention discloses a parameterized edge curve extraction method for point cloud objects, which pre-processes the original point cloud data; embeds a fusion residual attention mechanism module CASP on the basis of the PointNet++ network, combines the channel attention mechanism and the spatial attention mechanism, and embeds the position information into the channel attention, so that the edge information of the point cloud object can be highlighted, introduces a pyramid pooling mechanism, and classifies the feature points of the point cloud object through an improved network model; predicts the opening and closing points and the closing points according to the feature points, and generates the corresponding opening and closing candidate curves and the closing candidate curves; finally, the best curves are selected for the opening and closing candidate curves and the closing candidate curves by measuring the overlap degree of the point set and hierarchical clustering. The present invention improves the problem of inaccurate classification of feature points of point cloud objects and improves the accuracy of edge curve extraction of point cloud objects.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision and artificial intelligence, and relates to a parameterized edge curve extraction method for a point cloud object. Background Art

[0002] As laser scanning technology matures, 3D point cloud data has become more and more mature and reliable, and the expression of object information is more realistic. In the expression of 3D objects, edge information can accurately reflect the structural information of the object and plays an irreplaceable role in the simplified representation of the object. Therefore, it has always been widely concerned by scholars. Extracting the edge feature information of the object helps to achieve 3D reconstruction of industrial product models, data simplification, point cloud segmentation, point cloud registration, etc. Accurately extracting the edge features of the object has important theoretical significance and research value.

[0003] For a long time, researchers have mainly used traditional methods to extract edge curves of point cloud objects. However, traditional methods have problems such as taking a long time to extract edges when the number of point clouds is large; being easily affected by noise points; the extracted edge curves have structural defects; and they are prone to extracting many non-boundary points. In recent years, as the advantages of deep learning have gradually emerged, researchers have begun to try to use deep learning methods to extract feature edges of objects, but current methods still have problems such as inaccurate feature point classification and structural defects in edge curves. Therefore, it is very necessary to study and explore the precise edge extraction technology of point cloud objects. Summary of the invention

[0004] The purpose of the present invention is to provide a parameterized edge curve extraction method for a point cloud object, which solves the problem of inaccurate feature point classification in the prior art.

[0005] The technical solution adopted by the present invention is a parameterized edge curve extraction method for point cloud objects, which is specifically implemented according to the following steps:

[0006] Step 1: downsample the original point cloud data using the farthest point sampling algorithm to obtain downsampled point cloud data, then parameterize the point cloud data and obtain the normal vector of each point in the point cloud data based on the principal component analysis method;

[0007] Step 2: Design a network model structure to classify feature points of the downsampled point cloud data obtained in step 1.

[0008] Step 3, generating an open and closed candidate curve and a closed candidate curve according to the feature points obtained in step 2;

[0009] Step 4: Select the best curve based on the open and close candidate curves and the closed candidate curves obtained in step 3.

[0010] The present invention is also characterized in that

[0011] Step 1 is as follows:

[0012] Step 1.1, down-sampling the original point cloud data by using the farthest point sampling algorithm to obtain the down-sampled point cloud data;

[0013] Step 1.2, parameterizing the downsampled point cloud data obtained in step 1.1;

[0014] Step 1.3, estimate the normal vector of the downsampled point cloud data obtained in step 1.1 based on the principal component analysis method.

[0015] Step 1.1 is as follows:

[0016] Step 1.1.1, define and initialize sets A, B, and sample_data in the computer, where A is recorded as null, the original point cloud data is stored in B, and sample_data is recorded as null;

[0017] Step 1.1.2, randomly sort set B, select the first point, and store it in set A;

[0018] Step 1.1.3, calculate the distance from each point in set B to the midpoint of set A, select the point with the largest distance, and store it in set A;

[0019] Step 1.1.4, calculate the distances from point pB in set B to the midpoints in set A, and select the smallest distance value from the calculated distance values ​​as the distance from point pB to set A;

[0020] Step 1.1.5, for all points in set B, calculate a distance value for each point according to step 1.1.4, select the point corresponding to the maximum distance value and store it in set A;

[0021] Step 1.1.6, repeat steps 1.1.4-1.1.5 until 8096 points are selected and saved to sample_data.

[0022] Step 1.2 is as follows:

[0023] Step 1.2.1, parameterize the straight line by connecting two corner points;

[0024] Step 1.2.2, parameterize the circle as:

[0025] β=(p1,p2,p3) (1)

[0026] In formula (1), p1, p2, and p3 are any three non-collinear points on the circle. Formula (1) can be expressed using the normal, center, and radius of the circle as follows:

[0027] β=(n,c,r) (2)

[0028] In formula (2), n is the normal of the circle, c is the center of the circle, and r is the radius of the circle. Next, randomly sample points on the circle and express them as a function of β to generate random samples:

[0029] p(α|β)=c+r(ucos(α)+vsin(α)) (3)

[0030] In formula (3), α is a randomly sampled point on the circle, α∈[0,2π], and u=p1-c, v=u×n;

[0031] Step 1.2.3, using four control points To parameterize the B-spline curve, where p i It is the control vertex that represents the change of the curve, and B i,K(·) To represent the K-order B-spline curve of the i-th order basis function, and uniformly sample points α, α∈[0,1], to generate uniform random samples:

[0032] p(α|β)=∑ i p i B i,K(α) (4)

[0033] Step 1.3 is as follows:

[0034] Step 1.3.1, for the points in the initial point cloud data after downsampling, use the K nearest neighbor search method to find k points adjacent to it, k = 10. From the principal component analysis method, we can know that:

[0035]

[0036] Make the vector formed by the point and each of its neighboring points perpendicular to the normal vector. In formula (5), f(m) represents the minimum variance of the projection of all points on the constructed vector, c is the average value of all neighboring points and the point, m is the number of all points in the neighborhood, and x is the distance between the point and the normal vector. i for all points in the neighborhood;

[0037] Step 1.3.2, take c as the center point of all points in the neighborhood, that is:

[0038]

[0039] Let y i =x i -c,y i represents the decentralized point, so formula (5) is expressed as:

[0040]

[0041] Step 1.3.3, simplify formula (7) to obtain:

[0042] f(m)=minm T Sm (8)

[0043] Where S = (YY T ), Y represents the new data matrix after decentralization, st(m T m=1), S represents a 3×3 covariance matrix;

[0044] Step 1.3.4, solve the eigenvalues ​​and corresponding eigenvectors of the covariance matrix S obtained in step 1.3.3, which are λ1, λ2, λ3 and a1, a2, a3 respectively, where the eigenvector a3 corresponding to the minimum eigenvalue λ3 is the normal vector of the tangent plane at that point;

[0045] Step 1.3.5, repeat steps 1.3.1-1.3.4 to obtain the normal vectors of all points in the point cloud data and save them in the mat file.

[0046] Step 2 is as follows:

[0047] Step 2.1, perform initial feature extraction on the downsampled point cloud data obtained in step 1 through four point set abstraction layers and four feature propagation layers of the PointNet++ network;

[0048] Step 2.2, embed the residual module and the attention mechanism module CASP which combines the channel attention mechanism and the spatial attention mechanism after each feature propagation layer;

[0049] Step 2.3, embed the SPP spatial pyramid pooling mechanism module after the last attention mechanism module embedded in step 2.2;

[0050] Step 2.4, classify the initial feature points through the fully connected layer, and predict the probability that each point in the point cloud object belongs to an edge point or a corner point;

[0051] Step 2.5, the initial feature point classification result obtained in step 2.4 is optimized using the non-maximum suppression algorithm to obtain the final feature point classification result.

[0052] Step 2.1 is as follows:

[0053] Step 2.1.1, take the downsampled point cloud data obtained in step 1 as input data and represent it as an N×3 second-order tensor, where N represents the number of point clouds, the initial number of point clouds is 8096, and 3 corresponds to the coordinate dimension;

[0054] Step 2.1.2, through the four point set abstraction layers of the PointNet++ network, first use the farthest point sampling method to get npoint center points, then use the ball query method with radius radius to find nsample points near the center point to form npoint local areas, and use the multi-layer perceptron MLP to perform initial feature extraction for each local area. The point set abstraction layer is represented by SA(npoint, radius, nsample, mlp), where npoint is the total number of sampling points, radius is the sampling radius, nsample is the number of points sampled in each sampling area, mlp is the multi-layer perceptron parameter, and the four point set abstraction layers are SA(4096, 0.05, 32, [32, 64]), SA(2048, 0.1, 32, [64, 128]), SA(1024, 0.2, 32, [128, 256]) and SA(512, 0.4, 32, [256, 512]);

[0055] Step 2.1.3, use the inverse distance weighted interpolation method to upsample the point cloud data and extract point cloud features through the four feature propagation layers of the PointNet++ network to restore the original number of point clouds. The feature propagation layer is represented by FP(mlp), where mlp is the multi-layer perceptron parameter. The four feature propagation layers are FP(256,256), FP(256,256), FP(256,128), and FP(128,128,128);

[0056] Step 2.2 is as follows:

[0057] In step 2.2.1, a residual module is embedded after each feature propagation layer in step 2.1 to extract features.

[0058] Step 2.2.2: For the initial features obtained in step 2.2.1, use the pooling convolution kernel to aggregate the features in the horizontal and vertical directions respectively;

[0059] Step 2.2.3, a two-dimensional convolution operation and a Sigmoid activation operation are performed in the horizontal direction and the vertical direction respectively;

[0060] Step 2.2.4, perform element-by-element multiplication of the tensor obtained in step 2.2.3 and the input feature activated by the relu function to obtain the final channel attention output feature;

[0061] Step 2.2.5: Perform maximum pooling and average pooling operations on the final channel attention output features obtained in step 2.2.4 along the channel axis, then concat the two feature information, and then perform a two-dimensional convolution operation of size 7×7 and a Maxout activation operation;

[0062] Step 2.2.6, multiply the output feature obtained in step 2.2.5 by the final channel attention output feature obtained in step 2.2.4 element by element to obtain the output feature that combines the channel attention mechanism and the spatial attention mechanism.

[0063] Step 2.3 is as follows:

[0064] Step 2.3.1, the output features obtained in step 2.2 are divided and pooled at the maximum, with the number of channels being 128, and the divided features are concatenated to finally obtain an output feature of 8096*128.

[0065] Step 2.4 is as follows:

[0066] Step 2.4.1, estimate the probability value of each point in the point cloud object belonging to an edge point or a corner point through the fully connected layer;

[0067] Step 2.4.2: Set the edge point corresponding threshold τ based on the principle that the candidate points include the required main feature points and do not contain too many non-feature points e And the corner point corresponding threshold τ c , τ e =0.75, τ c =0.95;

[0068] Step 2.4.3, taking the points whose probability values ​​obtained in step 2.4.1 are greater than the corresponding threshold as the initial feature point classification results;

[0069] Step 2.4.4: In order to reduce the generalization error in the deep neural network, data batch normalization and Dropout regularization with a drop rate of 0.5 are performed after the fully connected layer.

[0070] Step 2.5 is as follows:

[0071] Step 2.5.1, define and initialize the sets S, C, and F in the computer, where the initial corner point data obtained in step 2.4 is stored in S, C is recorded as null, and F is recorded as null;

[0072] Step 2.5.2, use the distance threshold δ = 0.05l to divide the set S into h windows containing the corner points, where l is the length of the window diagonal;

[0073] Step 2.5.3, for window j, j = 1, 2, ..., h, where h is a natural number not equal to 0, the probability values ​​of every two corner points in j are sorted, the larger one is put into set C, and the point is deleted from S;

[0074] Step 2.5.4, take a point with the largest probability value in set C and put it into set F, and set set C to null;

[0075] Step 2.5.5, repeat steps 2.5.3-2.5.4, and the point data in set F is the final feature point classification result.

[0076] Step 3 is as follows:

[0077] Step 3.1: Generate a corner point combination O(M) based on the M corner points in the set F obtained in step 2. 2 ), and create the corner point pair {P i ={f i1 ,f i2}|f i1 ,f i2 ∈F}, each corner point pair will correspond to an opening and closing candidate curve;

[0078] Step 3.2, for each corner point pair, use a heuristic algorithm to find the center of the corner point (f i1 +f i2 ) / 2, the radius R is ||f i1 -f i2 Uniformly sample a subset within the sphere of || / 2 Get the corresponding opening and closing points;

[0079] Step 3.3, first embed the four point set abstraction layers and four feature propagation layers of the PointNet++ network, then embed the residual module after each feature propagation layer, the attention mechanism module CASP that combines the channel attention mechanism and the spatial attention mechanism, and embed the SPP spatial pyramid pooling mechanism module after the last feature propagation layer. Finally, for the L edge points obtained in step 2, the above network structure is used to generate the corresponding features C(·), where the multi-layer perceptron parameters in the point set abstraction layer and the feature propagation layer are SA(32,32,64), SA(64,64,128), SA(128,128,256) and SA(256,256,512) and FP(256,256), FP(256,256), FP(256,128), FP(128,128,128);

[0080] Step 3.4, create a similarity matrix D∈R through a multi-layer perceptron MLP L×L ,satisfy:

[0081] D ij =||C(λ i )-C(λ j )||2 (9)

[0082] In formula (9), D ijRepresents the characteristic C(λ i ) and the characteristic C(λ j ), and λ i With λ j is the probability value of the edge point, each row of the similarity matrix D is taken as an alternative subset, and the set As the edge point set of the mth candidate subset, is a threshold used to filter out points with greatly different feature scores, and the corresponding closed points are obtained;

[0083] Step 3.5, apply the opening and closing points obtained in step 3.2 to the PointNet network, using two MLP layers with parameters of [128, 256, 512] and [256, 256] respectively. Then, through the fully connected layer and the loss function, determine whether a specific point belongs to a candidate curve, identify the type of the candidate curve (straight line, circle or B-spline curve), and predict the parameters of the candidate curve to obtain the opening and closing candidate curves.

[0084] Step 3.6, through three points β = {p a +Δ a ,p b +Δ b ,p c +Δ c} to parameterize each circular closed candidate curve, and the edge point set F obtained in step 3.4 m Perform the farthest point sampling and use p a =p m Initialize and get {p a ,p b ,p c};

[0085] Step 3.7, set F m Applied to the PointNet network, the closed candidate curve parameters are predicted and the confidence is estimated through the fully connected layer and loss function to obtain the closed candidate curve.

[0086] Step 4 is as follows:

[0087] Step 4.1, given two sets of points associated with the two candidate opening and closing curves obtained in step 3, measure the degree of overlap of the two sets of points by O(A,B)=max{I(A,B) / A,I(A,B) / B}, where I(A,B) is the cardinality of the intersection of the two sets of points;

[0088] Step 4.2, if O(A,B) is greater than the threshold τ determined by hyperparameter tuning o , then merge the two sets of points and select the best opening and closing curve by retaining the one with the larger cardinality, where τo =0.8;

[0089] Step 4.3, for two closed candidate curves, when IoU(A,B)>τ iou When , hierarchical clustering is performed on closed candidate curves, where τ iou Set to 0.6;

[0090] Step 4.4: For each cluster obtained by clustering in step 4.3, the matching score of the candidate curve is measured by the angle distance, and finally the best closed curve is selected from the clusters obtained by clustering with the highest confidence.

[0091] The beneficial effects of the present invention are:

[0092] The present invention first preprocesses the original point cloud data, including downsampling through the farthest point sampling algorithm, parameterization and normal vector estimation based on principal component analysis, which improves the efficiency of edge extraction of point cloud objects; then, the residual fusion attention mechanism module CASP is embedded on the basis of the PointNet++ network, the channel attention mechanism and the spatial attention mechanism are combined, and the position information is embedded in the channel attention, so that the edge information of the point cloud object can be highlighted, and in order to further improve the network's ability to capture global context information, a pyramid pooling mechanism is introduced, and the feature points of point cloud objects are classified through an improved network model; then, the opening and closing points and the closing points are predicted according to the feature points, and the corresponding opening and closing candidate curves and the closing candidate curves are generated; finally, the best curves are selected for the opening and closing candidate curves and the closing candidate curves by measuring the overlap degree of the point set and hierarchical clustering. The present invention improves the problem of inaccurate classification of feature points of point cloud objects and improves the accuracy of edge curve extraction of point cloud objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 It is the down-sampled point cloud data in an embodiment of a parameterized edge curve extraction method for a point cloud object of the present invention;

[0094] Figure 2 It is a normal vector estimation result in an embodiment of a parameterized edge curve extraction method for a point cloud object of the present invention;

[0095] Figure 3 is an initial feature point classification result in an embodiment of a parameterized edge curve extraction method for a point cloud object of the present invention;

[0096] Figure 4 is the final feature point classification result in an embodiment of a parameterized edge curve extraction method for a point cloud object of the present invention;

[0097] Figure 5It is the prediction result of the opening and closing points and the closing points in an embodiment of a parameterized edge curve extraction method of a point cloud object of the present invention;

[0098] Figure 6 is a candidate curve extraction result in an embodiment of a parameterized edge curve extraction method for a point cloud object of the present invention;

[0099] Figure 7 It is the best curve extraction result in an embodiment of a parameterized edge curve extraction method for a point cloud object of the present invention. DETAILED DESCRIPTION

[0100] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0101] The present invention provides a parameterized edge curve extraction method for a point cloud object, which is specifically implemented according to the following steps:

[0102] Step 1: Downsample the original point cloud data using the farthest point sampling algorithm to obtain the downsampled point cloud data, such as Figure 1 As shown, the point cloud data is then parameterized and the normal vector of each point in the point cloud data is obtained based on the principal component analysis method, as shown in Figure 2 As shown; specifically:

[0103] Step 1.1, downsample the original point cloud data using the farthest point sampling algorithm to obtain the downsampled point cloud data; specifically:

[0104] Step 1.1.1, define and initialize sets A, B, and sample_data in the computer, where A is recorded as null, the original point cloud data is stored in B, and sample_data is recorded as null;

[0105] Step 1.1.2, randomly sort set B, select the first point, and store it in set A;

[0106] Step 1.1.3, calculate the distance from each point in set B to the midpoint of set A, select the point with the largest distance, and store it in set A;

[0107] Step 1.1.4, calculate the distances from point pB in set B to the midpoints in set A, and select the smallest distance value from the calculated distance values ​​as the distance from point pB to set A;

[0108] Step 1.1.5, for all points in set B, calculate a distance value for each point according to step 1.1.4, select the point corresponding to the maximum distance value and store it in set A;

[0109] Step 1.1.6, repeat steps 1.1.4-1.1.5 until 8096 points are selected and saved to sample_data;

[0110] Step 1.2, parameterize the downsampled point cloud data obtained in step 1.1; specifically:

[0111] Step 1.2.1, parameterize the straight line by connecting two corner points;

[0112] Step 1.2.2, parameterize the circle as:

[0113] β=(p1,p2,p3) (1)

[0114] In formula (1), p1, p2, and p3 are any three non-collinear points on the circle. Formula (1) can be expressed using the normal, center, and radius of the circle as follows:

[0115] β=(n,c,r) (2)

[0116] In formula (2), n is the normal of the circle, c is the center of the circle, and r is the radius of the circle. Next, randomly sample points on the circle and express them as a function of β to generate random samples:

[0117] p(α|β)=c+r(ucos(α)+vsin(α)) (3)

[0118] In formula (3), α is a randomly sampled point on the circle, α∈[0,2π], and u=p1-c, v=u×n;

[0119] Step 1.2.3, using four control points To parameterize the B-spline curve, where p i It is the control vertex that represents the change of the curve, and B i,K(·) To represent the K-order B-spline curve of the i-th order basis function, and uniformly sample points α, α∈[0,1], to generate uniform random samples:

[0120] p(α|β)=∑ i p i B i,K(α) (4)

[0121] Step 1.3, based on the principal component analysis method, the normal vector of the downsampled point cloud data obtained in step 1.1 is estimated; specifically:

[0122] Step 1.3.1, for the points in the initial point cloud data after downsampling, use the K nearest neighbor search method to find k points adjacent to it, k = 10. From the principal component analysis method, we can know that:

[0123]

[0124] Make the vector formed by the point and each of its neighboring points perpendicular to the normal vector. In formula (5), f(m) represents the minimum variance of the projection of all points on the constructed vector, c is the average value of all neighboring points and the point, m is the number of all points in the neighborhood, and x is the distance between the point and the normal vector. i for all points in the neighborhood;

[0125] Step 1.3.2, take c as the center point of all points in the neighborhood, that is:

[0126]

[0127] Let y i =x i -c,y i represents the decentralized point, so formula (5) is expressed as:

[0128]

[0129] Step 1.3.3, simplify formula (7) to obtain:

[0130] f(m)=minm T Sm (8)

[0131] Where S = (YY T ), Y represents the new data matrix after decentralization, st(m T m=1), S represents a 3×3 covariance matrix;

[0132] Step 1.3.4, solve the eigenvalues ​​and corresponding eigenvectors of the covariance matrix S obtained in step 1.3.3, which are λ1, λ2, λ3 and a1, a2, a3 respectively, where the eigenvector a3 corresponding to the minimum eigenvalue λ3 is the normal vector of the tangent plane at that point;

[0133] Step 1.3.5, repeat steps 1.3.1-1.3.4 to obtain the normal vectors of all points in the point cloud data and save them in the mat file;

[0134] Step 2: Design the network model structure to classify the feature points of the downsampled point cloud data obtained in step 1; specifically:

[0135] Step 2.1: Initial feature extraction is performed on the downsampled point cloud data obtained in step 1 through the four point set abstraction layers and four feature propagation layers of the PointNet++ network; specifically:

[0136] Step 2.1.1, take the downsampled point cloud data obtained in step 1 as input data and represent it as an N×3 second-order tensor, where N represents the number of point clouds, the initial number of point clouds is 8096, and 3 corresponds to the coordinate dimension;

[0137] Step 2.1.2, through the four point set abstraction layers of the PointNet++ network, first use the farthest point sampling method to get npoint center points, then use the ball query method with radius radius to find nsample points near the center point to form npoint local areas, and use the multi-layer perceptron MLP to perform initial feature extraction for each local area. The point set abstraction layer is represented by SA(npoint, radius, nsample, mlp), where npoint is the total number of sampling points, radius is the sampling radius, nsample is the number of points sampled in each sampling area, mlp is the multi-layer perceptron parameter, and the four point set abstraction layers are SA(4096, 0.05, 32, [32, 64]), SA(2048, 0.1, 32, [64, 128]), SA(1024, 0.2, 32, [128, 256]) and SA(512, 0.4, 32, [256, 512]);

[0138] Step 2.1.3, use the inverse distance weighted interpolation method to upsample the point cloud data and extract point cloud features through the four feature propagation layers of the PointNet++ network to restore the original number of point clouds. The feature propagation layer is represented by FP(mlp), where mlp is the multi-layer perceptron parameter. The four feature propagation layers are FP(256,256), FP(256,256), FP(256,128), and FP(128,128,128);

[0139] Step 2.2, embed the residual module and the attention mechanism module CASP which combines the channel attention mechanism and the spatial attention mechanism after each feature propagation layer; specifically:

[0140] In step 2.2.1, a residual module is embedded after each feature propagation layer in step 2.1 to extract features.

[0141] Step 2.2.2: For the initial features obtained in step 2.2.1, use the pooling convolution kernel to aggregate the features in the horizontal and vertical directions respectively;

[0142] Step 2.2.3, a two-dimensional convolution operation and a Sigmoid activation operation are performed in the horizontal direction and the vertical direction respectively;

[0143] Step 2.2.4, perform element-by-element multiplication of the tensor obtained in step 2.2.3 and the input feature activated by the relu function to obtain the final channel attention output feature;

[0144] Step 2.2.5: Perform maximum pooling and average pooling operations on the final channel attention output features obtained in step 2.2.4 along the channel axis, then concat the two feature information, and then perform a two-dimensional convolution operation of size 7×7 and a Maxout activation operation;

[0145] Step 2.2.6, multiply the output feature obtained in step 2.2.5 by the final channel attention output feature obtained in step 2.2.4 element by element to obtain the output feature that combines the channel attention mechanism and the spatial attention mechanism;

[0146] Step 2.3, embed the SPP spatial pyramid pooling mechanism module after the last attention mechanism module embedded in step 2.2; specifically:

[0147] Step 2.3.1, the output features obtained in step 2.2 are divided and pooled at the maximum, with the number of channels being 128, and the divided features are concatenated to finally obtain an output feature of 8096*128;

[0148] Step 2.4, the initial feature points are classified through the fully connected layer, and the probability of each point in the point cloud object belonging to an edge point or a corner point is predicted. The classification results are as follows: Figure 3 As shown; specifically:

[0149] Step 2.4.1, estimate the probability value of each point in the point cloud object belonging to an edge point or a corner point through the fully connected layer;

[0150] Step 2.4.2: Set the edge point corresponding threshold τ based on the principle that the candidate points include the required main feature points and do not contain too many non-feature points e And the corner point corresponding threshold τ c , τ e =0.75, τ c =0.95;

[0151] Step 2.4.3, taking the points whose probability values ​​obtained in step 2.4.1 are greater than the corresponding threshold as the initial feature point classification results;

[0152] Step 2.4.4, in order to reduce the generalization error in the deep neural network, data batch normalization and Dropout regularization with a drop rate of 0.5 are performed after the fully connected layer;

[0153] Step 2.5, the initial feature point classification result obtained in step 2.4 is optimized using the non-maximum suppression algorithm to obtain the final feature point classification result, such as Figure 4 As shown; specifically:

[0154] Step 2.5.1, define and initialize the sets S, C, and F in the computer, where the initial corner point data obtained in step 2.4 is stored in S, C is recorded as null, and F is recorded as null;

[0155] Step 2.5.2, use the distance threshold δ = 0.05l to divide the set S into h windows containing the corner points, where l is the length of the window diagonal;

[0156] Step 2.5.3, for window j, j = 1, 2, ..., h, where h is a natural number not equal to 0, the probability values ​​of every two corner points in j are sorted, the larger one is put into set C, and the point is deleted from S;

[0157] Step 2.5.4, take a point with the largest probability value in set C and put it into set F, and set set C to null;

[0158] Step 2.5.5, repeat steps 2.5.3-2.5.4, and the point data in set F is the final feature point classification result;

[0159] Step 3: Generate the open and close candidate curves and the closed candidate curves according to the feature points obtained in step 2; specifically:

[0160] Step 3.1: Generate a corner point combination O(M) based on the M corner points in the set F obtained in step 2. 2 ), and create the corner point pair {P i ={f i1 ,f i2}|f i1 ,f i2 ∈F}, each corner point pair will correspond to an opening and closing candidate curve;

[0161] Step 3.2, for each corner point pair, use a heuristic algorithm to find the center of the corner point (f i1 +f i2 ) / 2, the radius R is ||f i1 -f i2 Uniformly sample a subset within the sphere of || / 2 Get the corresponding opening and closing points;

[0162] Step 3.3, first embed the four point set abstraction layers and four feature propagation layers of the PointNet++ network, then embed the residual module after each feature propagation layer, the attention mechanism module CASP that combines the channel attention mechanism and the spatial attention mechanism, and embed the SPP spatial pyramid pooling mechanism module after the last feature propagation layer. Finally, for the L edge points obtained in step 2, the above network structure is used to generate the corresponding features C(•), where the parameters of the multi-layer perceptrons in the point set abstraction layer and the feature propagation layer are SA(32,32,64), SA(64,64,128), SA(128,128,256) and SA(256,256,512) and FP(256,256), FP(256,256), FP(256,128), FP(128,128,128);

[0163] Step 3.4, create a similarity matrix D∈R through a multi-layer perceptron MLP L×L ,satisfy:

[0164] D ij =||C(p i )-C(p j )||2 (9)

[0165] In formula (9), D ij Represents the characteristic C(λ i ) and the characteristic C(λ j ), and λ i With λ j is the probability value of the edge point, each row of the similarity matrix D is taken as an alternative subset, and the set As the edge point set of the mth candidate subset, is a threshold used to filter out points with large feature scores. The corresponding closed points obtained are the predicted open and closed points and the closed points. Figure 5 As shown;

[0166] Step 3.5, apply the opening and closing points obtained in step 3.2 to the PointNet network, using two MLP layers with parameters of [128, 256, 512] and [256, 256] respectively. Then, through the fully connected layer and the loss function, determine whether a specific point belongs to a candidate curve, identify the type of the candidate curve (straight line, circle or B-spline curve), and predict the parameters of the candidate curve to obtain the opening and closing candidate curves.

[0167] Step 3.6, through three points β = {p a +Δ a ,p b +Δ b ,p c +Δc} to parameterize each circular closed candidate curve, and the edge point set F obtained in step 3.4 m Perform the farthest point sampling and use p a =p m Initialize and get {p a ,p b ,p c};

[0168] Step 3.7, set F m Applied to the PointNet network, the closed candidate curve parameters are predicted and the confidence is estimated through the fully connected layer and loss function to obtain the closed candidate curve. The candidate curve extraction results are as follows: Figure 6 As shown;

[0169] Step 4: Select the best curve based on the open and close candidate curves and the closed candidate curves obtained in step 3; specifically:

[0170] Step 4.1, given two sets of points associated with the two candidate opening and closing curves obtained in step 3, measure the degree of overlap of the two sets of points by O(A,B)=max{I(A,B) / A,I(A,B) / B}, where I(A,B) is the cardinality of the intersection of the two sets of points;

[0171] Step 4.2, if O(A,B) is greater than the threshold τ determined by hyperparameter tuning o , then merge the two sets of points and select the best opening and closing curve by retaining the one with the larger cardinality, where τ o =0.8;

[0172] Step 4.3, for two closed candidate curves, when IoU(A,B)>τ iou When , hierarchical clustering is performed on closed candidate curves, where τ iou Set to 0.6;

[0173] Step 4.4: For each cluster obtained in step 4.3, the matching score of the candidate curve is measured by the angle distance, and finally the best closed curve is selected from the clusters with the highest confidence. The best curve extraction result is as follows: Figure 7 shown.

Claims

1. A parameterized edge curve extraction method for a point cloud object, characterized in that: The specific steps are as follows: Step 1: downsample the original point cloud data using the farthest point sampling algorithm to obtain downsampled point cloud data, then parameterize the point cloud data and obtain the normal vector of each point in the point cloud data based on the principal component analysis method; Step 2: Design a network model structure to classify feature points of the downsampled point cloud data obtained in step 1. The step 2 is specifically as follows: Step 2.1, perform initial feature extraction on the downsampled point cloud data obtained in step 1 through four point set abstraction layers and four feature propagation layers of the PointNet++ network; Step 2.2, embed the residual module and the attention mechanism module CASP which combines the channel attention mechanism and the spatial attention mechanism after each feature propagation layer; Step 2.3, embed the SPP spatial pyramid pooling mechanism module after the last attention mechanism module embedded in step 2.2; Step 2.4, classify the initial feature points through the fully connected layer, and predict the probability of each point in the point cloud object belonging to an edge point or a corner point; Step 2.5, the initial feature point classification result obtained in step 2.4 is optimized using the non-maximum suppression algorithm to obtain the final feature point classification result; Step 3, generating an open and closed candidate curve and a closed candidate curve according to the feature points obtained in step 2; Step 4, selecting the best curve according to the open and close candidate curves and the closed candidate curves obtained in step 3; The step 4 is specifically as follows: Step 4.1, given two sets of points associated with the two candidate opening and closing curves obtained in step 3, measure the degree of overlap of the two sets of points by O(A,B)=max{I(A,B) / A,I(A,B) / B}, where I(A,B) is the cardinality of the intersection of the two sets of points; Step 4.2, if O(A,B) is greater than the threshold τ determined by hyperparameter tuning o , then merge the two sets of points and select the best opening and closing curve by retaining the one with the larger cardinality, where τ o =0.8; Step 4.3, for two closed candidate curves, when IoU(A,B)>τ iou When , hierarchical clustering is performed on closed candidate curves, where τ iou Set to 0.6; Step 4.4: For each cluster obtained by clustering in step 4.3, the matching score of the candidate curve is measured by the angle distance, and finally the best closed curve is selected from the clusters obtained by clustering with the highest confidence.

2. The method for extracting a parameterized edge curve of a point cloud object according to claim 1, characterized in that: The step 1 is specifically as follows: Step 1.1, down-sampling the original point cloud data by using the farthest point sampling algorithm to obtain down-sampled point cloud data; Step 1.2, parameterizing the downsampled point cloud data obtained in step 1.1; Step 1.3, estimate the normal vector of the downsampled point cloud data obtained in step 1.1 based on the principal component analysis method.

3. The method for extracting a parameterized edge curve of a point cloud object according to claim 2, characterized in that: The step 1.1 is specifically as follows: Step 1.1.1, define and initialize sets A, B, and sample_data in the computer, where A is recorded as null, the original point cloud data is stored in B, and sample_data is recorded as null; Step 1.1.2, randomly sort set B, select the first point, and store it in set A; Step 1.1.3, calculate the distance from each point in set B to the midpoint of set A, select the point with the largest distance, and store it in set A; Step 1.1.4, calculate the distances from point pB in set B to the midpoints in set A, and select the smallest distance value from the calculated distance values ​​as the distance from point pB to set A; Step 1.1.5, for all points in set B, calculate a distance value for each point according to step 1.1.4, select the point corresponding to the maximum distance value and store it in set A; Step 1.1.6, repeat steps 1.1.4-1.1.5 until 8096 points are selected and saved to sample_data; The step 1.2 is specifically as follows: Step 1.2.1, parameterize the straight line by connecting two corner points; Step 1.2.2, parameterize the circle as: β=(p1,p2,p3)(1) In formula (1), p1, p2, and p3 are any three non-collinear points on the circle. Formula (1) can be expressed using the normal, center, and radius of the circle as follows: β=(n,c,r)(2) In formula (2), n is the normal of the circle, c is the center of the circle, and r is the radius of the circle. Next, randomly sample points on the circle and express them as a function of β to generate random samples: p(α|β)=c+r(ucos(α)+vsin(α)) (3) In formula (3), α is a randomly sampled point on the circle, α∈[0,2π], and u=p1-c, v=u×n; Step 1.2.3, using four control points To parameterize the B-spline curve, where p i It is the control vertex that represents the change of the curve, and B i,K(·) To represent the K-order B-spline curve of the i-th order basis function, and uniformly sample points α, α∈[0,1], to generate uniform random samples: p(α|β)=∑ i p i B i,K(α) (4) The step 1.3 is specifically as follows: Step 1.3.1, for the points in the initial point cloud data after downsampling, use the K nearest neighbor search method to find k points adjacent to it, k = 10. From the principal component analysis method, we can know that: Make the vector formed by the point and each of its neighboring points perpendicular to the normal vector. In formula (5), f(m) represents the minimum variance of the projection of all points on the vector formed, c is the average value of all neighboring points and the point, k is the number of all points in the neighborhood, and x is the distance between the point and the normal vector. i for all points in the neighborhood; Step 1.3.2, take c as the center point of all points in the neighborhood, that is: Let y i =x i -c,y i represents the decentralized point, so formula (6) is expressed as: Step 1.3.3, simplify formula (7) to obtain: f(m)=min m T Sm(8) Where S = (YY T ), Y represents the new data matrix after decentralization, st(m T m=1), S represents a 3×3 covariance matrix; Step 1.3.4, solving the eigenvalues ​​and corresponding eigenvectors of the covariance matrix S obtained in step 1.3.3 are λ1, λ2, λ3 and a1, a2, a3 respectively, wherein the eigenvector a3 corresponding to the minimum eigenvalue λ3 is the normal vector of the tangent plane at the point; Step 1.3.5, repeat steps 1.3.1-1.3.4 to obtain the normal vectors of all points in the point cloud data and save them in the mat file.

4. The method for extracting a parameterized edge curve of a point cloud object according to claim 1, characterized in that: The step 2.1 is specifically as follows: Step 2.1.1, taking the downsampled point cloud data obtained in step 1 as input data, and representing it as an N×3 second-order tensor, where N represents the number of point clouds, the initial number of point clouds is 8096, and 3 corresponds to the coordinate dimension; Step 2.1.2, through the four point set abstraction layers of the PointNet++ network, first use the farthest point sampling method to get npoint center points, then use the ball query method with radius radius to find nsample points near the center point to form npoint local areas, and use the multi-layer perceptron MLP to perform initial feature extraction for each local area. The point set abstraction layer is represented by SA(npoint, radius, nsample, mlp), where npoint is the total number of sampling points, radius is the sampling radius, nsample is the number of points sampled in each sampling area, mlp is the multi-layer perceptron parameter, and the four point set abstraction layers are SA(4096, 0.05, 32, [32, 64]), SA(2048, 0.1, 32, [64, 128]), SA(1024, 0.2, 32, [128, 256]) and SA(512, 0.4, 32, [256, 512]); Step 2.1.3, use the inverse distance weighted interpolation method to upsample the point cloud data and extract point cloud features through the four feature propagation layers of the PointNet++ network to restore the original number of point clouds. The feature propagation layer is represented by FP(mlp), where mlp is the multi-layer perceptron parameter. The four feature propagation layers are FP(256,256), FP(256,256), FP(256,128), and FP(128,128,128); The step 2.2 is specifically as follows: Step 2.2.1, embedding a residual module after each feature propagation layer in step 2.1 to extract features; Step 2.2.2, for the initial features obtained in step 2.2.1, use the pooling convolution kernel to aggregate the features in the horizontal direction and the vertical direction respectively; Step 2.2.3, a two-dimensional convolution operation and a Sigmoid activation operation are performed in the horizontal direction and the vertical direction respectively; Step 2.2.4, perform element-by-element multiplication of the tensor obtained in step 2.2.3 and the input feature activated by the relu function to obtain the final channel attention output feature; Step 2.2.5, the final channel attention output feature obtained in step 2.2.4 is subjected to maximum pooling and average pooling operations along the channel axis, and then the two feature information are concat-connected, and then subjected to a two-dimensional convolution operation of size 7×7 and a Maxout activation operation; Step 2.2.6, multiply the output feature obtained in step 2.2.5 by the final channel attention output feature obtained in step 2.2.4 element by element to obtain the output feature that combines the channel attention mechanism and the spatial attention mechanism.

5. The method for extracting a parameterized edge curve of a point cloud object according to claim 1, characterized in that: The step 2.3 is specifically as follows: Step 2.3.1, the output features obtained in step 2.2 are divided and maximum pooled, the number of channels is 128, the features obtained by division are spliced, and finally an output feature of 8096*128 is obtained.

6. The method for extracting a parameterized edge curve of a point cloud object according to claim 1, characterized in that: The step 2.4 is specifically as follows: Step 2.4.1, estimate the probability value of each point in the point cloud object belonging to an edge point or a corner point through the fully connected layer; Step 2.4.2: Set the edge point corresponding threshold τ based on the principle that the candidate points include the required main feature points and do not contain too many non-feature points e And the corner point corresponding threshold τ c , τ e =0.75, τ c =0.95; Step 2.4.3, taking the points whose probability values ​​obtained in step 2.4.1 are greater than the corresponding threshold as the initial feature point classification results; In step 2.4.4, in order to reduce the generalization error in the deep neural network, data batch normalization and Dropout regularization with a drop rate of 0.5 are performed after the fully connected layer.

7. The method for extracting a parameterized edge curve of a point cloud object according to claim 1, characterized in that: The step 2.5 is specifically as follows: Step 2.5.1, define and initialize the sets S, C, and F in the computer, where the initial corner point data obtained in step 2.4 is stored in S, C is recorded as null, and F is recorded as null; Step 2.5.2, use the distance threshold δ = 0.05l to divide the set S into h windows containing the corner points, where l is the length of the window diagonal; Step 2.5.3, for window j, j = 1, 2, ..., h, where h is a natural number not equal to 0, the probability values ​​of every two corner points in j are sorted, the larger one is put into set C, and the point is deleted from S; Step 2.5.4, take a point with the largest probability value in set C and put it into set F, and set set C to null; Step 2.5.5, repeat steps 2.5.3-2.5.4, and the point data in set F is the final feature point classification result.

8. The method for extracting a parameterized edge curve of a point cloud object according to claim 1, characterized in that: The step 3 is specifically as follows: Step 3.1: Generate a corner point combination O(M) based on the M corner points in the set F obtained in step 2. 2 ), and create the corner point pair {P i ={f i1 ,f i2 } i1 ,f i2 ∈F}, each corner point pair will correspond to an opening and closing candidate curve; Step 3.2, for each corner point pair, use a heuristic algorithm to find the center of the corner point (f i1 +f i2 ) / 2, the radius R is ||f i1 -f i2 Uniformly sample a subset within the sphere of || / 2 Get the corresponding opening and closing points; Step 3.3, first embed the four point set abstraction layers and four feature propagation layers of the PointNet++ network, then embed the residual module after each feature propagation layer, the attention mechanism module CASP that combines the channel attention mechanism and the spatial attention mechanism, and embed the SPP spatial pyramid pooling mechanism module after the last feature propagation layer. Finally, for the L edge points obtained in step 2, the above network structure is used to generate the corresponding features C(·), where the multi-layer perceptron parameters in the point set abstraction layer and the feature propagation layer are SA(32,32,64), SA(64,64,128), SA(128,128,256) and SA(256,256,512) and FP(256,256), FP(256,256), FP(256,128), FP(128,128,128); Step 3.4, create a similarity matrix D∈R through a multi-layer perceptron MLP L×L , should meet the following requirements: D ij =||C(p i )-C(p j )||2 (9) In formula (9), D ij Represents the characteristic C(λ i ) and the characteristic C(λ j ), and λ i With λ j is the probability value of the edge point, each row of the similarity matrix D is taken as an alternative subset, and the set As the edge point set of the mth candidate subset, is a threshold used to filter out points with greatly different feature scores, and the corresponding closed points are obtained; Step 3.5, applying the opening and closing points obtained in step 3.2 to the PointNet network, wherein two MLP layers are used, and the parameters are [128, 256, 512] and [256, 256] respectively, and then a fully connected layer and a loss function are used to determine whether a specific point belongs to a candidate curve, identify the type of the candidate curve (straight line, circle or B-spline curve), and predict the parameters of the candidate curve to obtain the opening and closing candidate curve; Step 3.6, through three points β = {p a +Δ a ,p b +Δ b ,p c +Δ c } to parameterize each circular closed candidate curve, and the edge point set F obtained in step 3.4 m Perform the farthest point sampling and use p a =p m Initialize and get {p a ,p b ,p c }; Step 3.7, set F m Applied to the PointNet network, the closed candidate curve parameters are predicted and the confidence is estimated through the fully connected layer and loss function to obtain the closed candidate curve.

Images