Crankshaft point cloud semantic segmentation method

Abstract

The invention discloses a crankshaft point cloud semantic segmentation method. The method comprises the steps of performing voxelization downsampling on a crankshaft point cloud model and establishing a Kd-Tree topological structure; calculating normal lines under two different support radiuses, adjusting the direction of the normal lines, and calculating a normal line differential (DoN) characteristic according to a normal line difference; obtaining the number of point neighborhoods with a small support radius, and calculating point cloud curvature and FPFH features; defining an initial target area by taking a point with the minimum number of neighborhood points as an initial seed point; searching a k neighbor point set of the seed points, taking an FPFH feature threshold value and a curvature threshold value as region growth criteria, and performing semantic judgment on a segmented region by using a DoN feature threshold value; when the neighborhood point meeting the growth criterion cannot be found, completing the growth of the current region and continuing the growth of the next region until the whole point cloud is traversed; and completing crankshaft point cloud semantic segmentation according to the number of the marked region points. The method is low in complexity, can accurately complete crankshaft point cloud semantic segmentation, and improves the universality of a crankshaft point cloud segmentation algorithm.

Description

Technical Field

[0001] The invention relates to the field of point cloud data processing, and in particular to a crankshaft point cloud semantic segmentation method. Background Art

[0002] The crankshaft is an important part of the engine, the heart of high-end equipment such as marine ships and automobiles. Precision inspection of the crankshaft is a prerequisite for ensuring engine performance. With the development of 3D scanning technology, 3D measuring equipment can obtain surface information of the object being measured more accurately and comprehensively, and has the advantages of high efficiency, high precision, and strong initiative. Combining 3D measuring equipment with online measurement systems has also become a favorable means of crankshaft inspection. The premise of using 3D point cloud models to detect parts is to segment the parts to be inspected and then complete feature extraction. The crankshaft journal is one of the key components in rotating machinery, and the detection of its size plays an important role in improving performance after assembly and extending service life. Therefore, when using a point cloud model to inspect the crankshaft, segmenting the crankshaft point cloud model to extract the journal part is a key link in the processing of crankshaft point cloud data.

[0003] Traditional region growing algorithms, clustering-based segmentation methods, and model fitting-based segmentation methods do not have semantic segmentation functions when segmenting point cloud models. In the subsequent crankshaft journal measurement, manual judgment of the segmentation area type is required. Point cloud segmentation based on deep learning models requires a large amount of training data, and manual annotation of training data and large amount of calculation make it difficult to apply. In addition, when collecting crankshaft point cloud data, due to the complex morphology of the crankshaft, the environment, and the influence of the crankshaft surface material, the collected point cloud data has holes and breaks between point clouds, which makes the point cloud segmentation more difficult when using the segmentation algorithm, and the segmented area is further increased. In addition, the segmented area without semantics will also affect the subsequent journal feature extraction. Summary of the invention

[0004] The purpose of the present invention is to provide a crankshaft point cloud semantic segmentation method, which can accurately complete the crankshaft point cloud semantic segmentation with low complexity and improve the universality of the crankshaft point cloud segmentation algorithm.

[0005] In order to achieve the above object, the present invention adopts the following solution:

[0006] A method for semantic segmentation of crankshaft point cloud comprises the following steps:

[0007] Step M1: Set the voxel edge length, downsample the crankshaft point cloud model voxel-wise and establish a Kd-Tree topology structure;

[0008] Step M2: For each downsampled point, use the principal component analysis method to calculate the maximum support radius r l , small support radius r sThe normal under the , adjust the normal direction, and calculate the normal differential (DoN) feature with the normal difference;

[0009] Step M3: With small support radius r s Get the number of point neighbors for the search radius and calculate the curvature and FPFH features of the point cloud;

[0010] Step M4: Use the point with the least number of neighborhood points as the initial seed point to define the initial target area;

[0011] Step M5: Search for the k-nearest neighbor point set of the seed point, use the FPFH feature threshold and the curvature threshold as the region growth criterion, and use the DoN feature threshold to make semantic judgments on the segmented region;

[0012] Step M6: When no neighboring points that meet the growth criteria can be found, the current region growth is completed and the next region growth is continued until the entire point cloud is traversed;

[0013] Step M7: Complete the semantic segmentation of the crankshaft point cloud according to the number of regional point markers.

[0014] The differential normal (DoN) feature in step M2 is calculated as follows:

[0015]

[0016] Where p is a point in the point cloud model after downsampling, For point p at the small support radius r s The normal line below, For point p at the large support radius r l The normal line below.

[0017] The step M4 of selecting the point with the least number of neighborhood points as the initial seed point is as follows: for the crankshaft point cloud model point set P = {p j ,j=1,2,...,N p}, click p i The number of neighboring points is like where k∈(1,2,...,i-1,i+1,...,N p ), then point p i is the point with the least number of neighborhood points, and p i as the initial seed point.

[0018] The region growth criterion in step M5 is specifically: the seed point p i The neighboring point p ij The FPFH eigenvalue f ij With the curvature value C ij All meet the FPFH feature threshold F threshold With the curvature threshold Cthreshold Mathematical relationship: f ij <F threshold , C ij <C threshold When the neighboring point p ij Add the growth area and update the seed point: point p ij As the new seed point of the growth area, and delete the previous seed point p i , otherwise re-find the seed point p i New neighborhood points.

[0019] In step M5, the DoN feature threshold performs semantic judgment on the segmented area, specifically including the following steps:

[0020] Step 5.1: Determine the seed point p i The neighboring point p ij The DoN eigenvalue D ij and DoN feature threshold D threshold Satisfied: D ij >D threshold ;

[0021] Step 5.2: If satisfied, then the current growth area Q i Number of non-journal points in the area m i Increase by 1, otherwise area Q i Number of journal points in the area n i Increase by 1.

[0022] In step M7, semantic segmentation of the crankshaft point cloud is completed according to the number of regional point markers, which specifically includes the following steps:

[0023] Step 7.1: Determine the segmented area Q i Whether it satisfies m i With n i The mathematical relationship is:

[0024]

[0025] In the formula, m i For region Q i The total number of non-journal points, n i For region Q i The total number of points of the middle journal point;

[0026] Step 7.2: If the above relationship is satisfied, determine the segmentation area Q i is the journal area, otherwise it is the shoulder area;

[0027] Step 7.3: After determining the types of all segmented regions, the semantic segmentation of the crankshaft point cloud is completed.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1) The present invention combines the normal differential features, FPFH features and curvature features of the point cloud to perform semantic segmentation on the crankshaft point cloud model. While utilizing the point cloud model with comprehensive information and high precision, it avoids the problems of high algorithm design and large amount of calculation, and improves the universality of the crankshaft point cloud segmentation algorithm;

[0030] 2) The present invention can accurately complete the semantic segmentation of the crankshaft point cloud. Through the semantic information, the area that does not require subsequent processing can be eliminated, and the key journal area to be measured can be retained, thereby avoiding the influence of the segmentation error of unnecessary point cloud information on the subsequent journal size measurement, which is beneficial to the subsequent journal size detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of a crankshaft point cloud semantic segmentation method according to the present invention;

[0032] Figure 2 This is a flow chart of completely processing the crankshaft point cloud in the embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0034] Taking the engine crankshaft point cloud model as the object, a contour scanner based on the laser triangulation principle is selected. A robot is selected to match the contour scanner to obtain the three-dimensional point cloud data of the part. By setting the robot arm path planning, the crankshaft three-dimensional point cloud data is collected.

[0035] Since one scan can only obtain one-view point cloud of the crankshaft, the complete crankshaft point cloud surface feature data is obtained by sequentially aligning the collected multi-view local point clouds of the crankshaft.

[0036] The steps to obtain the complete crankshaft point cloud model are as follows:

[0037] 1) The intrinsic shape feature (ISS) method is used to identify the point cloud features of the crankshaft surface and extract the key feature points of the point cloud;

[0038] 2) Establish FPFH feature description of key points;

[0039] 3) Select one of the multi-view crankshaft point clouds as the target point cloud, select another view point cloud adjacent to the target point cloud as the source point cloud, and import the point cloud;

[0040] 4) Use the Kd-Tree method to search for the FPFH features of each key point in the target point cloud and the source point cloud, and find the best matching candidate set of each feature point in the source point cloud in the target point cloud based on the similarity of the FPFH features;

[0041] 5) Estimate the initial rigid transformation by randomly sampling and matching candidate set point pairs, select the best initial alignment result using the sampling consistency algorithm, and complete the point cloud coarse registration;

[0042] 6) Use the Tr-ICP algorithm to complete the precise registration of two point clouds;

[0043] 7) Based on the coarse alignment, the remaining viewpoint point clouds are aligned successively, and the point cloud data are integrated into a unified world coordinate system. The point clouds of multiple parts are continuously aligned through a coarse-fine method to obtain a complete three-dimensional point cloud model of the crankshaft.

[0044] The candidate set sampling points of the source point cloud of the traditional sampling consistent initialization (SAC-IA) algorithm are random. The FPFH features of key points are used to roughly align the point cloud data, which improves the effectiveness of the SAC-IA algorithm alignment.

[0045] After obtaining the complete 3D point cloud model of the crankshaft, perform semantic segmentation of the point cloud model:

[0046] Step M1: Set the voxel edge length, voxelize and downsample the crankshaft point cloud model and establish a Kd-Tree topology structure.

[0047] Step M2: For each downsampled point, use the principal component analysis method to calculate the maximum support radius r l , small support radius r s The normals under the , adjust the normal direction, and calculate the normal differential (DoN) feature with the normal difference.

[0048] The normal is calculated as follows:

[0049] (1) Get a point p in the point cloud i (i∈[1,N p ]) and its neighboring point p ij (j∈[1,k]), calculate the centroid of the neighborhood point set

[0050]

[0051] (2) Calculate each neighborhood point p i With the centroid The vectors between and find the covariance matrix M of the vectors:

[0052]

[0053] (3) Decompose the eigenvalues ​​of the covariance matrix M to obtain its eigenvectors v1, v2, v3 and the corresponding eigenvalues ​​λ1, λ2, λ3. The eigenvector corresponding to the minimum eigenvalue is the normal vector of the point.

[0054] The adjustment of the normal direction is specifically as follows: for any point p in the point cloud i , search for its neighboring point p ij , if p i The normal n i and p ij The normal n ij The dot product is less than 0, that is, n i ·n ij <0, then point p ij The normals are adjusted in the opposite direction.

[0055] The Differential of Normal (DoN) feature is calculated as follows:

[0056]

[0057] Where p is a point in the point cloud model after downsampling, For point p at the small support radius r s The normal line below, For point p at the large support radius r l The normal line below.

[0058] Step M3: With small support radius r s Get the number of point neighbors for the search radius and calculate the curvature and FPFH features of the point cloud.

[0059] The calculation formula of the curvature of the point cloud is as follows:

[0060]

[0061] Where λ1, λ2, and λ3 are the eigenvalues ​​of the covariance matrix M decomposed in the normal calculation, and λ min is the smallest eigenvalue.

[0062] The calculation of the FPFH feature of the point cloud is as follows:

[0063] (1) For a given query point p q , calculate the positional relationship between it and the neighboring points, and generate a simplified point feature histogram (SPFH);

[0064] (2) Take the neighborhood points in the previous step as new query points, further determine the respective neighborhood point sets, and use the simplified feature histogram of the neighborhood points and the positional relationship of the original query points to obtain the query point p through weighted calculation. q A quick point feature histogram description of:

[0065]

[0066] In the formula, point p qj For p q Points within the neighborhood, weight w kFor p q and p qj The distance between.

[0067] Step M4: Use the point with the least number of neighborhood points as the initial seed point to define the initial target area.

[0068] The point with the least number of neighboring points is selected as the initial seed point: for the crankshaft point cloud model point set P = {p j ,j=1,2,...,N p}, click p i The number of neighboring points is n pi , if n pi <n pk ,where k∈(1,2,...,i-1,i+1,...,N p ), then point p i is the point with the least number of neighborhood points, and p i As the initial seed point, the point p with the least number of neighboring points i It is located at the edge of the point cloud model, and can avoid under-segmentation and over-segmentation to a certain extent when segmenting the target point cloud.

[0069] Step M5: Search for the k-nearest neighbor point set of the seed point, use the FPFH feature threshold and the curvature threshold as the region growing criteria, and use the DoN feature threshold to perform semantic judgment on the segmented region.

[0070] The region growth criterion is specifically: the seed point p i The neighboring point p ij The FPFH eigenvalue f ij With the curvature value C ij All meet the FPFH feature threshold F threshold With the curvature threshold C threshold Mathematical relationship: f ij <F threshold , C ij <C threshold When the neighboring point p ij Add the growth area and update the seed point: point p ij As the new seed point of the growth area, and delete the previous seed point p i , otherwise re-find the seed point p i New neighborhood points.

[0071] The DoN feature threshold performs semantic judgment on the segmented area, specifically including the following steps:

[0072] (1) Determine the seed point p i The neighboring point p ij The DoN eigenvalue D ij and DoN feature threshold Dthreshold Satisfied: D ij >D threshold ;

[0073] (2) If satisfied, then the current growth area Q i Number of non-journal points in the area m i Increase by 1, otherwise area Q i Number of journal points in the area n i Increase by 1.

[0074] Step M6: When no neighborhood points satisfying the growth criteria can be found, the current region growing is completed and the next region growing is continued until the entire point cloud is traversed.

[0075] Step M7: Complete the semantic segmentation of the crankshaft point cloud according to the number of regional point markers.

[0076] The semantic segmentation of the crankshaft point cloud is completed according to the number of regional point marks, and specifically includes the following steps:

[0077] (1) Determine the segmented area Q i Whether it satisfies m i With n i The mathematical relationship is:

[0078]

[0079] In the formula, m i For region Q i The total number of non-journal points, n i For region Q i The total number of points of the middle journal point;

[0080] (2) If the above relationship is satisfied, the segmentation area Q is determined. i is the journal area, otherwise it is the shoulder area;

[0081] (3) After determining the types of all segmented areas, the semantic segmentation of the crankshaft point cloud is completed.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for semantic segmentation of crankshaft point cloud, comprising the following steps: Step M1: Set the voxel edge length, downsample the crankshaft point cloud model voxel-wise and establish a Kd-Tree topology structure; Step M2: For each downsampled point, use the principal component analysis method to calculate the maximum support radius r l , small support radius r s The normal under the , adjust the normal direction, and calculate the normal differential (DoN) feature with the normal difference; Step M3: With small support radius r s Get the number of point neighbors for the search radius and calculate the curvature and FPFH features of the point cloud; Step M4: Use the point with the least number of neighborhood points as the initial seed point to define the initial target area; Step M5: Search for the k-nearest neighbor point set of the seed point, use the FPFH feature threshold and the curvature threshold as the region growth criterion, and use the DoN feature threshold to make semantic judgment on the segmented region; Step M6: When no neighboring points that meet the growth criteria can be found, the current region growth is completed and the next region growth is continued until the entire point cloud is traversed; Step M7: Complete the semantic segmentation of the crankshaft point cloud according to the number of regional point markers.

2. The method for semantic segmentation of crankshaft point cloud according to claim 1, characterized in that: The differential normal (DoN) feature in step M2 is calculated as follows: Where p is a point in the point cloud model after downsampling, For point p at the small support radius r s The normal line below, For point p at the large support radius r l The normal line below.

3. The crankshaft point cloud semantic segmentation method according to claim 1, characterized in that: The step M4 of selecting the point with the least number of neighborhood points as the initial seed point is as follows: for the crankshaft point cloud model point set P = {p j ,j=1,2,...,N p }, click p i The number of neighboring points is n pi , if n pi <n pk ,where k∈(1,2,...,i-1,i+1,...,N p ), then point p i is the point with the least number of neighborhood points, and p i as the initial seed point.

4. The method for semantic segmentation of crankshaft point cloud according to claim 1, characterized in that: The region growth criterion in step M5 is specifically: the seed point p i The neighboring point p ij The FPFH eigenvalue f ij With the curvature value C ij All meet the FPFH feature threshold F threshold With the curvature threshold C threshold Mathematical relationship: f ij <F threshold , C ij <C threshold When the neighboring point p ij Add the growth area and update the seed point: point p ij As the new seed point of the growth area, and delete the previous seed point p i , otherwise re-find the seed point p i New neighborhood points.

5. The method for semantic segmentation of crankshaft point cloud according to claim 1, characterized in that: In step M5, the DoN feature threshold performs semantic judgment on the segmented area, specifically including the following steps: Step 5.1: Determine the seed point p i The neighboring point p ij The DoN eigenvalue D ij and DoN feature threshold D threshold Satisfied: D ij >D threshold ; Step 5.2: If satisfied, then the current growth area Q i Number of non-journal points in the area m i Increase by 1, otherwise area Q i Number of journal points in the area n i Increase by 1.

6. The method for semantic segmentation of crankshaft point cloud according to claim 1, characterized in that: In step M7, semantic segmentation of the crankshaft point cloud is completed according to the number of regional point markers, which specifically includes the following steps: Step 7.1: Determine the segmented area Q i Whether it satisfies m i With n i The mathematical relationship is: In the formula, m i For region Q i The total number of non-journal points, n i For region Q i The total number of points of the middle journal point; Step 7.2: If the above relationship is satisfied, determine the segmentation area Q i is the journal area, otherwise it is the shoulder area; Step 7.3: After determining the types of all segmented regions, the semantic segmentation of the crankshaft point cloud is completed.

Images