Complex part scanning area planning method based on matching key point feature intensity
By using a deep learning-based point cloud matching model and hierarchical clustering algorithm, the key point feature intensity of complex parts is obtained, and the scanning area is planned. This solves the problems of low accuracy and efficiency in point cloud matching of complex parts and achieves efficient point cloud acquisition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN UNIV OF TECH
- Filing Date
- 2023-05-16
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies suffer from poor stability in feature extraction during point cloud matching of complex parts, making it difficult to cope with noise and deformation. Furthermore, the complex operation results in low matching accuracy and efficiency.
A point cloud matching model based on deep learning is adopted. By obtaining the feature intensity of matching key points, the point cloud data is segmented using a hierarchical clustering algorithm, the scanning area is planned, and areas with high feature intensity are selected for key collection.
It improves point cloud matching accuracy and acquisition efficiency, reduces hardware memory consumption and computational complexity, is highly adaptable, and can effectively cope with the feature differences of complex parts.
Smart Images
Figure CN116758003B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of laser point cloud acquisition technology, specifically to a method for planning the scanning area of complex parts based on matching the feature intensity of key points. Background Technology
[0002] Laser point cloud acquisition technology is a technique that uses a laser scanner to acquire three-dimensional information about the surface of an object. It has wide applications in industrial inspection, 3D reconstruction, and robot measurement-machining integration. Especially in robot measurement-machining integration, the main purpose of laser point cloud acquisition is to match the point cloud obtained by scanning with the workpiece CAD model, thereby achieving accurate workpiece positioning and determining the machining allowance, thus guiding the robot's precision machining.
[0003] Keypoints are points in a 3D point cloud that possess significant geometric features and repeatability. Essentially, a small number of keypoints are sufficient for successful point cloud alignment, and well-positioned keypoints can further improve registration accuracy. Traditional feature-based point cloud matching algorithms primarily utilize manually designed local feature descriptors extracted from the point cloud, followed by the use of the Random Sample Consensus Algorithm (RANSAC) to eliminate incorrectly matched point pairs. Manually designed local features require designers to possess extensive domain knowledge and experience, increasing design costs and limiting feature versatility. Furthermore, manually designed features are susceptible to noise and deformation, resulting in poor feature descriptor stability, a lack of adaptability and learning ability, and difficulty in applying them to other domains and tasks.
[0004] In recent years, with the rapid development of deep learning technology, point cloud network models based on deep learning have been gradually promoted. Deep learning-based point cloud feature extraction models can automatically learn feature representation methods from large amounts of data, and the feature extraction process is adaptive. Therefore, for complex parts in industrial inspection, a suitable scanning area planning method is needed to improve point cloud acquisition efficiency and point cloud matching accuracy. Patent application CN 202310051345 discloses a method and system for scanning UAV point cloud data in rainy conditions. It utilizes the layered characteristics of raindrop interference during rain, setting corresponding scanning layers and scanning angles to improve the efficiency and accuracy of point cloud data acquisition. However, this method requires pre-analysis of the structural properties of the target object and manual calibration of flat areas and symmetrical points, increasing the workload and error of operators. Furthermore, this method does not consider other types of noise points, such as weak reflection signals caused by the surface material of the object or lighting conditions. The invention patent with application number CN 202010341002.2 discloses a 3D point cloud recognition method based on the local surface feature histogram of key points and spatial relationships. First, the point cloud is automatically filtered by moving regions. Then, key points are found based on the mean curvature. Feature descriptors are calculated based on the local surface histogram. Feature matching is performed based on the key point descriptors and spatial relationships. Finally, multi-threaded recognition is performed using a global container and overlap rate. However, this method is not robust enough to handle noise, occlusion, or deformation in the point cloud, and it cannot handle large-scale or complex scenes. Summary of the Invention
[0005] The purpose of this invention is to provide a method for planning the scanning region of complex parts based on the feature intensity of matching key points. This method can solve the problem of different feature intensities in different regions of complex parts, which leads to differences in point cloud matching accuracy. It achieves the effect of evaluating the feature intensity of point clouds to improve matching accuracy and can improve the efficiency of point cloud acquisition.
[0006] This invention is implemented as follows:
[0007] A method for planning the scanned region of a complex part based on the feature intensity of matching key points includes the following steps:
[0008] S1. Obtain point cloud data: Sample the CAD model of the part to be scanned to obtain discrete point cloud on the surface of the part;
[0009] S2. Obtain matching key points: Input the discrete point cloud of the part surface into the point cloud matching model based on deep learning to obtain several matching key points of the discrete point cloud of the part surface and their corresponding matching weights.
[0010] S3. Obtain matching key clusters: Select N matching key points with larger matching weights and cluster them to obtain M matching key clusters after clustering;
[0011] S4. Obtain the scanning point cloud planning region: Based on the matching key clusters, the scanning region is planned for the discrete point cloud on the surface of the part, and M candidate point cloud scanning regions and their corresponding feature intensity coefficients are obtained.
[0012] A further step is:
[0013] In step S1, the sampling method for the part's CAD model is uniform sampling, which involves uniformly extracting points from the 3D curved surface of the part's CAD model and then downsampling to control the final point cloud quantity. In general practical operation, it is necessary to ensure that the number of point clouds input to the network does not exceed 100,000, while also not being too few; the number of point clouds generally needs to be at least more than 5,000. Therefore, during the sampling process, different sampling densities are set according to the size of the part to control the required number of point clouds.
[0014] A further step is:
[0015] In step S2, the point cloud matching model based on deep learning is the D3Feat point cloud matching model.
[0016] A further step is:
[0017] In step S2, the evaluation method for the matching weight of the matching key point is that the higher the weight of the matching key point, the stronger the feature information of the matching key point.
[0018] A further step is:
[0019] In step S3, the clustering algorithm is a hierarchical clustering algorithm.
[0020] A further step is:
[0021] In step S3, the hierarchical clustering algorithm is a bottom-up aggregation method, which treats each data point as an initial cluster and then gradually merges the most similar clusters into larger clusters until the set clustering termination condition is met.
[0022] A further step is:
[0023] In step S3, the constraint condition for the hierarchical clustering algorithm is: the maximum lateral area of the minimum bounding box of the matching key clusters is equal to the threshold. At that point, the cluster that matches the key category stops further clustering. This represents the maximum scan area that the scanner can capture at the optimal imaging distance.
[0024] A further step is:
[0025] In step S4, the scanning area planning method is as follows:
[0026] Let the discrete point cloud on the surface of the part be denoted as The key matching cluster is ,
[0027] (1) For key matching clusters , obtain Minimum directional bounding box ;
[0028] (2) Discretize the point cloud on the surface of the part Located in the smallest direction bounding box The inner part is denoted as point cloud. ;
[0029] (3) Point cloud based on principal component analysis Projected onto a two-dimensional plane , will dot clouds In plane The area of the minimum bounding rectangle of the upper projection is denoted as ;
[0030] (4) Clustering key matching categories The content contains matching key points Projected onto a two-dimensional plane , will the plane The number of unique matching keypoints is denoted as . ;
[0031] (5) Calculate key matching clusters Characteristic intensity coefficient :
[0032]
[0033] in, To match key points The corresponding matching weight.
[0034] The feature intensity coefficients of all key matching clusters are calculated, and the candidate point cloud scanning regions and their corresponding feature intensity coefficients are finally obtained.
[0035] A further step is:
[0036] In step S4, the evaluation method for the feature intensity coefficient corresponding to the candidate point cloud scanning region is that the higher the feature intensity coefficient, the stronger the feature information of the candidate point cloud scanning region, so as to select the point cloud region with higher feature intensity for collection.
[0037] A further step is:
[0038] When evaluating the planning results of a point cloud scanned area, the matching source point cloud is selected. Matching target point cloud After matching, the mean square error (RMSE) of the matching points between the two point clouds is used as the evaluation metric, and its formula is as follows:
[0039]
[0040] in To match the source point cloud The point in the middle, It is to match the target point cloud. The point in, and It is the original After performing matrix transformation, the target point cloud is matched. After mid-range transformation The closest point.
[0041] This invention can be applied to flywheel housing parts, vehicle body components, and other parts.
[0042] This invention has at least the following technical effects:
[0043] Complex parts possess intricate structures, posing a challenge to point cloud matching. The existence of complex structures implies the presence of points with significant geometric features—these points are the keypoints in the 3D point cloud. On the other hand, the large volume of point cloud data leads to high memory consumption and low computational efficiency. One solution is to downsample the point cloud, retaining only a representative subset of points. However, existing downsampling methods (such as voxel-based sampling, farthest-point sampling, and random sampling) have limitations, such as inaccurate localization and points potentially appearing in non-salient areas, resulting in non-discriminative descriptors. Therefore, detecting keypoints with significant geometric features is crucial for improving point cloud matching accuracy. A small number of keypoints are sufficient for successful point cloud alignment, and well-localized keypoints can further improve registration accuracy. By obtaining the feature information of the scanned area, this area can be prioritized for scanning in practice, as it contains more keypoints, thus enabling point cloud registration without needing to scan all zero points, thereby improving point cloud acquisition efficiency. The method of this invention can solve the problem of different feature intensities in different regions of complex parts, which leads to differences in point cloud matching accuracy. It achieves the effect of evaluating point cloud feature intensity to improve matching accuracy and improves point cloud acquisition efficiency. Attached Figure Description
[0044] Figure 1 is a flowchart of a complex part scanning region planning method based on the feature intensity of matching key points;
[0045] Figure 2 is a schematic diagram of the extracted key points for flywheel housing matching;
[0046] Figure 3 is a schematic diagram of the key points for matching the extracted vehicle body components;
[0047] Figure 4 is a schematic diagram of the hierarchical clustering algorithm (aggregation method);
[0048] Figure 5 is a schematic diagram of key cluster matching after clustering of key points for flywheel shell matching;
[0049] Figure 6 A schematic diagram showing the matching of key clusters after clustering key points for vehicle body components. Detailed Implementation
[0050] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0051] Example 1
[0052] As attached Figure 1 As shown, a method for planning the scanned region of a complex part based on matching the feature intensity of key points includes the following steps:
[0053] S1. Obtain point cloud data: Sample the CAD model of the flywheel housing part to be scanned to obtain discrete point clouds on the surface of the flywheel housing part; the sampling method of the flywheel housing CAD model is uniform sampling, that is, uniformly extracting points from the three-dimensional curved surface of the flywheel housing CAD model, with a sampling density of 5, and then downsampling is performed to control the final number of point clouds. The downsampling method uses the voxel downsampling method, with the voxel size set to 8, resulting in a final number of 42990 point clouds.
[0054] S2. Obtaining Matching Key Points: Input the discrete point cloud of the part surface into a deep learning-based point cloud matching model to obtain several matching key points of the discrete point cloud of the part surface and their corresponding matching weights; a schematic diagram of the extracted matching key points is attached. Figure 2 As shown, the point cloud matching model based on deep learning is the D3Feat point cloud matching model. The evaluation method for the matching weights of matching key points is that the higher the weight of a matching key point, the stronger its feature information.
[0055] S3. Obtain Matching Key Clusters: Select 1000 matching key points with relatively high matching weights and cluster them to obtain 5 matching key clusters, as shown in the appendix. Figure 5 As shown;
[0056] The clustering algorithm is a hierarchical clustering algorithm.
[0057] In step S3, the hierarchical clustering algorithm is a bottom-up aggregation method, where each data point is considered as an initial cluster, and the most similar clusters are gradually merged into larger clusters until the set clustering termination condition is met. (See attached...) Figure 4 As shown.
[0058] In step S3, the constraint condition of the hierarchical clustering algorithm is: when the maximum lateral area of the minimum bounding box of the matching key cluster is equal to the threshold. At that point, the cluster that matches the key category stops further clustering. This represents the maximum scan area that the scanner can capture at the optimal imaging distance.
[0059] S4. Obtain the scanning point cloud planning region: Based on the matching key clusters, the scanning region is planned for the discrete point cloud on the surface of the flywheel housing part, and 5 candidate point cloud scanning regions and their corresponding feature intensity coefficients are obtained.
[0060] The scanning area planning method is as follows:
[0061] Let the discrete point cloud on the surface of the part be denoted as The key matching cluster is ,
[0062] (1) For key matching clusters , obtain Minimum directional bounding box ;
[0063] (2) Discretize the point cloud on the surface of the part Located in the smallest direction bounding box The inner part is denoted as point cloud. ;
[0064] (3) Point cloud based on principal component analysis Projected onto a two-dimensional plane , will dot clouds In plane The area of the minimum bounding rectangle of the upper projection is denoted as ;
[0065] (4) Clustering key matching categories The content contains matching key points Projected onto a two-dimensional plane , will the plane The number of unique matching keypoints is denoted as . ;
[0066] (5) Calculate key matching clusters Characteristic intensity coefficient :
[0067]
[0068] in, To match key points The corresponding matching weight.
[0069] The feature intensity coefficients of all key matching clusters are calculated, and finally five candidate point cloud scanning regions and their corresponding feature intensity coefficients are obtained.
[0070] In step S4, the evaluation method for the feature intensity coefficient corresponding to the candidate point cloud scanning region is that the higher the feature intensity coefficient, the stronger the feature information of the candidate point cloud scanning region, so as to select the point cloud region with higher feature intensity for collection.
[0071] When evaluating the planning results of a point cloud scanned area, the matching source point cloud is selected. Matching target point cloud After matching, the mean square error (RMSE) of the matching points between the two point clouds is used as the evaluation metric, and its formula is as follows:
[0072]
[0073] in To match the source point cloud The point in the middle, It is to match the target point cloud. The point in, and It is the original After performing matrix transformation, the target point cloud is matched. After mid-range transformation The closest point.
[0074] Example 2
[0075] A method for planning the scanned region of a complex part based on the feature intensity of matching key points includes the following steps:
[0076] S1. Obtain point cloud data: Sample the CAD model of the body component parts to be scanned to obtain discrete point clouds on the surface of the body component parts; the sampling method of the body component CAD model is uniform sampling, that is, uniformly extracting points from the three-dimensional curved surface of the body component CAD model, the sampling density is set to 5, and then downsampling is performed to control the final number of point clouds. The downsampling method uses the voxel downsampling method, the voxel size is set to 8, and the final number of point clouds is 43548.
[0077] S2. Obtaining Matching Key Points: Input the discrete point cloud of the part surface into a deep learning-based point cloud matching model to obtain several matching key points of the discrete point cloud of the part surface and their corresponding matching weights; a schematic diagram of the extracted matching key points is attached. Figure 3As shown, the point cloud matching model based on deep learning is the D3Feat point cloud matching model. The evaluation method for the matching weights of matching key points is that the higher the weight of a matching key point, the stronger its feature information.
[0078] S3. Obtain Matching Key Clusters: Select 1000 matching key points with relatively high matching weights and cluster them to obtain 6 matching key clusters, as shown in the appendix. Figure 6 As shown;
[0079] The clustering algorithm is a hierarchical clustering algorithm.
[0080] In step S3, the hierarchical clustering algorithm is a bottom-up aggregation method, where each data point is considered as an initial cluster, and the most similar clusters are gradually merged into larger clusters until the set clustering termination condition is met. (See attached...) Figure 4 As shown.
[0081] In step S3, the constraint condition of the hierarchical clustering algorithm is: when the maximum lateral area of the minimum bounding box of the matching key cluster is equal to the threshold. At that point, the cluster that matches the key category stops further clustering. This represents the maximum scan area that the scanner can capture at the optimal imaging distance.
[0082] S4. Obtain the scanning point cloud planning region: Based on the matching key clusters, the scanning region is planned for the discrete point cloud on the surface of the vehicle body component parts, resulting in 6 candidate point cloud scanning regions and their corresponding feature intensity coefficients.
[0083] The scanning area planning method is as follows:
[0084] Let the discrete point cloud on the surface of the part be denoted as The key matching cluster is ,
[0085] (6) For key matching clusters , obtain Minimum directional bounding box ;
[0086] (7) Discretize the point cloud on the surface of the part Located in the smallest direction bounding box The inner part is denoted as point cloud. ;
[0087] (8) Point cloud based on principal component analysis Projected onto a two-dimensional plane , will dot clouds In plane The area of the minimum bounding rectangle of the upper projection is denoted as ;
[0088] (9) Clustering key matching categories The content contains matching key points Projected onto a two-dimensional plane , will the plane The number of unique matching keypoints is denoted as . ;
[0089] (10) Calculate key matching clusters Characteristic intensity coefficient :
[0090]
[0091] in, To match key points The corresponding matching weight.
[0092] The feature intensity coefficients of all key matching clusters are calculated, and finally 6 candidate point cloud scanning regions and their corresponding feature intensity coefficients are obtained.
[0093] In step S4, the evaluation method for the feature intensity coefficient corresponding to the candidate point cloud scanning region is that the higher the feature intensity coefficient, the stronger the feature information of the candidate point cloud scanning region, so as to select the point cloud region with higher feature intensity for collection.
[0094] When evaluating the planning results of a point cloud scanned area, the matching source point cloud is selected. Matching target point cloud After matching, the mean square error (RMSE) of the matching points between the two point clouds is used as the evaluation metric, and its formula is as follows:
[0095]
[0096] in To match the source point cloud The point in the middle, It is to match the target point cloud. The point in, and It is the original After performing matrix transformation, the target point cloud is matched. After mid-range transformation The closest point.
[0097] Although the present invention has been described herein with reference to illustrative embodiments, the above embodiments are merely preferred embodiments of the present invention, and the implementation of the present invention is not limited to the above embodiments. It should be understood that those skilled in the art can devise many other modifications and implementations, which will fall within the scope and spirit of the principles disclosed in this application.
Claims
1. A method for planning the scanning region of complex parts based on the feature intensity of matching key points, characterized in that... Includes the following steps: S1. Obtain point cloud data: Sample the CAD model of the part to be scanned to obtain discrete point cloud on the surface of the part; S2. Obtain matching key points: Input the discrete point cloud of the part surface into the point cloud matching model based on deep learning to obtain several matching key points of the discrete point cloud of the part surface and their corresponding matching weights. S3. Obtain matching key clusters: Select N matching key points with larger matching weights and cluster them to obtain M matching key clusters after clustering; S4. Obtain the scanning point cloud planning region: Based on the matching key clusters, the scanning region planning is performed on the discrete point cloud on the surface of the part to obtain M candidate point cloud scanning regions and their corresponding feature intensity coefficients; In step S4, the scanning area planning method is as follows: Let the discrete point cloud on the surface of the part be denoted as The key cluster for matching is , (1) For matching key clusters , obtain Minimum directional bounding box ; (2) Discretize the point cloud on the surface of the part Located in the smallest direction bounding box The inner part is denoted as point cloud. ; (3) Point cloud based on principal component analysis Projected onto a two-dimensional plane , will dot clouds In plane The area of the minimum bounding rectangle of the upper projection is denoted as ; (4) Matching key clusters The content contains matching key points Projected onto a two-dimensional plane , will the plane The number of unique matching keypoints is denoted as . ; (5) Calculate the matching key clusters Characteristic intensity coefficient : in, To match key points The corresponding matching weight; Calculate the feature intensity coefficients of all matching key clusters to obtain the candidate point cloud scan regions and their corresponding feature intensity coefficients.
2. The method for planning the scanning region of complex parts based on the feature intensity of matching key points according to claim 1, characterized in that: In step S1, the sampling method for the part CAD model is uniform sampling, that is, uniformly extracting points from the three-dimensional curved surface of the part CAD model and then downsampling to control the final point cloud quantity.
3. The method for planning the scanning region of complex parts based on the feature intensity of matching key points according to claim 2, characterized in that: In step S2, the point cloud matching model based on deep learning is the D3Feat point cloud matching model.
4. The method for planning the scanning region of complex parts based on the feature intensity of matching key points according to claim 2, characterized in that: In step S2, the evaluation method for the matching weight of the matching key point is that the higher the weight of the matching key point, the stronger the feature information of the matching key point.
5. The method for planning the scanning region of complex parts based on the feature intensity of matching key points according to claim 3 or 4, characterized in that: In step S3, the clustering algorithm is a hierarchical clustering algorithm.
6. The method for planning the scanning region of complex parts based on the feature intensity of matching key points according to claim 5, characterized in that: In step S3, the hierarchical clustering algorithm is a bottom-up aggregation method, which treats each data point as an initial cluster and then gradually merges the most similar clusters into larger clusters until the set clustering termination condition is met.
7. The method for planning the scanning region of complex parts based on the feature intensity of matching key points according to claim 6, characterized in that: In step S3, the constraint condition for the hierarchical clustering algorithm is: the maximum lateral area of the minimum bounding box of the matching key clusters is equal to the threshold. At that point, the cluster that matches the key category stops further clustering. This represents the maximum scan area that the scanner can capture at the optimal imaging distance.
8. The method for planning the scanning region of complex parts based on the feature intensity of matching key points according to claim 1, characterized in that: In step S4, the evaluation method for the feature intensity coefficient corresponding to the candidate point cloud scanning region is that the higher the feature intensity coefficient, the stronger the feature information of the candidate point cloud scanning region, so as to select the point cloud region with higher feature intensity for collection.
9. The method for planning the scanning region of complex parts based on the feature intensity of matching key points according to claim 8, characterized in that: When evaluating the planning results of a point cloud scanned area, the matching source point cloud is selected. Matching target point cloud After matching, the mean square error (RMSE) of the matching points between the two point clouds is used as the evaluation metric, and its formula is as follows: in To match the source point cloud The point in the middle, It is to match the target point cloud. The point in, and It is the original After performing matrix transformation, the target point cloud is matched. After mid-range transformation The closest point.