Semantic annotation result-based point cloud registration method for hull segmented complex structure

Abstract

The invention discloses a large-scale actual measurement point set generated for ship body segmented large components, which is a coarse registration method based on an RANSAC principle and a significant structure and a point cloud-model fine registration method considering various information, and provides a fast and efficient coarse registration and fine registration method. The point cloud registration based on the deep learning method can fully consider wide information such as process constraint, measurement point cloud inherent information and point cloud local semantic information, the intelligence and rationality of a registration process are greatly improved, meanwhile, the engineering guidance of a registration result is improved, rapid error evaluation of a machined part is achieved, and the precision of the machined part is improved. And a basis is provided for fitting assembly parameters.

Description

Technical Field

[0001] The invention belongs to the field of computer graphics and three-dimensional point cloud processing, and relates to a point cloud registration method based on semantic annotation results for a complex structure of a hull segment. Background Art

[0002] In the process of applying point cloud data to assembly and construction precision control, point cloud registration is particularly important. Compared with traditional point cloud registration technology, intelligent registration can fully consider a wide range of information such as process constraints, inherent information of measured point clouds, and local semantic information of point clouds, greatly improving the intelligence and rationality of the registration process, while improving the engineering guidance of the registration results.

[0003] Point cloud recognition and annotation is the core of intelligent registration algorithm and the basic technology for realizing intelligent registration detection. In addition, traditional point cloud deep recognition methods are difficult to apply to engineering measured point clouds due to various factors. Therefore, it is very important to develop efficient and feasible deep learning methods for engineering measured point clouds.

[0004] To perform complex processing on the original point cloud, recognition and semantic annotation must be completed in advance. Traditional manually designed description methods are difficult to meet the recognition requirements of large-scale point clouds, so it is inevitable to introduce deep learning methods to realize internal recognition and annotation of point clouds based on big data samples and deep neural network structures.

[0005] Based on the recognition and annotation results, the original point cloud can be pre-processed by denoising and simplifying it, providing an information-rich data set for registration and error measurement. Based on the recognition and annotation results, it is necessary to develop a matching point cloud registration algorithm to achieve the goal of assisting registration with deep learning methods, and then guide the rapid and lean assembly of large components.

[0006] For industrial precision detection problems, the traditional registration concept is to place the measured point cloud and the CAD design model in the same coordinate system in a suitable posture through spatial rigid body transformation to achieve error assessment. The most core issue at this time is the determination of the transformation parameters. The point cloud registration theory is usually divided into two parts: coarse registration and fine registration. Coarse registration refers to roughly aligning the point cloud with the CAD model by certain means to provide a relatively good initial position guess for subsequent processing; fine registration refers to using the initial posture obtained by coarse registration as the initial value of the algorithm to iteratively solve the local optimal solution. The two-step registration algorithm essentially corresponds to the optimal solution interval search and interval convex optimization in the non-convex function optimization problem. The traditional registration algorithm is limited by the high complexity of the three-dimensional space geometric topology information, and it is difficult to provide efficient and reasonable coarse registration results. For complex objects, it is often necessary to use manual auxiliary matching to quickly obtain a good point cloud posture; at the same time, the nearest point correspondence principle is used to achieve fine registration iteration, and the final result cannot truly reflect the error relationship between the point cloud and the design model. The root of the above problems is that traditional methods can only consider artificially designed information such as curvature, normal, single point coordinates, simple areas (planes, cylinders, etc.), descriptor values, etc., and the dimensions and levels of the processed data are too thin. Expanding point cloud information and introducing efficient "recognition" technology have become the direction of efforts to improve and develop traditional registration methods. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a point cloud registration method based on semantic annotation results for complex structures of hull segments in view of the deficiencies in the above-mentioned prior art, and to study the application of large-scale three-dimensional point clouds to the rapid lean assembly of large components. Point cloud registration based on deep learning methods can fully consider a wide range of information such as process constraints, inherent information of measured point clouds, and local semantic information of point clouds, greatly improving the intelligence and rationality of the registration process, while improving the engineering guidance of the registration results, realizing rapid error evaluation of processed parts, and providing a basis for the formulation of assembly parameters.

[0008] The present invention discloses a method for coarse registration and fine registration of large-scale measured point sets generated by large parts of hull segments, based on the RANSAC principle and significant structures, and a method for fine registration of point cloud and model considering multiple information, to provide a fast and efficient coarse registration and fine registration method, that is, a point cloud registration method based on semantic annotation results for complex structures of hull segments. It is mainly divided into the following two steps:

[0009] 1. Rough registration algorithm based on salient structures:

[0010] (1) Automatic coarse registration algorithm: The coarse registration method for constraining the spatial rigid body degrees of freedom is extended to include salient components in constraining the spatial rigid body degrees of freedom. At the same time, sufficient constraint combinations of all identifiable components are solved and recorded as a constraint combination table T. For the salient component group, the components are arranged according to the component category labels and the aforementioned constraint combination table T to form all feasible coarse registration constraint sets, and finally the optimal coarse registration result is calculated using the RANSAC principle.

[0011] (2) Manual coarse registration algorithm: For the segmented and annotated original point cloud, the coarse registration can be completed by manually selecting the significant components according to the constraint combination table T in the above automatic coarse registration algorithm;

[0012] 2. After selecting the automatic coarse registration algorithm or the manual coarse registration algorithm, perform the point cloud fine registration algorithm based on probability correspondence:

[0013] (1) For the simplified point cloud, the point correspondence probability function fc is used to give the local correspondence probability matrix between the point to be considered and its neighboring CAD points, lines, and surfaces based on the Gaussian function and spatial grid subdivision;

[0014] (2) Based on the CPD probability matrix and the point cloud-CAD target optimization formula, an optimization function considering the corresponding probability matrix is ​​formulated as follows:

[0015]

[0016] in:

[0017] N: the number of moving point cloud points;

[0018] M: the number of fixed point cloud points;

[0019] P nm : corresponds to the nth row and mth column element in the probability matrix;

[0020] R: the rotation matrix to be solved;

[0021] t: displacement matrix to be solved;

[0022] x n : Move the nth point in the point cloud;

[0023] y m : The mth point in the fixed point cloud

[0024] According to the quaternion derivation method of Horn et al., the optimization equation is solved; finally, the ICP algorithm is improved based on the solution of the optimization equation, and the simplified point cloud and CAD model are used as the registration objects. Each step iterates through each point in the point cloud, calculates the local probability correspondence matrix, and assembles the calculated local probability correspondence matrix into a global probability matrix, and then solves the optimization objective function, gradually approaching the local optimal solution.

[0025] For the technical solution described above, further, the fully constrained combination solution of all identifiable components is to find two structural parts of the same category in the two point clouds to be registered, use the PCA method to perform coarse registration of the two components, and evaluate the registration result through the coarse registration residual.

[0026] For the technical solution described above, further, the method for obtaining the significant components is to formulate a significant measurement function fs that comprehensively considers the surface information, component category information and process weight information. After the significant measurement function fs is calculated, the top-ranked components are automatically selected as significant components, and their collection is used as a significant component group.

[0027] For the technical solution described above, further, the step of manually selecting significant components includes selecting any group of component combinations that meet the requirements of the constraint combination table T in the point cloud display interface.

[0028] For the technical solution described above, further, the step of selecting any group of component combinations that meet the requirements of the constraint combination table T includes selecting any three significant plane combinations that are not parallel to each other.

[0029] The coarse registration algorithm in step 1 above is based on the calibrated significant structure information to obtain candidate solutions for point cloud-model coarse registration, considering the spatial displacement and rotation constraints corresponding to the point cloud coarse registration, and studying the spatial rigid body degree of freedom full constraint combination method applicable to various types of significant components of large hull parts, so as to achieve fast calculation of feasible solutions for coarse registration, and at the same time greatly reduce the number of groups of coarse registration correspondences and the scale of solution sets, and reduce the time consumption of coarse registration enumeration combinations. The automatic coarse registration algorithm and the manual coarse registration algorithm system are developed separately to meet different engineering needs.

[0030] For the technical solution described above, further, the point corresponding probability function fc in step 2 (1) takes into account its inherent three types of simple geometric information - curvature, normal, and three-dimensional coordinates, the supervoxel identification information to which the point belongs, the key part information of the point and the process constraint information of the adjacent CAD model, and the surface area information to which the point belongs.

[0031] For the technical solution described above, further, the point cloud simplification method in step 2 (1) comprises the following steps:

[0032] ① Based on the previous point cloud recognition and annotation results, the neighboring CAD model components of each supervoxel are calculated under the coarse registration pose, and the deviation value between the recognition information and the neighboring component information is calculated. If there is no similar CAD model component near the supervoxel, it is a redundant structure and is eliminated;

[0033] ② For the valid point cloud retained in step ①, the supervoxel is used as a unit, and the intersection of the regions is solved as the key parts based on the point cloud region growing algorithm, and it is marked as key points and key lines based on the principal component analysis method and edge detection method, and the key surface is selected according to the number of points and smoothness of each region;

[0034] ③Finally, each supervoxel is traversed for non-uniform simplification, and a larger sampling coefficient is set for the key part (for example, the sampling coefficient of the key part is 0.25, and the sampling coefficient of the rest is 0.1), so as to obtain a small flux point cloud for subsequent precise registration.

[0035] For the technical solution described above, further, in the point cloud simplification method ①, the method for calculating the neighboring CAD model components of each supervoxel under the coarse registration posture is to use the Hausdorff distance for calculation.

[0036] For the technical solution described above, further, the key parts of ③ in the point cloud simplification method include key points, key lines, and key surfaces.

[0037] The precise registration method based on the probability correspondence optimization function in the above step 2 is to complete the point cloud simplification and key information screening through point cloud recognition feature information, study the appropriate point cloud-model local correspondence probability evaluation method, based on the probability correspondence idea in the precise registration theory, realize the auxiliary role of point cloud multivariate information in the precise registration process, improve the point cloud-CAD model registration optimization objective function, derive the theoretical solution of the optimization function, and finally obtain more reasonable and effective point cloud registration results and error assessment information.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. Improve registration accuracy:

[0040] o Multi-information fusion: The point correspondence probability function fc of the present invention takes into account a variety of information, including inherent geometric information (curvature, normal, 3D coordinates), supervoxel recognition information, key part information, adjacent CAD model process constraint information and surface information. This multi-information fusion method significantly improves the accuracy and robustness of point cloud registration.

[0041] ο Gaussian function and spatial grid subdivision: Through Gaussian function and spatial grid subdivision, the local correspondence probability matrix between the point to be considered and its neighboring CAD points, lines, and surfaces can be effectively calculated, further improving the accuracy of registration.

[0042] οExperimental verification: Experimental data show that the root mean square error (RMSE) of the present invention in multiple test segments is significantly lower than that of other methods. For example, in Block I, RMSE1 is 119.78 mm and RMSE2 is 4.53 mm, which are much lower than other methods.

[0043] 2. Improve registration efficiency:

[0044] o Fast coarse registration: The automatic coarse registration algorithm of the present invention can quickly achieve preliminary alignment of point clouds through salient components and RANSAC principles, significantly reducing the time consumption of coarse registration.

[0045] ο Efficient and precise registration: The ICP optimization function of the point cloud-CAD model based on probability correspondence iteratively calculates the local probability correspondence matrix and gradually approaches the local optimal solution, thereby improving the efficiency of precise registration.

[0046] ο Experimental verification: Experimental data show that the running time of the present invention in multiple test segments is significantly lower than that of other methods. For example, in Block I, the running time is 31 seconds, which is much lower than methods such as PointNet and PointNet++.

[0047] 3. Improve robustness and stability:

[0048] οMultimodal data processing: The present invention can process hull segment structures of different complexity, and improves the robustness and stability of the registration through multi-information fusion and probabilistic correspondence methods.

[0049] ο Experimental verification: It can be seen from the data of multiple test segments that the performance of the present invention in different scenarios is very stable, and both the time consumption and the registration accuracy show good robustness and consistency.

[0050] 4. Reduce manpower and time costs:

[0051] o High degree of automation: The automatic coarse registration algorithm and the fine registration algorithm based on probability correspondence of the present invention can achieve a high degree of automation, reduce the need for manual intervention, and reduce labor costs.

[0052] οGood real-time performance: The present invention has high real-time performance in practical applications, can complete point cloud registration in a short time, and improves work efficiency.

[0053] ο Experimental verification: Experimental data show that the running time of the present invention in multiple test segments is relatively short, and it can achieve rapid alignment in actual engineering.

[0054] 5. Improve engineering guidance:

[0055] ο Rapid error assessment: The present invention can realize rapid error assessment of processed parts, provide a basis for the formulation of assembly parameters, and improve the guidance and accuracy of engineering projects.

[0056] ο Practical application cases: The present invention has been applied in many practical projects. For example, in the assembly process of a large ship hull segment, the assembly accuracy and efficiency were significantly improved through the registration method of the present invention.

[0057] 6. Promote technological innovation and industry development:

[0058] οTechnological innovation: This invention combines deep learning and probabilistic correspondence methods to realize the intelligence and automation of point cloud registration, and promotes technological innovation in the fields of computer graphics and 3D point cloud processing.

[0059] οIndustry application: The present invention has broad application prospects in multiple industries such as shipbuilding, aerospace, and automobile manufacturing, and can significantly improve the production efficiency and product quality of these industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a flow chart of the rough registration algorithm based on salient structure extraction;

[0061] Figure 2 This is the flow chart of the point cloud precise registration algorithm based on probability correspondence;

[0062] Figure 3 This is an example of an automatic coarse registration algorithm based on salient structures;

[0063] Figure 4 This is an example of a manual coarse registration algorithm based on salient structures. DETAILED DESCRIPTION

[0064] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0065] Example 1

[0066] The invention is a point cloud registration method based on semantic annotation results for complex structure of hull segments, and its key technology lies in a coarse registration method based on RANSAC principle and significant structure, and a point cloud-model fine registration method considering multiple information.

[0067] 1. Rough registration algorithm based on salient structures:

[0068] like Figure 1As shown in the figure, based on the supervoxel recognition and annotation results, the automatic coarse registration algorithm and the manual coarse registration algorithm system are developed respectively:

[0069] 1. For the automatic coarse registration algorithm:

[0070] A significant measurement function fs is formulated that comprehensively considers the surface information, component category information and process weight information. After calculating the significant measurement function fs, the components ranked at the top are automatically selected as the significant component group; the popular rough registration method of three points, three surfaces, three spheres, cylinders and cones to constrain the spatial rigid body degrees of freedom is expanded - the significant components are also used to constrain the spatial rigid body degrees of freedom, so as to achieve effective constraints on the spatial displacement of the identifiable components, and at the same time solve (find two structural parts of the same category in the two point clouds to be registered, use the PCA method to perform rough registration on the two components, and evaluate the registration results through the rough registration residual) to obtain the sufficient constraint combination of all identifiable components and record them as the constraint combination table T (including the constraint combination of the constrained spatial rigid body degrees of freedom, such as a significant component and a point, surface, or sphere to form a constraint combination); for the significant component group, according to the component category label and the aforementioned constraint combination table T, the components are arranged to form all feasible rough registration constraint sets, and finally the optimal rough registration result is calculated using the RANSAC principle.

[0071] Figure 3 The registration process of a segment and its design model using this method is shown. This process uses plane information as a significant feature, and the significant measurement function fs of all planes is equal. The constraint combination table T records the pairing relationship between the segment point cloud and all possible three-plane groups in the model. RANSAC randomly selects T table elements to match the segment point cloud and the model, and obtains the following Figure 3 The rough registration result shown on the far right shows the relative position of the segmented point cloud and the design model in three-dimensional space, and they are roughly aligned. This process not only improves the accuracy and efficiency of the registration, but also provides a basis for subsequent more detailed analysis and processing.

[0072] Construction details of constraint combination table T: automatically select the top-ranked component groups by saliency calculation; expand popular rough registration methods such as three points, three surfaces, three spheres, cylinders, and cones, and attach them to all identifiable components to achieve effective constraints on spatial displacement of identifiable components. At the same time, solve the sufficient constraint combinations of all identifiable components and record them in the list.

[0073] RANSAC parameter setting and optimization process: Use the point cloud recognition and annotation results to filter the point cloud to obtain a new point set that only retains the plane points. On this basis, the plane point set is divided into multiple point clusters using the angle between the normal vectors of the points as the distance standard, and each point in each cluster belongs to mutually parallel planes. Furthermore, each point cluster is divided into sub-clusters based on the point offset as the classification basis to obtain multiple self-clusters, and the points in each sub-cluster belong to the same plane. The above clusters are filtered by the point number threshold to obtain the significant clusters and complete the significant plane extraction. The number threshold is set to 50,000.

[0074] The data set used in the present invention is derived from the point cloud data of the actual hull segment structure, which is collected by a laser scanner. The data preprocessing steps include point cloud cleaning, denoising and normalization to ensure the quality and consistency of the data. The results are shown in Table 1:

[0075] (1) Experimental setup

[0076] Hardware configuration:

[0077] Processor: Intel Core i7-9700K

[0078] Memory: 32GB DDR4

[0079] Graphics card: NVIDIA RTX 2080Ti

[0080] Software environment:

[0081] Operating system: Ubuntu 20.04

[0082] Programming language: Python 3.7

[0083] Deep learning framework: PyTorch 1.7.1

[0084] (2) Model training

[0085] Training parameters:

[0086] Learning rate: 0.001

[0087] Batch size: 32

[0088] Iterations: 100 epochs

[0089] Training process:

[0090] Data loading: Use PyTorch's DataLoader class to load the training dataset.

[0091] Model initialization: Use the pre-trained ResNet as the base model and add a custom point cloud processing module.

[0092] Loss function: The mean squared error (MSE) loss function is used.

[0093] Optimizer: Use Adam optimizer.

[0094] (3) Testing process

[0095] Test Dataset:

[0096] The test dataset contains 4 different hull segment structures, each of which contains multiple point cloud data blocks, ensuring the diversity and representativeness of the test dataset.

[0097] Evaluation Metrics:

[0098] Loss function: Calculates the average loss of the model on the test set.

[0099] Accuracy: Calculate the classification accuracy of the model on the test set.

[0100] Running time: records the average running time of the model on the test set.

[0101] Root mean square error (RMSE): RMSE1 and RMSE2 of the model on the test set are calculated, which represent the overall error and local error of point cloud registration respectively.

[0102] Table 1

[0103]

[0104] From the analysis of the data in the above table, we can see that:

[0105] Low training and testing loss: Experimental data show that our network exhibits low loss values ​​during training and testing. Especially in the testing phase, the loss value is only 0.65, which is much lower than other methods, indicating that the model has good generalization ability.

[0106] High training and testing accuracy: In terms of training and testing accuracy, the network of the present invention performs well, with a test accuracy of 93.35%, which is comparable to or even slightly better than existing advanced methods (such as PointNet++). This shows that the model is very accurate in identifying and annotating point clouds.

[0107] Short running time: The running time of the present invention in multiple test segments is significantly lower than that of most comparison methods. For example, in Block I and Block IV, the running time is 31 seconds and 25 seconds respectively, which is much lower than methods such as PointNet and PointNet++. This shows that the present invention has high efficiency in practical applications.

[0108] Low root mean square error (RMSE): The RMSE1 and RMSE2 of the present invention in multiple test segments are excellent. In particular, in Block I and Block II, the RMSE1 is 119.78mm and 44.42mm, respectively, which is much lower than other methods. RMSE2 remains at a low level in all test segments, especially in Block I, where the RMSE2 is only 4.53, showing extremely high registration accuracy.

[0109] Robustness and stability: It can be seen from the data of multiple test segments that the performance of the present invention in different scenarios is very stable, and both time consumption and registration accuracy show good robustness and consistency. This shows that the present invention has wide applicability and reliability in practical applications.

[0110] In summary, the present invention significantly improves the accuracy and efficiency of point cloud registration through deep learning and probabilistic correspondence methods, and is particularly suitable for the rapid lean assembly of complex ship segment structures. The experimental data and analysis results fully demonstrate the superior performance and broad prospects of the present invention in practical applications.

[0111] 2. For the manual coarse registration algorithm:

[0112] A simple and direct human-computer interaction interface is developed to display the segmented and labeled original point cloud in different color levels. The operator only needs to complete the manual selection of significant components according to the constraint combination table T in the above-mentioned automatic coarse registration algorithm, and select any group of component combinations that meet the requirements of the constraint combination table T (for example: any combination of three non-parallel significant planes) in the point cloud display interface to complete the coarse registration.

[0113] Figure 4 The human-computer interaction registration process of the segmented point cloud and the design model is shown. The plane information is still used as a significant feature. The constraint combination table T records the pairing relationship between the segmented point cloud and all possible three-plane groups in the model. A set of feature correspondences in T is manually selected through human-computer interaction to obtain Figure 4 The rough registration result shown on the far right shows that after manual rough registration, the relative positions of the segmented point cloud and the data model in the three-dimensional space have been roughly aligned, proving that the rough registration can be completed by selecting any set of component combinations that meet the requirements of the constraint combination table T, with good results and simple and convenient operation.

[0114] 2. Point cloud precise registration algorithm based on probability correspondence:

[0115] like Figure 2 As shown in the figure, the flow chart of the point cloud precise registration algorithm based on probability correspondence.

[0116] 1. Point cloud simplification:

[0117] (1) Based on the previous point cloud recognition and annotation results, calculate (using Hausdorff distance) the neighboring CAD model components of each supervoxel under the coarse registration pose, and calculate the deviation value (category deviation, for example, the same category deviation value is 0, and the different category deviation value is +∞) between the recognition information and the neighboring component information. If there is no similar CAD model component near the supervoxel, it is a redundant structure and is eliminated;

[0118] (2) For the valid point cloud retained in step (1), the supervoxel volume is used as a unit, and the region intersection part is solved as the key part based on the point cloud region growing algorithm, and it is marked as key points and key lines according to the principal component analysis method and edge detection method, and the key surface is selected according to the number of points and smoothness of each region;

[0119] (3) Finally, each supervoxel is traversed for non-uniform simplification, and a larger sampling coefficient is set for the key parts (key points, key lines, and key surfaces) (for example, the sampling coefficient of the key parts is 0.25, and the sampling coefficient of the rest is 0.1), so as to obtain a small-throughput point cloud for subsequent precise alignment.

[0120] Supervoxel partitioning parameters: First, the original point cloud is voxelized. Voxels are small squares with several points in them. The original point cloud is divided into voxels. Then, based on the current voxel growth theory, the regional growth is completed by predefining the characteristic parameters of each voxel. In order to comprehensively consider the distribution state of the point cloud in the voxel, the characteristic parameters are selected, the growth standard and growth threshold are formulated, and each growth area is grown. At the same time, considering that the face plate and web of T-profiles and angle steels are at a 90-degree angle, the two cannot be grown together during regional growth, so secondary regional growth is required. Since the size of the web and face plate of the profile is small, the voxels in the area they constitute are also small, while the size of the plate is large, the voxels in the area it constitutes are also large, so the area after the first growth is divided, and the area with fewer voxels in the area is separated from the area with more voxels. The area with fewer voxels is grown twice, and the web and face plate of the profile are merged into one area.

[0121] Region growing algorithm parameters: To maximize the segmentation of the plane, the curvature of each point needs to be calculated. First, the points along the sharp edge are removed by the curvature threshold, and then the points with the smallest curvature are selected as "seed points" by descending curvature. This "seed point" selection method ensures that the algorithm grows from the flattest point, minimizes the number of segments of the segmented plane, and obtains a complete plane. The normal angle is selected as the classification condition. When the normal angle exceeds the threshold, the growth stops.

[0122] Edge detection parameters: Region growing is an iterative process, each seed point is iteratively grown until all the data in the point cloud has been processed, thus forming different regions whose boundaries are defined by the growth stop set.

[0123] Non-uniform simplification parameters, sampling coefficients for key parts, sampling coefficients for non-key parts. Non-uniform sampling technology is used for each area composed of plates and profiles, and the area where the plates are located is downsampled on a larger scale, and the area where the profiles are located is downsampled on a smaller scale, so that the original point cloud is highly simplified.

[0124] 2. Point corresponding probability function fc:

[0125] For the simplified point cloud, the point correspondence probability function fc considers its inherent three types of simple geometric information - curvature, normal, and three-dimensional coordinates, the supervoxel identification information to which the point belongs, the key part information of the point and the process constraint information of the adjacent CAD model, and the surface domain information to which the point belongs. Based on the Gaussian function and spatial grid decomposition, the local correspondence probability matrix (calculated by the point correspondence probability function) between the point to be considered and its adjacent CAD points, lines, and surfaces is given.

[0126] Gaussian kernel parameter (σ): Gaussian kernel function can map input data into high-dimensional space, while other kernel functions can only map into finite-dimensional space, which is defined as:

[0127]

[0128] Where ρ(e) is the cost function induced by maximizing the relevant entropy, satisfying:

[0129] (1)ρ(e)≥0;

[0130] (2)ρ(0)=0;

[0131] (3)ρ(e)=ρ(-e);

[0132] (4) If |e_i|>|e_j|, then ρ(e_i)≥ρ(e_j).

[0133] Multi-information fusion: The point correspondence probability function fc takes into account a variety of information, including inherent geometric information (curvature, normal, 3D coordinates), supervoxel recognition information, key part information, adjacent CAD model process constraint information and surface information. This multi-information fusion method significantly improves the accuracy and robustness of point cloud registration.

[0134] Gaussian function and spatial grid subdivision: Through Gaussian function and spatial grid subdivision, the local correspondence probability matrix between the point to be considered and its neighboring CAD points, lines, and surfaces can be effectively calculated, thereby improving the accuracy of registration.

[0135] Gaussian kernel parameter (σ): The setting of Gaussian kernel parameter (σ) enables the input data to be mapped into a high-dimensional space, thereby better capturing the complex structure and features of point cloud data.

[0136] 3. Point cloud-CAD model ICP optimization function based on probability correspondence:

[0137] Based on the CPD probability matrix and point cloud-CAD target optimization formula, an optimization function considering the corresponding probability matrix is ​​formulated as follows: (The specific meanings of the variables involved in the function are as follows) According to the quaternion derivation method of Horn et al., the optimization equation is solved. Finally, the ICP algorithm is improved based on the solution of the optimization equation. The simplified point cloud and CAD model (STL triangle patch set) are used as the registration objects. Each step iterates through each point in the point cloud, calculates the local probability correspondence matrix, and assembles the calculated local probability correspondence matrix into a global probability matrix, and then solves the optimization objective function, gradually approaching the local optimal solution.

[0138] N: The number of moving point cloud points

[0139] M: The number of fixed point cloud points

[0140] P nm : corresponds to the nth row and mth column element in the probability matrix

[0141] R: The rotation matrix to be solved

[0142] t: displacement matrix to be solved

[0143] x n : Move the nth point in the point cloud

[0144] y m : fix the mth point in the point cloud.

[0145] Optimization function: Based on the CPD probability matrix and the point cloud-CAD target optimization formula, the formulated optimization function can effectively consider the correspondence between the point cloud and the CAD model, thereby improving the accuracy and robustness of the registration.

[0146] Quaternion derivation method: The quaternion derivation method of Horn et al. can efficiently solve the optimization equation and improve the computational efficiency.

[0147] Iterative process: Each step iterates through each point in the point cloud, calculates the local probability correspondence matrix, and assembles it into a global probability matrix, gradually approaching the local optimal solution. This iterative process ensures high accuracy and stability of the registration results.

[0148] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A point cloud registration method based on semantic annotation results for complex structure of hull segments, characterized in that: The following steps are involved:

1. Rough registration algorithm based on salient structures: (1) Automatic coarse registration algorithm: The coarse registration method for constraining the spatial rigid body degrees of freedom is extended to include significant components in constraining the spatial rigid body degrees of freedom. At the same time, sufficient constraint combinations of all identifiable components are solved and recorded as constraint combination table T. For the significant component group, according to the component category label and the aforementioned constraint combination table T, the components are arranged to form all feasible rough registration constraint sets, and finally the optimal rough registration result is calculated using the RANSAC principle; (2) Manual coarse registration algorithm: For the segmented and annotated original point cloud, the coarse registration can be completed by manually selecting the significant components according to the constraint combination table T in the above automatic coarse registration algorithm; 2. After selecting the automatic coarse registration algorithm or the manual coarse registration algorithm, perform the point cloud fine registration algorithm based on probability correspondence: (1) For the simplified point cloud, the point correspondence probability function fc is used to give the local correspondence probability matrix between the point to be considered and its neighboring CAD points, lines, and surfaces based on the Gaussian function and spatial grid subdivision; (2) Based on the CPD probability matrix and the point cloud-CAD target optimization formula, an optimization function considering the corresponding probability matrix is ​​formulated as shown in Formula I. According to the quaternion derivation method of Horn et al., the optimization equation is solved. Finally, the ICP algorithm is improved based on the solution of the optimization equation. The simplified point cloud and CAD model are used as the registration objects. Each step iterates through each point in the point cloud, calculates the local probability correspondence matrix, and assembles the calculated local probability correspondence matrix into a global probability matrix. Then, the optimization objective function is solved and the local optimal solution is gradually approached. Here: The formula I is: N: the number of moving point cloud points; M: the number of fixed point cloud points; P nm : corresponds to the nth row and mth column element in the probability matrix; R: the rotation matrix to be solved; t: the displacement matrix to be solved; x n : Move the nth point in the point cloud; y m : fix the mth point in the point cloud.

2. The method according to claim 1, characterized in that: The fully constrained combination solution of all identifiable components is to find two structural parts with the same category in the two point clouds to be registered, use the PCA method to perform coarse registration on the two components, and evaluate the registration result through the coarse registration residual.

3. The method according to claim 1, characterized in that The method for obtaining the significant components is to formulate a significant measurement function fs that comprehensively considers the surface area information, component category information and process weight information. After calculating the significant measurement function fs, the components ranked at the top are automatically selected as significant components, and their collection is used as a significant component group.

4. The method according to claim 1, characterized in that: The step of manually selecting significant components includes selecting any group of component combinations that meet the requirements of the constraint combination table T in the point cloud display interface.

5. The method according to claim 1, characterized in that The step of selecting any group of component combinations that meet the requirements of the constraint combination table T includes selecting any three significant plane combinations that are not parallel to each other.

6. The method according to claim 1, characterized in that The point corresponding probability function fc in step 2 (1) takes into account its inherent three types of simple geometric information - curvature, normal, and three-dimensional coordinates, the supervoxel identification information to which the point belongs, the key part information of the point and the process constraint information of the adjacent CAD model, and the surface area information to which the point belongs.

7. The method according to claim 1, characterized in that The point cloud simplification method in step (1) of the second step comprises the following steps: ① Based on the previous point cloud recognition and annotation results, the neighboring CAD model components of each supervoxel are calculated under the coarse registration pose, and the deviation value between the recognition information and the neighboring component information is calculated. If there is no similar CAD model component near the supervoxel, it is a redundant structure and is eliminated; ② For the valid point cloud retained in step ①, the supervoxel is used as a unit, and the intersection of the regions is solved as the key parts based on the point cloud region growing algorithm, and it is marked as key points and key lines based on the principal component analysis method and edge detection method, and the key surface is selected according to the number of points and smoothness of each region; ③Finally, each supervoxel is traversed for non-uniform simplification, and a larger sampling coefficient is set for the key part to obtain a small flux point cloud for subsequent precise registration.

8. The method according to claim 7, characterized in that In the point cloud simplification method, the method of calculating each supervoxel neighbor CAD model component under the coarse registration posture is to use the Hausdorff distance for calculation.

9. The method according to claim 7, characterized in that: The key parts of ③ in the point cloud simplification method include key points, key lines, and key surfaces.

Images