A point cloud registration method based on semantic annotation results for complex ship segment structures

Abstract

The present invention discloses a large-scale measured point set generated for large hull segment components. It provides fast and efficient coarse and fine registration methods based on the RANSAC principle and a coarse registration method of salient structures and a point cloud-model fine registration method that considers multiple information. The point cloud registration based on deep learning methods can fully consider a wide range of information such as process constraints, inherent information of the measured point cloud, and local semantic information of the point cloud, greatly improving the intelligence and rationality of the registration process, while enhancing the engineering guidance of the registration results, realizing rapid error evaluation of processed components, and providing a basis for the formulation of assembly parameters.

Description

Technical Field

[0001] The invention belongs to the fields of computer graphics and three-dimensional point cloud processing, and relates to a point cloud registration method based on semantic annotation results for complex segmented hull structures. Background Art

[0002] When applying point cloud data to assembly and construction precision control, point cloud registration is particularly important. Compared to traditional point cloud registration technologies, intelligent registration fully considers a wide range of information, including process constraints, inherent information in the measured point cloud, and local semantic information in the point cloud. This significantly improves the intelligence and rationality of the registration process, while also enhancing the engineering guidance of the registration results.

[0003] Point cloud recognition and annotation are the core of intelligent registration algorithms and the foundational technology for intelligent registration detection. Furthermore, traditional point cloud deep recognition methods are difficult to apply to actual engineering point clouds due to various factors. Therefore, developing efficient and feasible deep learning methods for actual engineering point clouds is crucial.

[0004] To perform complex processing on raw point clouds, recognition and semantic annotation must be completed in advance. Traditional manually designed description methods are difficult to meet the recognition requirements of large-scale point clouds. Therefore, it is necessary to introduce deep learning methods to achieve internal recognition and annotation of point clouds based on large data samples and deep neural network structures.

[0005] Based on the recognition and annotation results, preprocessing tasks such as denoising and simplification of the original point cloud can be performed, providing an information-rich dataset for registration and error measurement. Based on the recognition and annotation results, supporting point cloud registration algorithms need to be developed to achieve the goal of deep learning-assisted registration, thereby guiding the rapid and lean assembly of large components.

[0006] For industrial precision inspection, the traditional registration concept involves juxtaposing the measured point cloud and the CAD design model in the same coordinate system at an appropriate pose using a spatial rigid-body transformation to assess errors. The key issue here is determining the transformation parameters. Point cloud registration theory is generally divided into coarse and fine registration. Coarse registration involves roughly aligning the point cloud with the CAD model through certain means, providing a good initial position guess for subsequent processing. Fine registration involves iteratively finding a local optimal solution using the initial pose obtained from coarse registration as the algorithm's starting point. This two-step registration algorithm essentially corresponds to the optimal solution interval search and convex optimization within the interval in non-convex function optimization problems. Traditional registration algorithms, limited by the high complexity of 3D geometric topology information, struggle to provide efficient and reasonable coarse registration results. For complex objects, manual matching is often required to quickly obtain a good point cloud pose. Fine registration iterations, meanwhile, rely on the closest point correspondence principle, resulting in results that fail to accurately reflect the error relationship between the point cloud and the design model. The root cause of these problems is that traditional methods only consider artificially designed information such as curvature, normals, single-point coordinates, simple regions (planes, cylinders, etc.), and descriptor values, resulting in a limited number of dimensions and layers of data. Efforts to improve and develop traditional registration methods are focused on expanding point cloud information and introducing efficient "recognition" technologies. Summary of the Invention

[0007] The technical problem to be solved by this invention is to address the shortcomings of the above-mentioned existing technologies by providing a semantically annotated point cloud registration method for complex ship segment structures, and to explore the application of large-scale 3D point clouds to the rapid and lean assembly of large components. Point cloud registration based on deep learning methods can fully consider a wide range of information, including process constraints, inherent information of the measured point cloud, and local semantic information of the point cloud. This significantly improves the intelligence and rationality of the registration process, while also enhancing the engineering guidance of the registration results, enabling rapid error assessment of processed components and providing a basis for the formulation of assembly parameters.

[0008] This paper discloses a method for quickly and efficiently performing coarse and fine registration of large-scale measured point sets generated for large ship hull segments. This method uses the RANSAC principle and a coarse registration method based on salient structures, and a point cloud-model fine registration method that considers multiple information points. This method provides a point cloud registration method for complex ship hull segment structures based on semantic annotation results. The method mainly involves the following two steps:

[0009] 1. Rough registration algorithm based on salient structure:

[0010] (1) Automatic coarse registration algorithm: The coarse registration method for constraining spatial rigid body degrees of freedom is extended to include salient components in constraining spatial rigid body degrees of freedom. At the same time, sufficient constraint combinations for all identifiable components are solved and recorded as a constraint combination table T. For a group of salient components, the components are arranged according to the component category labels and the aforementioned constraint combination table T to form all feasible coarse registration constraint sets. Finally, the optimal coarse registration result is calculated using the RANSAC principle.

[0011] (2) Manual coarse registration algorithm: For the segmented and annotated original point cloud, the coarse registration can be completed by manually selecting the significant components according to the constraint combination table T in the above-mentioned automatic coarse registration algorithm;

[0012] 2. After selecting either the automatic coarse registration algorithm or the manual coarse registration algorithm, perform the probability-based point cloud fine registration algorithm:

[0013] (1) For the simplified point cloud, the point correspondence probability function fc is used to give the local correspondence probability matrix between the point to be considered and its neighboring CAD points, lines, and surfaces based on the Gaussian function and spatial grid subdivision;

[0014] (2) Based on the CPD probability matrix and the point cloud-CAD target optimization formula, an optimization function considering the corresponding probability matrix is ​​formulated as follows:

[0015]

[0016] in:

[0017] N: the number of moving point cloud points;

[0018] M: the number of fixed point cloud points;

[0019] P nm : corresponds to the nth row and mth column element in the probability matrix;

[0020] R: the rotation matrix to be solved;

[0021] t: displacement matrix to be solved;

[0022] x n : Move the nth point in the point cloud;

[0023] y m : fix the mth point in the point cloud

[0024] Based on the quaternion derivation method of Horn et al., the optimization equation is solved. Finally, the ICP algorithm is improved based on the solution of the optimization equation. The simplified point cloud and CAD model are used as the registration objects. Each step iterates through each point in the point cloud, calculates the local probability correspondence matrix, and assembles the calculated local probability correspondence matrix into a global probability matrix, and then solves the optimization objective function, gradually approaching the local optimal solution.

[0025] For the technical solution described above, further, the fully constrained combination solution of all identifiable components is to find two structural parts with the same category in the two point clouds to be aligned, use the PCA method to perform coarse alignment on the two components, and evaluate the alignment results through the coarse alignment residual.

[0026] For the technical solution described above, further, the method for obtaining significant components is to formulate a significant measurement function fs that comprehensively considers the surface area information, component category information and process weight information. After the significant measurement function fs is calculated, the components ranked at the top are automatically selected as significant components, and their collection is used as a significant component group.

[0027] For the technical solution described above, further, the step of manually selecting significant components includes selecting any group of component combinations that meet the requirements of the constraint combination table T in the point cloud display interface.

[0028] For the technical solution described above, further, the step of selecting any group of component combinations that meet the requirements of the constraint combination table T includes selecting any three significant plane combinations that are not parallel to each other.

[0029] The coarse registration algorithm in step 1 above uses calibrated salient structural information to determine candidate solutions for point cloud-model coarse registration. Considering the spatial displacement and rotation constraints associated with point cloud coarse registration, a method for combining spatial rigid-body degrees of freedom (DOFs) with sufficient constraints applicable to various salient components of large hull components is developed. This method enables rapid computation of feasible solutions for coarse registration, significantly reduces the number of coarse registration correspondences and the size of the solution set, and reduces the time required for coarse registration enumeration. Separate automated and manual coarse registration algorithms are being developed to meet diverse engineering needs.

[0030] For the technical solution described above, further, the point corresponding probability function fc in step 2 (1) takes into account its inherent three types of simple geometric information - curvature, normal, and three-dimensional coordinates, the supervoxel identification information to which the point belongs, the key part information of the point and the process constraint information of the adjacent CAD model, and the surface area information to which the point belongs.

[0031] Regarding the technical solution described above, further, the point cloud simplification method in step 2 (1) includes the following steps:

[0032] ① Based on the previous point cloud recognition and annotation results, the neighboring CAD model components of each supervoxel are calculated in the coarse registration pose, and the deviation between the recognition information and the neighboring component information is calculated. If there is no similar CAD model component near the supervoxel, it is considered a redundant structure and is eliminated;

[0033] ② For the valid point cloud retained in step ①, use the supervoxel as the unit, and use the point cloud region growing algorithm to solve the regional intersection as the key part. Then use the principal component analysis method and edge detection method to mark it as key points and key lines, and select the key surface according to the number of points and flatness of each region;

[0034] ③ Finally, each supervoxel is traversed for non-uniform simplification, and a larger sampling coefficient is set for the key part (for example, the sampling coefficient of the key part is 0.25, and the sampling coefficient of the rest is 0.1), so as to obtain a low-flux point cloud for subsequent fine alignment.

[0035] For the technical solution described above, further, in the point cloud simplification method, the method of calculating the neighboring CAD model components of each super voxel under the coarse registration posture in ① is to use Hausdorff distance for calculation.

[0036] Regarding the technical solution described above, further, the key parts of ③ in the point cloud simplification method include key points, key lines, and key surfaces.

[0037] The precision registration method based on the probability correspondence optimization function in step 2 above is to simplify the point cloud and screen key information through point cloud recognition feature information, study appropriate point cloud-model local correspondence probability evaluation methods, and based on the probabilistic correspondence concept in precision registration theory, realize the auxiliary role of multivariate point cloud information in the precision registration process, improve the point cloud-CAD model registration optimization objective function, derive the theoretical solution of this optimization function, and ultimately obtain more reasonable and effective point cloud registration results and error assessment information.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. Improve registration accuracy:

[0040] Multi-information fusion: The point correspondence probability function fc of this invention considers a variety of information, including inherent geometric information (curvature, normal, 3D coordinates), supervoxel identification information, key component information, adjacent CAD model process constraints, and surface area information. This multi-information fusion approach significantly improves the accuracy and robustness of point cloud registration.

[0041] ο Gaussian function and spatial grid subdivision: Through Gaussian function and spatial grid subdivision, the local correspondence probability matrix between the point to be considered and its neighboring CAD points, lines, and surfaces can be effectively calculated, further improving the accuracy of registration.

[0042] οExperimental verification: Experimental data show that the root mean square error (RMSE) of the present invention in multiple test segments is significantly lower than that of other methods. For example, in Block 1, RMSE1 is 119.78 mm and RMSE2 is 4.53 mm, which are much lower than those of other methods.

[0043] 2. Improve registration efficiency:

[0044] o Fast coarse registration: The automatic coarse registration algorithm of the present invention can quickly achieve preliminary alignment of point clouds through salient components and RANSAC principles, significantly reducing the time consumption of coarse registration.

[0045] ο Efficient and precise registration: The ICP optimization function of the point cloud-CAD model based on probability correspondence iteratively calculates the local probability correspondence matrix, gradually approaches the local optimal solution, and improves the efficiency of precise registration.

[0046] Experimental verification: Experimental data show that the running time of the present invention in multiple test segments is significantly lower than that of other methods. For example, in Block 1, the running time is 31 seconds, which is much lower than methods such as PointNet and PointNet++.

[0047] 3. Improve robustness and stability:

[0048] o Multimodal data processing: The present invention can process hull segment structures of different complexities, and improves the robustness and stability of the registration through multi-information fusion and probabilistic correspondence methods.

[0049] ο Experimental verification: It can be seen from the data of multiple test segments that the performance of the present invention in different scenarios is very stable, and both time consumption and registration accuracy show good robustness and consistency.

[0050] 4. Reduce manpower and time costs:

[0051] o High degree of automation: The automatic coarse registration algorithm and the fine registration algorithm based on probability correspondence of the present invention can achieve a high degree of automation, reducing the need for manual intervention and lowering labor costs.

[0052] οGood real-time performance: The present invention has high real-time performance in practical applications, can complete point cloud registration in a short time, and improves work efficiency.

[0053] ο Experimental verification: Experimental data show that the running time of the present invention in multiple test segments is relatively short, and it can achieve rapid alignment in actual engineering.

[0054] 5. Improve engineering guidance:

[0055] ο Rapid error assessment: The present invention can achieve rapid error assessment of processed parts, provide a basis for the formulation of assembly parameters, and improve the guidance and accuracy of engineering projects.

[0056] ο Practical application cases: The present invention has been applied in many practical projects. For example, in the assembly process of a large ship hull segment, the registration method of the present invention significantly improved the assembly accuracy and efficiency.

[0057] 6. Promote technological innovation and industry development:

[0058] οTechnological innovation: This invention combines deep learning and probabilistic correspondence methods to achieve intelligent and automated point cloud registration, promoting technological innovation in the fields of computer graphics and 3D point cloud processing.

[0059] οIndustry application: The present invention has broad application prospects in multiple industries such as shipbuilding, aerospace, and automobile manufacturing, and can significantly improve the production efficiency and product quality of these industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is the flow chart of the coarse registration algorithm based on salient structure extraction;

[0061] Figure 2 This is the flow chart of the point cloud precise registration algorithm based on probability correspondence;

[0062] Figure 3 This is an example of an automatic coarse registration algorithm based on salient structures;

[0063] Figure 4 This is an example of a manual coarse registration algorithm based on salient structures. DETAILED DESCRIPTION

[0064] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0065] Example 1

[0066] The present invention is a point cloud registration method based on semantic annotation results for complex structure of hull segments. The key technology lies in a coarse registration method based on RANSAC principle and significant structure, and a point cloud-model fine registration method considering multiple information.

[0067] 1. Rough registration algorithm based on salient structure:

[0068] like Figure 1As shown in the figure, based on the supervoxel recognition and annotation results, the automatic coarse registration algorithm and the manual coarse registration algorithm system are developed respectively:

[0069] 1. For the automatic coarse registration algorithm:

[0070] A saliency metric function fs is developed that comprehensively considers surface area information, component category information, and process weight information. After calculating the saliency metric function fs, the top-ranked components are automatically selected as the salient component group. The popular coarse registration method for constraining spatial rigid body degrees of freedom using three points, three surfaces, three spheres, cylinders, and cones is expanded—salient components are also used to constrain spatial rigid body degrees of freedom, achieving effective constraints on spatial displacements of identifiable components. At the same time, sufficient constraint combinations for all identifiable components are solved (finding two structural parts of the same category in the two point clouds to be registered, using the PCA method to perform coarse registration on the two components, and evaluating the registration results through the coarse registration residuals) and recorded as a constraint combination table T (containing constraint combinations that can constrain spatial rigid body degrees of freedom, such as a significant component forming a constraint combination with a point, surface, or sphere). For the salient component group, the components are arranged according to the component category label and the aforementioned constraint combination table T to form all feasible coarse registration constraint sets, and the optimal coarse registration result is finally calculated using the RANSAC principle.

[0071] Figure 3 The registration process of a segment and its design model using this method is shown. This process uses plane information as a significant feature, and the significant measurement function fs of all planes is equal. The constraint combination table T records all possible pairing relationships between the segment point cloud and the model. RANSAC randomly selects T table elements to match the segment point cloud and the model, and obtains the following Figure 3 The coarse registration result shown on the far right shows the relative positions of the segmented point cloud and the design model in three-dimensional space, and they are roughly aligned. This process not only improves the accuracy and efficiency of the registration, but also provides a basis for subsequent more detailed analysis and processing.

[0072] Construction details of constraint combination table T: The top component groups are automatically selected based on saliency calculations; popular coarse registration methods such as three-point, three-surface, three-sphere, cylinder, and cone are expanded and attached to all identifiable components to achieve effective constraints on spatial displacement of identifiable components. At the same time, sufficient constraint combinations for all identifiable components are solved and recorded in a list.

[0073] RANSAC parameter setting and optimization process: The point cloud is filtered using the point cloud recognition and annotation results to obtain a new point set that retains only planar points. Based on this, the planar point set is divided into multiple point clusters using the angle between the normal vectors of the points as the distance criterion. Each point within a cluster belongs to mutually parallel planes. Furthermore, each point cluster is sub-clustered using the point offset as the classification basis, resulting in multiple self-clusters. Points within each sub-cluster belong to the same plane. These clusters are filtered using a point count threshold to obtain significant clusters and complete salient plane extraction. The threshold is set to 50,000.

[0074] The dataset used in this paper is derived from point cloud data of actual ship hull segment structures, collected by laser scanner. Data preprocessing steps include point cloud cleaning, denoising, and normalization to ensure data quality and consistency. The results are shown in Table 1:

[0075] (1) Experimental setup

[0076] Hardware configuration:

[0077] Processor: Intel Core i7-9700K

[0078] Memory: 32GB DDR4

[0079] Graphics card: NVIDIA RTX 2080Ti

[0080] Software environment:

[0081] Operating system: Ubuntu 20.04

[0082] Programming language: Python 3.7

[0083] Deep learning framework: PyTorch 1.7.1

[0084] (2) Model training

[0085] Training parameters:

[0086] Learning rate: 0.001

[0087] Batch size: 32

[0088] Iterations: 100 epochs

[0089] Training process:

[0090] Data loading: Use PyTorch's DataLoader class to load the training dataset.

[0091] Model initialization: Use the pre-trained ResNet as the base model and add a custom point cloud processing module.

[0092] Loss function: Use the mean squared error (MSE) loss function.

[0093] Optimizer: Use Adam optimizer.

[0094] (3) Testing process

[0095] Test dataset:

[0096] The test dataset contains 4 different hull segment structures, each of which contains multiple point cloud data blocks, ensuring the diversity and representativeness of the test dataset.

[0097] Evaluation Metrics:

[0098] Loss function: Calculates the average loss of the model on the test set.

[0099] Accuracy: Calculates the classification accuracy of the model on the test set.

[0100] Runtime: records the average run time of the model on the test set.

[0101] Root mean square error (RMSE): Calculate the RMSE1 and RMSE2 of the model on the test set, which represent the overall error and local error of point cloud registration, respectively.

[0102] Table 1

[0103]

[0104] From the analysis of the data in the above table, we can see that:

[0105] Low training and test loss: Experimental data shows that our network exhibits low loss during training and testing. In particular, during the test phase, the loss is only 0.65, much lower than other methods, indicating that the model has good generalization ability.

[0106] High training and testing accuracy: Our network performs well in both training and testing, achieving a test accuracy of 93.35%, comparable to or even slightly better than existing state-of-the-art methods such as PointNet++. This demonstrates the model's high accuracy in identifying and annotating point clouds.

[0107] Short Runtime: Our method achieved significantly lower runtimes than most comparison methods across multiple test segments. For example, in Block I and Block IV, the runtimes were 31 seconds and 25 seconds, respectively, significantly lower than methods like PointNet and PointNet++. This demonstrates the high efficiency of our method in practical applications.

[0108] Low Root Mean Square Error (RMSE): Our method achieves excellent RMSE1 and RMSE2 across multiple test segments. In particular, in Block I and Block II, RMSE1 is 119.78 mm and 44.42 mm, respectively, significantly lower than other methods. RMSE2 remains low across all test segments, particularly in Block I, where it reaches a low RMSE2 of 4.53, demonstrating exceptionally high registration accuracy.

[0109] Robustness and Stability: Data from multiple test segments demonstrates the robustness and consistency of the proposed method across various scenarios, demonstrating both robustness and consistency in both time consumption and registration accuracy. This demonstrates the wide applicability and reliability of the proposed method in practical applications.

[0110] In summary, this invention significantly improves the accuracy and efficiency of point cloud registration through deep learning and probabilistic correspondence methods, making it particularly suitable for the rapid and lean assembly of complex ship hull segments. Experimental data and analytical results fully demonstrate the superior performance and promising prospects of this invention in practical applications.

[0111] 2. For manual coarse registration algorithm:

[0112] A simple and direct human-computer interaction interface was developed to display the segmented and annotated original point cloud in different color gradations. The operator only needs to complete the manual selection of salient components according to the constraint combination table T in the above-mentioned automatic coarse registration algorithm. In the point cloud display interface, any combination of components that meets the requirements of the constraint combination table T (for example, any combination of three non-parallel salient planes) can complete the coarse registration.

[0113] Figure 4 The human-computer interaction registration process of segmented point cloud and design model is presented. Plane information is still used as a significant feature. The constraint combination table T records all possible pairing relationships between segmented point cloud and three-plane groups in the model. A set of feature correspondences in T is manually selected through human-computer interaction to obtain Figure 4 The coarse registration result shown on the far right demonstrates that after manual coarse registration, the relative positions of the segmented point cloud and the data model in 3D space are roughly aligned. This proves that coarse registration can be completed by selecting any combination of components that meet the requirements of constraint combination table T, with good results and simple and convenient operation.

[0114] 2. Point cloud precise registration algorithm based on probability correspondence:

[0115] like Figure 2 As shown in the figure, the flow chart of the point cloud precise registration algorithm based on probability correspondence is shown in the figure.

[0116] 1. Point cloud simplification:

[0117] (1) Based on the previous point cloud recognition and annotation results, the neighboring CAD model components of each supervoxel are calculated (using Hausdorff distance) under the coarse registration pose, and the deviation value between the recognition information and the neighboring component information is calculated (category deviation, for example, the deviation value of the same category is 0, and the deviation value of different categories is +∞). If there is no similar CAD model component near the supervoxel, it is considered a redundant structure and is eliminated;

[0118] (2) For the valid point cloud retained in step (1), the supervoxel volume is used as a unit, and the intersection of the regions is calculated as the key parts based on the point cloud region growing algorithm, and it is marked as key points and key lines based on the principal component analysis method and edge detection method, and the key surface is selected according to the number of points and flatness of each region;

[0119] (3) Finally, each supervoxel is traversed for non-uniform simplification, and a larger sampling coefficient is set for the key parts (key points, key lines, and key surfaces) (for example, the sampling coefficient of the key parts is 0.25, and the sampling coefficient of the rest is 0.1), thereby obtaining a low-throughput point cloud for subsequent fine alignment.

[0120] Supervoxel Partition Parameters: First, the original point cloud is voxelized. Voxels are small squares containing a number of points. This divides the original point cloud into individual voxels. Based on current voxel growth theory, region growing is performed by pre-defining characteristic parameters for each voxel. To comprehensively consider the distribution of the point cloud within the voxel, characteristic parameters are selected, and growth criteria and thresholds are established to generate individual growth regions. Furthermore, considering that the faceplate and web of T-shaped profiles and angles are at a 90-degree angle, region growing cannot grow them together, so a secondary region growing process is required. Since the web and faceplate of the profile are smaller, the regions they form contain fewer voxels, while the plate is larger, the regions they form contain more voxels. Therefore, the region after the initial growth is partitioned, separating regions with fewer voxels from regions with more voxels. A secondary growth process is performed on the regions with fewer voxels, merging the web and faceplate of the profile into a single region.

[0121] Region Growing Algorithm Parameters: To maximize plane segmentation, the curvature of each point is calculated. First, a curvature threshold is applied to remove points along sharp edges. Then, points with the smallest curvature are selected in descending order of curvature to serve as "seed points." This "seed point" selection method ensures that the algorithm grows from the flattest areas, minimizing the number of segmented planes and ultimately obtaining a complete plane. The normal angle is selected as the classification criterion; growth stops when the normal angle exceeds the threshold.

[0122] Edge detection parameters: Region growing is an iterative process, where each seed point is iteratively grown until all the data in the point cloud has been processed, thus forming different regions whose boundaries are defined by the growing stop set.

[0123] Non-uniform simplification parameters, key part sampling coefficients, and non-key part sampling coefficients. Non-uniform sampling technology is used for each area composed of plates and profiles, with a larger scale of downsampling performed on the area where the plates are located and a smaller scale of downsampling performed on the area where the profiles are located, thus highly simplifying the original point cloud.

[0124] 2. Point corresponding probability function fc:

[0125] For the simplified point cloud, the point correspondence probability function fc considers its inherent three types of simple geometric information - curvature, normal, and three-dimensional coordinates, the supervoxel identification information to which the point belongs, the key part information of the point and the process constraint information of the adjacent CAD model, and the surface information to which the point belongs. Based on the Gaussian function and spatial grid decomposition, the local correspondence probability matrix between the point to be considered and its adjacent CAD points, lines, and surfaces is given (calculated by the point correspondence probability function).

[0126] Gaussian kernel parameter (σ): The Gaussian kernel function can map the input data into a high-dimensional space, while other kernel functions can only map into a finite-dimensional space. It is defined as:

[0127]

[0128] Where ρ(e) is the cost function induced by maximizing the relevant entropy, satisfying:

[0129] (1)ρ(e)≥0;

[0130] (2)ρ(0)=0;

[0131] (3)ρ(e)=ρ(-e);

[0132] (4) If |e_i|>|e_j|, then ρ(e_i)≥ρ(e_j).

[0133] Multi-information fusion: The point correspondence probability function fc takes into account a variety of information, including inherent geometric information (curvature, normal, 3D coordinates), supervoxel identification information, key component information, adjacent CAD model process constraints, and surface information. This multi-information fusion approach significantly improves the accuracy and robustness of point cloud registration.

[0134] Gaussian function and spatial grid subdivision: Through Gaussian function and spatial grid subdivision, the local correspondence probability matrix between the point to be considered and its neighboring CAD points, lines, and surfaces can be effectively calculated, thereby improving the accuracy of registration.

[0135] Gaussian kernel parameter (σ): The setting of the Gaussian kernel parameter (σ) enables the input data to be mapped into a high-dimensional space, thereby better capturing the complex structure and characteristics of the point cloud data.

[0136] 3. ICP optimization function of point cloud-CAD model based on probability correspondence:

[0137] Based on the CPD probability matrix and point cloud-CAD target optimization formula, the optimization function considering the corresponding probability matrix is ​​formulated as follows: (The specific meanings of the variables involved in the function are as follows.) The optimization equation was solved using the quaternion derivation method of Horn et al. Ultimately, the ICP algorithm was improved based on the results of the optimization equation solution. A simplified point cloud and a CAD model (a collection of STL triangles) were used as the registration objects. Each step iterated through each point in the point cloud, calculating a local probability correspondence matrix. These calculated local probability correspondence matrices were assembled into a global probability matrix, and the optimization objective function was then solved, gradually approaching the local optimal solution.

[0138] N: the number of moving point cloud points

[0139] M: The number of fixed point cloud points

[0140] P nm : corresponds to the nth row and mth column element in the probability matrix

[0141] R: rotation matrix to be solved

[0142] t: displacement matrix to be solved

[0143] x n : Move the nth point in the point cloud

[0144] y m : fix the m-th point in the point cloud.

[0145] Optimization function: Based on the CPD probability matrix and the point cloud-CAD target optimization formula, the developed optimization function can effectively consider the correspondence between the point cloud and the CAD model, thereby improving the accuracy and robustness of the registration.

[0146] Quaternion derivation method: The quaternion derivation method of Horn et al. can efficiently solve the optimization equation and improve computational efficiency.

[0147] Iterative process: Each step iterates through each point in the point cloud, calculates the local probability correspondence matrix, and assembles it into a global probability matrix, gradually approaching the local optimal solution. This iterative process ensures high accuracy and stability of the registration results.

[0148] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A point cloud registration method based on semantic annotation results for complex ship segment structures, characterized by: The following steps are involved:

1. Rough registration algorithm based on salient structure: (1) Automatic coarse registration algorithm: The coarse registration method for constraining spatial rigid body degrees of freedom is extended to include significant components in constraining spatial rigid body degrees of freedom. At the same time, sufficient constraint combinations for all identifiable components are solved and recorded as the constraint combination table T. For the salient component group, the components are arranged to form all feasible rough registration constraint sets according to the component category labels and the aforementioned constraint combination table T. Finally, the optimal rough registration result is calculated using the RANSAC principle. (2) Manual coarse registration algorithm: For the segmented and annotated original point cloud, the coarse registration can be completed by manually selecting the significant components according to the constraint combination table T in the above-mentioned automatic coarse registration algorithm; 2. After selecting either the automatic coarse registration algorithm or the manual coarse registration algorithm, perform the probability-based point cloud fine registration algorithm: (1) For the simplified point cloud, the point correspondence probability function fc is used to give the local correspondence probability matrix between the point to be considered and its neighboring CAD points, lines, and surfaces based on the Gaussian function and spatial grid subdivision; (2) Based on the CPD probability matrix and the point cloud-CAD target optimization formula, an optimization function considering the corresponding probability matrix is ​​formulated as shown in Formula I. According to the quaternion derivation method of Horn et al., the optimization equation is solved. Finally, the ICP algorithm is improved based on the solution of the optimization equation. The simplified point cloud and CAD model are used as the registration objects. Each step iterates through each point in the point cloud, calculates the local probability correspondence matrix, and assembles the calculated local probability correspondence matrix into a global probability matrix. Then, the optimization objective function is solved and the local optimal solution is gradually approached. The formula I is: N: the number of moving point cloud points; M: the number of fixed point cloud points; P nm : corresponds to the nth row and mth column element in the probability matrix; R: the rotation matrix to be solved; t: the displacement matrix to be solved; x n : Move the nth point in the point cloud; y m : fix the m-th point in the point cloud.

2. The method according to claim 1, characterized in that The fully constrained combination solution of all identifiable components is to find two structural parts with the same category in the two point clouds to be registered, use the PCA method to perform coarse registration on the two components, and evaluate the registration result through the coarse registration residual.

3. The method according to claim 1, characterized in that The method for obtaining significant components is to formulate a significant measurement function fs that comprehensively considers surface area information, component category information and process weight information. After calculating the significant measurement function fs, the components ranked at the top are automatically selected as significant components, and their collection is used as a significant component group.

4. The method according to claim 1, wherein The step of manually selecting significant components includes selecting any set of component combinations that meet the requirements of the constraint combination table T in the point cloud display interface.

5. The method according to claim 1, characterized in that The step of selecting any group of component combinations that meet the requirements of the constraint combination table T includes selecting any three significant plane combinations that are not parallel to each other.

6. The method according to claim 1, characterized in that The point corresponding probability function fc in step 2 (1) takes into account its inherent three types of simple geometric information - curvature, normal, and three-dimensional coordinates, the supervoxel identification information to which the point belongs, the key part information of the point and the process constraint information of the adjacent CAD model, and the surface area information to which the point belongs.

7. The method according to claim 1, characterized in that The point cloud simplification method in step (1) of the second step comprises the following steps: ① Based on the previous point cloud recognition and annotation results, the neighboring CAD model components of each supervoxel are calculated in the coarse registration pose, and the deviation between the recognition information and the neighboring component information is calculated. If there is no similar CAD model component near the supervoxel, it is considered a redundant structure and is eliminated; ② For the valid point cloud retained in step ①, use the supervoxel as the unit, and use the point cloud region growing algorithm to solve the regional intersection as the key part. Then use the principal component analysis method and edge detection method to mark it as key points and key lines, and select the key surface according to the number of points and flatness of each region; ③Finally, each supervoxel is traversed for non-uniform simplification, and a larger sampling coefficient is set for the key parts to obtain a low-flux point cloud for subsequent fine registration.

8. The method according to claim 7, characterized in that In the point cloud simplification method, the method of calculating the neighboring CAD model components of each supervoxel under the coarse registration posture is to use the Hausdorff distance for calculation.

9. The method according to claim 7, characterized in that The key parts of ③ in the point cloud simplification method include key points, key lines, and key surfaces.

Images