Neural network based search radius adaptive icp registration method

Abstract

The application discloses a search radius adaptive ICP registration method based on a neural network, relates to the field of multi-view three-dimensional point cloud registration technology, and comprises the following steps: collecting multi-view point cloud data and preprocessing; converting the point cloud from a camera coordinate system to a mechanical arm base coordinate system, combining a Z threshold and a clustering method, and performing preliminary coarse registration through two-stage processing; adaptively predicting a search radius through a trained multi-perception neural network according to the characteristics of an overlap rate, a source point cloud density and an overlapping area point cloud density; and based on a point-to-plane iterative closest point algorithm, introducing normal vector information of target points in error modeling, and registering point cloud data of different views with point cloud data of a main view. Therefore, the search radius adaptive ICP registration method based on the neural network can ensure good registration accuracy, improve the robustness and automation degree of registration, and significantly reduce artificial intervention and overall workload.

Description

Technical Field

[0001] This invention relates to multi-view Figure 3 The field of point cloud registration technology, especially the adaptive ICP registration method based on search radius using neural networks. Background Technology

[0002] With the rapid development of industrial automation and intelligent manufacturing technologies, multi-view Figure 3 Point cloud registration has been widely applied in engineering tasks such as painting, assembly, polishing, and welding of complex workpieces. As one of the mainstream registration algorithms, the Iterative Closest Point (ICP) algorithm has many advantages. Based on the principles of nearest neighbor correspondence and least squares optimization, the ICP concept is intuitive and easy to implement, lowering the application threshold. Its performance in terms of accuracy is particularly outstanding. Through iterative optimization, it continuously refines the transformation relationships between point clouds, achieving sub-voxel level accuracy under suitable conditions, meeting the needs of high-precision scenarios such as assembly.

[0003] The ICP algorithm is highly versatile. Besides its excellent performance in industrial workpiece processing, it has also been effectively applied in fields such as robotics, medical imaging, 3D reconstruction, and autonomous driving. ICP also exhibits strong scalability, leading to numerous variants tailored to different scenarios and needs, such as point-to-point ICP variants, enhanced ICP algorithms based on point cloud curvature similarity, and ICP algorithms that jointly learn local feature detection and description.

[0004] While existing ICP algorithms have made significant progress in feature extraction, global optimization, and matching mechanisms, most methods still rely on manually setting the search radius or neighborhood size. This not only increases the difficulty of parameter tuning but also limits the algorithm's adaptability in dynamic and complex environments. To address this issue, this invention proposes a search radius prediction method based on a multilayer perceptron (MLP). By training a model, it automatically infers the optimal search radius for different point cloud registration tasks, achieving parameter adaptation and significantly improving the robustness and automation of ICP registration, thus better adapting to the complex needs of multi-view point cloud processing. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive ICP registration method based on a neural network, which can adaptively adjust the optimal search radius under different point cloud registration tasks, making it suitable for multi-view point cloud processing in dynamic and complex environments and enhancing the robustness of ICP registration.

[0006] To achieve the above objectives, this invention provides a neural network-based adaptive ICP registration method for search radius, comprising the following steps:

[0007] S1. For complex workpieces in engineering, a robotic arm equipped with a Mech-Mind camera is used to collect point cloud data from different perspectives, and the collected point cloud data is preprocessed; the point cloud data from each perspective has an overlapping area with the point cloud data from the main perspective that meets the registration requirements.

[0008] S2. The preprocessed point cloud data is converted to the robot arm base coordinate system. Combined with Z-thresholding and clustering methods, preliminary coarse registration is performed through two-stage processing to obtain point cloud data with background and isolated noise removed.

[0009] S3. An adaptive search radius prediction method based on neural networks uses overlap rate, source point cloud density, and overlapping region point cloud density as input features. In different point cloud registration tasks, the trained multi-perceptron neural network is used to adaptively predict the search radius.

[0010] S4. Based on the predicted search radius, the point cloud data from different perspectives are registered with the point cloud data from the main perspective using the iterative nearest point algorithm from point to surface.

[0011] Among them, the iterative nearest point algorithm from point to surface introduces the normal vector information of the target point in error modeling, so that the registration process focuses on the consistency of geometric structure.

[0012] Furthermore, S2 includes adaptive estimation of the foreground extent on the Z-axis and density filtering of the coarse foreground using DBSCAN clustering.

[0013] Furthermore, the foreground range is adaptively estimated on the Z-axis, including obtaining the maximum height of the point cloud in the robot arm's base coordinate system, and traversing downwards through a sliding window with a fixed step size until the number of points in the sliding window is less than a set threshold. If the number of points is less than a set threshold, it is considered that the target bottom surface has been crossed, and the height at this point is set as the minimum height of the point cloud. Based on the minimum and maximum height values, the point cloud is initially divided into coarse foreground and background. The coarse foreground includes the target and isolated noise points.

[0014] Furthermore, density filtering of the coarse foreground is performed using DBSCAN clustering, which includes performing DBSCAN clustering on the point cloud of the coarse foreground, retaining the categories that meet the minimum number of points to retain clusters, and obtaining point cloud data with background and isolated noise removed.

[0015] Furthermore, S3 includes constructing a training sample set, using overlap rate, source point cloud density, and overlapping region point cloud density as input features, and the search radius as output, to train the multi-perceptron neural network.

[0016] Furthermore, in S4, the objective function expression for error modeling is as follows:

[0017]

[0018] In the formula, E(R,T) represents the sum of squared distances from a point to a surface, and p i q represents a point in the source point cloud. i Indicates the target point cloud and p i Matching corresponding points, n i For the corresponding point q i The normal vector, R and T are the rotation matrix and translation vector, respectively;

[0019] By minimizing the objective function, the registration of point clouds from different viewpoints with the main viewpoint point cloud is achieved.

[0020] Therefore, the present invention employs the above-mentioned neural network-based adaptive ICP registration method with the search radius, which has the following technical advantages:

[0021] Based on the overlap rate, source point cloud density, and overlapping region point cloud density characteristics, this invention, with the assistance of a multilayer perceptron neural network, enables the ICP registration process to dynamically adjust parameters according to actual scene characteristics. This ensures good registration accuracy while improving the robustness and automation of registration, significantly reducing manual intervention and overall workload.

[0022] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0023] Figure 1 This is a flowchart of the neural network-based adaptive ICP registration method based on search radius.

[0024] Figure 2 This is a schematic diagram of the neural network structure in an embodiment of the adaptive ICP registration method based on the search radius of a neural network;

[0025] Figure 3 This is a partial point cloud registration visualization of Example 2 of the adaptive ICP registration method based on neural network search radius, where (a) search radius r = 0.1, (b) search radius r = 0.3, (c) search radius r = 0.5, (d) search radius r = 5, (e) search radius r = 7, and (f) adaptive search radius. Detailed Implementation

[0026] The present invention will be explained in more detail through the following embodiments. The purpose of disclosing the present invention is to protect all changes and modifications within the scope of the present invention. The present invention is not limited to the following embodiments.

[0027] Example 1

[0028] like Figure 1 As shown, this invention provides a neural network-based adaptive ICP registration method for search radius, comprising:

[0029] For complex workpieces in engineering, a Fanuc robotic arm equipped with a MekaRobotics camera is used to collect multi-viewpoint cloud data, and each frame of point cloud and the main-viewpoint cloud (i.e., the target point cloud) have overlapping regions sufficient to meet the registration requirements. Specifically, first, the camera is calibrated for hand-eye calibration, and then data is collected from 14 viewpoints on the front and back of the workpiece respectively to obtain the initial point cloud data, and then the point cloud is preprocessed by statistical filtering.

[0030] Due to the existence of hand-eye calibration errors and the gradual accumulation of errors during the multi-view fusion process, in this embodiment, the collected point cloud is transformed from the camera coordinate system to the robotic arm base coordinate system to achieve preliminary rough registration, providing a good initial pose for the fine registration of the subsequent ICP algorithm, thereby improving the convergence speed and accuracy of the overall registration.

[0031] Among them, the transformation matrix of the coordinate system is:

[0032]

[0033] In the formula, is the rotation matrix, is the translation vector.

[0034] Let the point cloud of the i-th frame in the camera coordinate system be P i [[ID= / / ]]Not sure what's after this in the original, assuming it's a subscript or something. For now, just keeping the text as is. / / Then the point cloud is represented as

[0035] After converting all the point cloud data to the robotic arm base coordinate system, it is found that the messy background is concentrated in the Z-axis direction below the workpiece, and there are also a small number of miscellaneous points around the workpiece. For this, this embodiment adopts a two-stage processing method that does not rely on a priori models:

[0036] In the first stage, the foreground range is adaptively estimated on the Z-axis as follows:

[0037] Let the point cloud P = {p i =(x i , y, z i )|i = 1,..., N} in the robotic arm base coordinate system, where N is the number of point clouds. First, obtain the maximum height z max of the point cloud, and traverse downward from z max with a fixed step size Δz. When the number of points n(z) in the sliding window W(z) = {p i |z - w < z i < z} is less than τ, it is considered that the bottom surface of the target has been crossed, and z min = z is recorded. Among them, w is the height of the sliding window, and τ is the point number threshold. Then, according to the interval [z min , z max , the point cloud is divided into in, Includes target and isolated noise. The background that was removed.

[0038] In the second stage, DBSCAN clustering is used to perform density filtering on the coarsely segmented foreground. Specifically, in the obtained... Perform DBSCAN clustering on the point cloud, and set the cluster label l. i ∈{-1,0,…,K-1}, where K is the effective cluster point. The set of retained clustering indices K. keep Satisfy |{p i |l i =k}|≥γ, where γ is the threshold for the minimum number of points to retain. The clusters that satisfy this condition are taken as the final target point cloud, i.e. Isolated noise points are represented as

[0039] In the registration process of multi-view point clouds, to achieve high-precision alignment between point clouds in different frames, an Iterative Closest Point (ICP) algorithm based on point to surface is adopted. Unlike the traditional point-to-point ICP algorithm, point-to-surface ICP introduces the normal vector information of the target point in error modeling, thus paying more attention to geometric consistency during matching and improving the convergence speed and accuracy of registration. The objective function is defined as follows:

[0040]

[0041] In the formula, p i q represents a point in the source point cloud. i Indicates the target point cloud and p i Matching corresponding points, n i For the corresponding point q i The normal vector is given by R, and the rotation matrix and translation vector are given by T, respectively. Minimizing the objective function means minimizing the sum of squared orthogonal projection distances of the source point onto the corresponding tangent plane in the target point cloud after a rigid transformation, i.e., minimizing the sum of squared distances from the point to the surface, thereby achieving more robust geometric alignment.

[0042] In the point-to-surface ICP algorithm, one of the key steps is to find the corresponding point in the target point cloud for each point in the source point cloud in order to establish point pairs (p... i ,q i A rigid transformation estimation is performed. This correspondence is typically established by finding the nearest neighbor in the target point cloud with a fixed search radius. The setting of the search radius directly affects the registration quality, and there are two main problems:

[0043] (1) Search radius too small: If the radius r is set too small, some source point clouds will not find any neighboring target corresponding points, that is, the corresponding points are missing or mismatched. Let the set of points where the search failed be... These points cannot participate in the error function optimization, which is equivalent to removing them from the function summation, leading to bias in the residual function estimation.

[0044]

[0045] Reducing the number of registration points will decrease the binding force of the registration, which may easily lead to unstable or divergent registration results.

[0046] (2) Search radius is too large: If the radius r is set too large, it may cause the matching to points in the structural discontinuity region (such as surface transitions or edges), resulting in an incorrect corresponding point q'. i Even if q' i Distance p i It is close, but its normal vector n' i There may be a significant deviation from the actual tangent plane direction, resulting in an inner product term (R·p) i +Tq' i )·n' i This cannot reflect the true distance from a point to a surface, ultimately causing the overall error function to converge to an incorrect local optimum.

[0047] Because point-to-surface errors are highly sensitive to the direction of the normal vector, the aforementioned problems can be severely amplified in complex scenes. Traditional methods often rely on experience or manual parameter tuning to set the search radius, but in multi-frame, multi-view point cloud fusion tasks, manually adjusting the search radius for each frame is extremely inefficient and difficult to maintain consistency. Therefore, this embodiment proposes a neural network-based adaptive search radius prediction method as a preprocessing module for ICP registration. This method automatically adjusts the appropriate search radius for different scenes and point cloud densities, enhancing the generalization ability and practicality of the point-to-surface ICP algorithm.

[0048] To fit the mapping relationship between the features of different point clouds and their corresponding suitable search radii, this embodiment uses a multi-layer perceptron (MLP) neural network for supervised regression modeling. This model has strong nonlinear fitting capabilities and is suitable for continuous variable prediction tasks in high-dimensional feature spaces, as detailed below:

[0049] First, a training sample set was constructed, with each sample consisting of three dimensions of input features: overlap rate, source point cloud density, and overlapping region point cloud density. The corresponding output is the search radius required for ICP registration. Input vector x (i) and output vector y (i) It can be represented as:

[0050]

[0051] The training set is defined as N' is the number of sample points in the training set.

[0052] Then, a fully connected neural network with a single hidden layer was designed, such as... Figure 2 As shown. The mathematical expression is as follows:

[0053] Input layer:

[0054] Hidden layer: h1 = ReLU(w1h0 + b1);

[0055] Output layer:

[0056] Among them, weight bias i = 1, 2; ReLU is the activation function, defined as ReLU(x) = max(0, x), which is used in both the hidden and output layers to ensure the non-negativity of the output radius. Simultaneously, mean squared error is used as the loss function for the regression model, and the Adam optimizer is used to update the model parameters.

[0057] A suitable search radius is predicted using a trained neural network, and point-to-surface ICP (Internal Point Collation) registration is performed. Since the purpose of this embodiment is to select a suitable search radius, and the commonly used evaluation metric for the ICP algorithm, RMSE, relies on the calculation of interior points used in the algorithm, as the radius increases, the ICP algorithm will introduce more outliers and misidentify them as interior points, leading to an increase in RMSE. Therefore, this embodiment selects the Chamfer distance of the overlapping region as the evaluation metric. This metric is similar to RMSE but does not depend on interior points. Specifically, the Chamfer distance is calculated by taking the distance from each point in the source point cloud to the nearest point in the target point cloud, and the distance from each point in the target point cloud to the nearest point in the source point cloud, thus reflecting the spatial coverage and shape similarity between the two. The expression is as follows:

[0058]

[0059] Where P is the source point cloud set, Q is the target point cloud set, and |P| and |Q| are the number of points in P and Q, respectively.

[0060] In this embodiment, the workpiece main point cloud in the first frame of the 14-frame view is taken as the target point cloud, and the remaining 13 frame point clouds are taken as the source point clouds. The registration accuracy of the complete workpiece is represented by the average Chamfer distance, which is the average Chamfer distance between the 13 frame source point clouds and the target point cloud after registration. The smaller the average Chamfer distance, the better the registration effect.

[0061] Example 2

[0062] To verify the effectiveness of the neural network-based adaptive ICP registration method, this embodiment selects the front of workpiece 1 as the research object. The registration results of multiple frame point clouds with the same manually selected search radius are compared and analyzed with the method of this invention. Figure 3 As shown in Table 1.

[0063] Table 1. Manually selected radius and registration parameters of the method of the present invention

[0064] algorithm Radius (mm) Mean Chamfer distance (mm) Manual selection 0.1 6.917 Manual selection 0.15 6.112 Manual selection 0.2 6.049 Manual selection 0.25 5.672 Manual selection 0.3 4.716 Manual selection 0.35 4.497 Manual selection 0.4 5.287 Manual selection 0.45 4.478 Manual selection 0.5 4.836 Manual selection 5 2.899 Manual selection 6 2.932 Manual selection 7 2.967 This invention Adaptive 2.870

[0065] As shown in Table 1, the present invention exhibits the smallest mean Chamfer distance, resulting in better registration performance. In summary... Figure 3 It can be seen that when the search radius is too small, the registration effect is poor, and some point clouds at certain viewpoints may even deviate from the main point cloud. On the other hand, when the radius is too large, the registration may get stuck in a local optimum. Even if the overall registration effect is good, there may still be deviations in the registration of some details of the point cloud. Among them, r=5 is closest to the registration effect of the method of the present invention, but the registration effect of the lower right corner of the workpiece is better under the method of the present invention.

[0066] Example 3

[0067] To verify the applicability of the neural network-based adaptive ICP registration method under different point cloud conditions, this embodiment selects the front and back faces of workpiece 1 and another complex workpiece 2 as research objects. The registration effects under different point clouds are compared and analyzed, and the measured values ​​of the registered workpieces are analyzed in conjunction with the actual measured values, as shown in Tables 2 and 3. Table 2 shows that the mean Chamfer distance of the method of this invention reaches the minimum under different point clouds. Furthermore, the visualization effect of ICP registration is significantly better than that of traditional ICP registration with manually selected radii.

[0068] Table 2. Manually selected radius and registration parameters of the present invention under different point clouds.

[0069]

[0070] Table 3 Comparison of Actual Measured Values ​​and Registration Results Measured Values

[0071]

[0072]

[0073] As shown in Table 3, the maximum absolute registration error of the method of the present invention does not exceed 4 mm, and the minimum absolute error is less than 0.1 mm. Therefore, the method of the present invention has high registration accuracy and practicality.

[0074] Therefore, the above-mentioned neural network-based adaptive ICP registration method can ensure good registration accuracy, while improving the robustness and automation of registration, and significantly reducing manual intervention and overall workload.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A neural network-based adaptive ICP registration method for search radius, characterized in that, Includes the following steps: S1. For complex workpieces in engineering, a robotic arm equipped with a Mech-Mind camera is used to collect point cloud data from different perspectives, and the collected point cloud data is preprocessed; the point cloud data from each perspective has an overlapping area with the point cloud data from the main perspective that meets the registration requirements. S2. The preprocessed point cloud data is converted to the robot arm base coordinate system. Combined with Z-thresholding and clustering methods, preliminary coarse registration is performed through two-stage processing to obtain point cloud data with background and isolated noise removed. This includes adaptive estimation of the foreground extent on the Z-axis and density filtering of the coarsely segmented foreground using DBSCAN clustering; The foreground range is adaptively estimated on the Z-axis, including obtaining the maximum height of the point cloud in the robot arm's base coordinate system, and traversing downwards through a sliding window with a fixed step size until the number of points in the sliding window is less than a set threshold. At this point, it is considered that the target bottom surface has been crossed, and the height at this time is set as the minimum height of the point cloud. Based on the minimum and maximum height values, the point cloud is initially divided into coarse foreground and background. The coarse foreground includes the target and isolated noise points. Density filtering of coarse foreground is performed using DBSCAN clustering, which includes performing DBSCAN clustering on the point cloud of coarse foreground, retaining the categories that meet the minimum number of points to retain clusters, and obtaining point cloud data with background and isolated noise removed. S3. An adaptive search radius prediction method based on neural networks uses overlap rate, source point cloud density, and overlapping region point cloud density as input features. In different point cloud registration tasks, the trained multi-perceptron neural network is used to adaptively predict the search radius. S3 includes constructing a training sample set, using overlap rate, source point cloud density, and overlapping region point cloud density as input features, and using search radius as output to train the multi-perceptron neural network; S4. Based on the predicted search radius, the point cloud data from different perspectives are registered with the point cloud data from the main perspective using the iterative nearest point algorithm from point to surface. Among them, the iterative nearest point algorithm from point to surface introduces the normal vector information of the target point in error modeling, so that the registration process focuses on the consistency of geometric structure.

2. The neural network-based adaptive ICP registration method for search radius as described in claim 1, characterized in that, In S4, the objective function expression for error modeling is as follows: ； In the formula, This represents the sum of the squares of the distances from a point to a surface. Represents points in the source point cloud. Indicates the target point cloud and Matching corresponding points, For corresponding points The normal vector, , These are the rotation matrix and the translation vector, respectively. By minimizing the objective function, the registration of point clouds from different viewpoints with the main viewpoint point cloud is achieved.

Images