Anti-rotation color point cloud registration network

Abstract

The invention discloses an anti-rotation color point cloud registration network which comprises a feature extraction trunk, a geometric color overpoint matching module and a point matching-registration module which are sequentially arranged in the transmission direction of a data stream. The feature extraction trunk comprises a residual block and a nonlinear layer, the residual block is used for acquiring sparse point cloud features and dense point cloud features of the input point cloud, and geometric color rotation isovariant features are extracted through the nonlinear layer; the geometric color super-point matching module is used for encoding a global context related to transformation invariance so as to obtain a steady patch corresponding relation, so that geometric color features are kept stable under rotation change, and the influence of minimizing rotation is realized; besides, the dual gradient constraint of geometric color information is used for remarkably enhancing the discrimination capability of a local structure, so that the dual interference of rotation transformation on feature extraction and matching is systematically solved, and excellent registration performance is shown in a scene with obvious rotation and low overlap.

Description

Technical Field

[0001] This invention relates to the field of color point cloud registration technology, specifically to a rotation-resistant color point cloud registration network. Background Technology

[0002] As a core task in computer vision and robotics, point cloud registration aims to align point cloud data collected from different perspectives or sensors into a unified coordinate system by estimating rigid transformations. Its accuracy directly determines the reliability of downstream tasks such as 3D reconstruction, autonomous driving, and augmented reality.

[0003] Although point cloud registration has been extensively studied, in highly challenging scenarios with single geometric features and extremely low overlap, geometric information alone is insufficient to extract accurate and discriminative features, leading to mismatches. Color data, as supplementary information, offers new possibilities for solving this problem. Existing color point cloud registration methods typically employ rotation-sensitive networks to extract features and utilize rotation enhancement to roughly learn approximate invariant properties, making the registration network highly sensitive to rotation. For example, Color-ICP introduces color consistency constraints into the ICP framework and optimizes pose error through CIRLAB color space transformation and photometric error minimization; 4DICP converts color information to the HSV color space to improve accuracy and elevate registration to a better dimensional space; PEAL uses RGB color channels as independent input dimensions, utilizes existing 2D image techniques to detect overlapping regions from images, and then transmits them to a 3D registration network as prior information; ColorPCR, based on KPConv-FPN, uses color-enhanced feature extraction to achieve multi-stage geometric color fusion. However, most of the aforementioned methods employ rotation-sensitive networks to extract features or use color as prior information to detect key points, resulting in the inability to fully utilize effective information during rotational changes and the loss of key information. Therefore, designing an anti-rotation network for color point cloud registration has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a rotation-resistant color point cloud registration network to solve the technical problems mentioned in the prior art.

[0005] A rotation-resistant color point cloud registration network includes a feature extraction backbone, a geometric color superpoint matching module, and a point matching-registration module arranged sequentially along the data flow transmission direction, wherein;

[0006] The feature extraction backbone includes residual blocks and nonlinear layers. The residual blocks are configured to: downsample the input point cloud to obtain sparse point cloud features, and upsample the sparse point cloud features to obtain dense point cloud features. The nonlinear layers are configured to: extract the sparse point cloud features and fuse the dense point cloud features to obtain geometric color rotation and other similar features.

[0007] The geometric color superpoint matching module is configured to: extract the color gradient embedding vector of the geometric color rotation isovariant feature in the superpoint domain according to the covariance analysis method; calculate the geometric relative position embedding vector of the geometric color rotation isovariant feature using geometric coordinates; and calculate the angle embedding vector of the geometric color rotation isovariant feature using the triplet of the superpoint; and construct a superpoint feature vector set based on the one-to-one correspondence between the geometric relative position embedding vector, the color gradient embedding vector, and the angle embedding vector.

[0008] The point matching-registration module is configured to: use SVD to calculate the local changes of the superpoint feature vector set, and select the optimal solution to output the point correspondence.

[0009] Optionally, the residual block includes a downsampling layer, a convolutional kernel, and a nearest neighbor upsampling layer, and the nearest neighbor upsampling layers are connected in skip connections.

[0010] Optionally, the convolution kernel is set to a 3×3 convolutional layer.

[0011] Optionally, the first two convolutional layers upsample the color information of the input point cloud.

[0012] Optionally, the nonlinear layer includes two rotationally invariant layers;

[0013] One of the rotation-invariant layers has its input connected to the output of the convolutional kernel for extracting the sparse point cloud features;

[0014] The input of the remaining rotation-invariant layer is connected to the output of the nearest neighbor upsampling layer to extract the dense point cloud features.

[0015] Optionally, the rotationally invariant layer is configured as follows:

[0016] Point cloud features are learned from rotationally isotropic spatial statistics using the VN-MLP module;

[0017] The size of the vector neurons is calculated by using an L2-Norm layer, and the learned point cloud features are converted into a rotation-invariant form to obtain the geometric color rotation-invariant features.

[0018] The geometric color rotation equivariant feature vector is mapped to K dimensions using the MLP module to obtain the relevant vector;

[0019] The correlation scores of the geometric color rotation isomorphic features are output using the Softmax operation.

[0020] The beneficial effects that this invention can produce include:

[0021] This invention provides a rotation-resistant color point cloud registration network. This network achieves rotation-equivariant geometric color feature enhancement and cross-modal deep fusion through two core modules (RE-ResBlock and VN-Block modules), and a hierarchical feature extraction backbone, REConv-FE, guided by rotation-equivariant kernel convolution. This allows for hierarchical extraction of geometric color information and enhances the network's expressive power to better learn unique geometric color features. Simultaneously, a geometric color superpoint matching module encodes global context related to transformation invariance, thereby obtaining robust patch correspondences that ensure the stability of geometric color features under rotational changes, minimizing the impact of rotation. Furthermore, dual gradient constraints on geometric color information are used to significantly enhance the discriminative ability of local structures, systematically addressing the dual interference of rotational transformations on feature extraction and matching, and exhibiting excellent registration performance in scenes with significant rotation and low overlap. Attached Figure Description

[0022] Figure 1 The network model of the anti-rotation color point cloud registration network designed for this invention;

[0023] Figure 2 This is a schematic diagram of the REConv-FE structure, the feature extraction backbone designed in this invention.

[0024] Figure 3 A schematic diagram of the RE-ResBlock module and VN-Block module in the REConv-FE feature extraction backbone designed in this invention;

[0025] Figure 4 This is a schematic diagram of the registration results between the ColorPCR and RR-Net models in this invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Please see Figure 1As shown, this invention provides a rotation-resistant color point cloud registration network, comprising a feature extraction backbone, a geometric color superpoint matching module, and a point matching-registration module arranged sequentially along the data flow transmission direction. The feature extraction backbone includes residual blocks and nonlinear layers. The residual blocks are configured to: downsample the input point cloud to obtain sparse point cloud features, and upsample the sparse point cloud features to obtain dense point cloud features. The nonlinear layers are configured to: extract and fuse sparse and dense point cloud features to obtain geometric color rotation equivariant features. The geometric color superpoint matching module is configured to: extract the color gradient embedding vector of the geometric color rotation equivariant features in the superpoint domain using covariance analysis, calculate the geometric relative position embedding vector of the geometric color rotation equivariant features using geometric coordinates, and calculate the angle embedding vector of the geometric color rotation equivariant features using superpoint triples, and construct a superpoint feature vector set based on the one-to-one correspondence between the geometric relative position embedding vector, the color gradient embedding vector, and the angle embedding vector. The point matching-registration module is configured to: calculate the local changes of the superpoint feature vector set using SVD, and select the optimal solution to output the point correspondence.

[0028] In the above, such as Figure 1 As shown, the residual block includes a downsampling layer, a convolutional kernel, and a nearest-neighbor upsampling layer, with skip connections between the nearest-neighbor upsampling layers to pass intermediate functions from the encoder to the decoder. The convolutional kernel is set to a 3×3 convolutional layer, and each convolutional layer contains three RE-ResBlock modules. The first RE-ResBlock module in any convolutional layer is a strided block that performs convolution on the downsampling points. Simultaneously, the first two convolutional layers upsample the color information of the input point cloud, allowing color information to be freely injected into the first two convolutional layers, ensuring stability during the superpoint feature extraction process.

[0029] In the above, such as Figure 1 As shown, the nonlinear layer includes two rotation-invariant layers; the input of one rotation-invariant layer is connected to the output of the convolutional kernel to extract sparse point cloud features; the input of the remaining rotation-invariant layer is connected to the output of the nearest neighbor upsampling layer to extract dense point cloud features. Specifically, as... Figure 2 As shown, the rotation-invariant layer is configured as follows:

[0030] (1) Learn point cloud features (including sparse point cloud features and dense point cloud features) from rotationally equivariant spatial statistics using the VN-MLP module;

[0031] (2) The size of the vector neurons is calculated through the L2-Norm layer, and the learned point cloud features are converted into rotation-invariant forms to obtain geometric color rotation-invariant features;

[0032] (3) The geometric color rotation equivariant vector is mapped to K dimensions through the MLP module to obtain the relevant vector;

[0033] (4) Use the Softmax operation to output the correlation scores of geometric color rotation and other similar features.

[0034] The design method of the rotation-resistant color point cloud registration network proposed in this invention includes the following steps:

[0035] Step 1: Construct a hierarchical feature extraction backbone REConv-FE guided by rotationally equivariant kernel convolution.

[0036] Step 1.1: Design of REConv rotational isotropic kernel convolution

[0037] Given two colored point clouds P = {P i ∈R 3 P, i = 1, 2, ..., N c ={P ci ∈[0,1] 3 {i = 1, 2, ..., N} and Q = {q} j ∈R 3 ,j=1,2,…,M},Q c ={q cj ∈[0,1] 3 The expression is defined as follows: N = 1, 2, ..., M. Here, N is the number of point clouds P, M is the number of point clouds Q, Pc is the color of point cloud P, Qc is the color of point cloud Q, and R is the set of real numbers. 3 It is a three-dimensional real number space, that is, a set consisting of ordered arrays of all three real number components.

[0038] Previous point cloud registration methods mainly used KPConv (a deformable convolutional network over point clouds) to extract point cloud features. However, KPConv is inherently sensitive to rotation and requires data augmentation to learn an approximately invariant fractional matrix mapping. This results in the model being trained on a finite number of samples in a continuous SO(3) space (where SO(3) is a special orthogonal group and the set of all rotation matrices in 3D space), which is highly sensitive to rotational changes. Therefore, this invention aims to address this problem, such as... Figure 2 The diagram illustrates a rotationally equivariant kernel convolution (REConv) to learn unique features. Here, given features from P∈R... |P|×3 point x i and from F∈R C×3 Additional features f i ;P∈R |P|×3It is a matrix representation of the point cloud, where |P| represents the total number of points in the point cloud P (i.e., |P| = N). Each row of the matrix corresponds to the three-dimensional coordinates of a point, and the whole matrix forms an N×3 real matrix; F is the feature set of the point cloud. At this point, the kernel function g at point x... i The general convolution of F at a given point can be defined as:

[0039]

[0040] Where, N x ={x i ∈P,||x i The point x ∈ R is defined by the expression -x||≤r, where r∈R. i Let K be the set of K nearest neighbors, where K is the number of nearest neighbors.

[0041] Traditional definitions of the kernel function g ignore positional information and simply use a linear layer WF. i Then, local information is aggregated through VN nonlinear layers and VN pooling layers. Conversely, this invention defines a set of learnable weight matrices {W}. m The shadow kernel points of the point cloud (m = 1, 2, ..., M) differ from the correspondence established between each pixel in image convolution. Due to the irregularity of the point cloud, establishing the correspondence between points and kernels is quite difficult. Therefore, this invention establishes a soft assignment between data points and kernel points according to KPConv. The kernel function can be defined as:

[0042]

[0043] Among them, y i =x i -x is x i The relative coordinates, h(·), are the m and y coordinates. i The relative function between them, W∈R 3×3 This is a learnable weight matrix.

[0044] It should be noted that in KPConv, the relative function uses linear dependence. in Here, σ represents the three-dimensional coordinates of the m-th kernel point, and σ is the distance influence factor described in KPConv, which should be adjusted according to the input density. However, since the distance from the kernel point to the data point is sensitive to rotation, rotational equivariance of the convolution cannot be guaranteed. Therefore, this invention designs a convolution function REConv that effectively acquires the spatial information of the patch while ensuring rotational equivariance of the convolution. Figure 2 As shown, point cloud features, such as x, are first learned from rotationally equivariant spatial statistics using the VN-MLP module. i , And its cross product; then, the vector neuron is calculated through an L2-Norm layer. The size of the feature vector is determined to convert the point cloud features into a rotation-invariant form; finally, the MLP module maps this feature vector to M dimensions to obtain the correlation vector. Then, the relevant scores are output using the Softmax operation. Specifically, REConv can be defined as:

[0045]

[0046] Step 1.2: Feature Extraction Backbone Design

[0047] like Figure 1 and Figure 3 As shown, this invention proposes a hierarchical feature extraction backbone based on REConv, which differs from previous works utilizing four convolutional layers. First, the input point cloud is downsampled, and then three convolutional layers are used to learn features on the sparse point cloud. Each convolutional layer has three RE-ResBlock modules, with the first RE-ResBlock module (RE block) being a strided block that performs convolution on the downsampled points. Then, two nearest-neighbor upsampling layers are used to decrypt the learned sparse point cloud features from sparse points into dense points, resulting in dense point cloud features. It is worth noting that for points in the dense layer, the skipped features are connected to the features of the sparse points closest to the dense points in Euclidean space, and these features are fused by a VN-Block module (VN block). Finally, two rotation-invariant layers are used to obtain geometrically and colorally rotationally equivalent features. In the feature extraction backbone, to ensure stability during superpoint feature extraction, color information is freely injected into the first two convolutional layers, and the last convolutional layer only involves feature concatenation. The specific process of geometrically and colorally rotationally equivalent feature fusion is as follows:

[0048] Given any n convolutional layers, x∈X n From point X on the previous level n The coarse feature set extracted It can be calculated as follows:

[0049]

[0050] Among them, G g Representing REConv, For the point set of layer n-1, It is the feature set of layer n-1.

[0051] Color guidance feature f e It can be calculated using the following formula:

[0052] f e =G c (C n ,X n ,F r,g)(x) (5)

[0053] in, Let G be the set of color points in the nth layer. c A convolution function that includes a color guidance process can be calculated using the following formula:

[0054]

[0055] Among them, f i c It is point x i Features combined with color, f i c =concat([f i ,c i / σ c ]). c i ={h i ,s i ,v i} is point x in the HSV color space i Color, σ c σ is a hyperparameter that controls the intensity of merged colors; in this embodiment, σ c =1.0.

[0056] By combining formulas (3) and (6), color can be successfully incorporated into point features, and geometric color rotation isotropic features with better expressiveness can be obtained.

[0057] Step 2: Construct a geometric color superpoint matching module

[0058] Given two superpoints and its hue (Using the HSV color space to separate the most stable hues), their geometric color position information embedding consists of pair-wise hue-rotation-invariant distance embedding (Pair-wise-HRID) and triplet-wise double-gradient angular embedding (Triplet-wise-DGA), as described in detail below:

[0059] Wherein, for hue rotation invariant distance embedding: given and The hue distance between as well as and Distance between points Color principal components and geometric principal components of the superpoint neighborhood are extracted using covariance analysis. First, for color information, the most stable hue vector {c1,c2,c…,c…} separated from the HSV color space is... n Construct the covariance matrix:

[0060]

[0061] After eigenvalue decomposition (CV = CA), the main color principal component matrices ColorPCA1 and ColorPCA2 are obtained through selection. Here, CV = CA is the expression for eigenvalue decomposition of the color covariance matrix C. V is a matrix composed of eigenvectors, which represent the directions with the maximum variance in the color data; A is a diagonal matrix whose diagonal elements are the corresponding eigenvalues, the magnitude of which reflects the degree of dispersion of the data in the direction represented by the corresponding eigenvector. Similarly, the geometric covariance matrix G is calculated based on the geometric coordinates of the superpoints, and the main geometric principal component matrices GeoPCA1 and GeoPCA2 are obtained after eigenvalue decomposition. To ensure that features of different dimensions and magnitudes have equal weight and comparability in subsequent fusion and calculation, and to avoid interference with model learning due to differences in feature scale, the color and geometric principal component matrices are normalized. Then, rotation-invariant geometric color descriptors are constructed. And combined with hue distance embedding to obtain HRID embedding:

[0062]

[0063] Among them, F s σ is a sine function. HD It is a hyperparameter that can adjust the hue change; α is the balance coefficient used to adjust... Weights in the fusion vector for and The hue distance between them, ρ is the superpoint and The distance between points. The projection matrix for HD embedding. Let d be the projection matrix of the RI embedding. t It is F s The input dimension.

[0064] Specifically, for the ternary dual-gradient angle embedding: this invention follows GeoTransformer (an open-source geospatial data transformation library) to calculate the angle embedding using superpoint triples, while simultaneously utilizing dual gradient constraints of geometric and color information to significantly enhance the discriminative ability of local structures. For superpoints... and Calculate relative coordinates and color gradient Calculating geometric angle features θ based on the principal direction of the local neighborhood in the point cloud geometric space. geo Reference direction e x And the calculation of color angle features θ based on specific axis directions in the color space. color Reference direction e r Combined with reference direction e x and e r Construct angular features:

[0065]

[0066] Where ||Δx|| is the L2 norm of the relative coordinate Δx, used to normalize the vector and ensure the accuracy of angle calculation. ||e x || is the reference direction e x The L2 norm is also used for normalization to ensure that the result of the dot product operation is within a reasonable range, so that the angle can be calculated using the inverse cosine function. Similarly, ||Δc|| is the L2 norm of the color gradient Δc, and ||e r || is the reference direction e r The L2 norm.

[0067] Based on the relative coordinates Δx and color gradient Δc of the two superpoints, the relative position and color gradient embedding vectors are then generated by normalizing and feature mapping these original relative position and color gradient information. And combined with the triplet angle embedding used in the superpoint structure embedding in GeoTransformer, we obtain the DGA embedding:

[0068]

[0069] Where, α k For, σ A These are hyperparameters used to adjust the sensitivity to angle changes. This is the projection matrix for angle embedding. β is a balancing coefficient used to adjust the weights of relative position and color gradient embeddings in the fusion vector. It is the projection matrix of the RPCG embedding. Because It integrates multiple information sources (3D relative position Δx and 1D key color gradient features, with a total feature dimension of 4d). t To map this vector to a d consistent with the angle embedding features. t Dimensions, to achieve effective fusion, projection matrix W RPCG The number of rows must match Input dimension (i.e., 4D) t The number of columns corresponds to the output dimension (i.e., d). t Therefore, the dimension is 4d.t ×d t .

[0070] Finally, the geometric color position information embedding r is calculated by combining HRID Embedding (pairwise hue rotation invariant distance embedding) and DGA Embedding (triple double gradient angle embedding). i,j :

[0071] r i,j =r HRID +r DGA (11)

[0072] In the above, superpoint features with unique characteristics can also be introduced into GeoTransformer (geometric color position information embedding) to extract global contextual information.

[0073] Step 3: Fine-grained registration and transformation estimation

[0074] Fine-grained registration is performed using the point matching module of GeoTransformer. Specifically, SVD is used to calculate the local changes of the superpoint feature vector set, and the optimal solution is selected to output the point correspondence, resulting in the final transformation {R,t}, thus achieving precise registration.

[0075] In this embodiment, to evaluate the performance of RR-Net, state-of-the-art methods are compared with the RR-Net of this invention, and numerous experiments are conducted on the Color3DMatch and Color3DLoMatch datasets collected indoors. All experiments are performed on a system equipped with an RTX4090D GPU and an Intel(R)Core(TM) i9-13900K CPU. The performance evaluation method for RR-Net specifically includes:

[0076] 1) Dataset and Evaluation Metrics

[0077] Data sets collected: Color3DMatch (C3DM) and Color3DLoMatch (C3DLM) are color point cloud datasets obtained by colorizing 3DMatch, where each point has an RGB value.

[0078] 3DMatch contains 62 scenes, configured in a training:validation:test ratio of 23:4:4. C3DM is used to test point cloud pairs with more than 30% overlap, while C3DLM is only used to test point cloud pairs with 10%-30% overlap.

[0079] Evaluation metrics: This invention uses the following five metrics to evaluate registration performance, namely: Inlier Ratio (IR), Feature Matching Recall (FMR), Registration Recall (RR), Relative Rotation Error (RRE), and Relative Translation Error (RTE).

[0080] 2) Quantitative experimental results

[0081] This invention compares RR-Net with recent state-of-the-art RANSAC estimator-based techniques, including FCGF (Fully Convolutional Geometric Feature Extraction), SpinNet (A Neural Network Algorithm for 3D Point Cloud Registration), Predator (A Novel Image Edge Detection Technique Combining Biological Evolutionary Theory and Optimization Algorithms), YOHO (A Deep Neural Network-Based Object Detection Algorithm), CoFiNet (A Point Cloud Matching Algorithm), GeoTransformer (A Deep Learning Algorithm for Point Cloud Processing), RoReg (A Novel Point Cloud Registration Framework), RoITR (An Intrinsic Rotation-Invariant Model for Point Cloud Matching), PEAL (A Load Tracking Algorithm), and ColorPCR (A Chrome extension for coloring PCR classes and test / quiz classes with different colors). The performance of each method was evaluated at five different correspondence counts (5000, 2500, 1000, 500, and 250), and the experimental results are shown in Table 1. As can be seen from Table 1, RR-Net achieves the highest Feature Matching Recall (FMR) on the C3DM and C3DLM datasets across all sampled correspondences. Compared to baseline ColorPCR, RR-Net achieved a more significant improvement in inlier ratio (IR), improving IR by approximately 0.8%–1.3% on C3DM and by approximately 0.9%–4.7% on C3DLM. For registration recall (RR), RR-Net improved to 97.3% on C3DM and 89.7% on C3DLM. This demonstrates that RR-Net's feature matching performance is significantly higher than other models, especially on C3DLM, because REConv-FE can learn more discriminative structural features.

[0082] Table 1. Evaluation results on the C3DM and C3DLM datasets with different sample sizes.

[0083]

[0084]

[0085]

[0086] Furthermore, this invention compares RRE and RTE with recent state-of-the-art techniques based on and without RANSAC estimators, as shown in Table 2. Experimental results show that RR-Net performs best on C3DM and C3DLM.

[0087] Table 2 shows the relative rotation and translation errors of different models on the C3DM and C3DLM datasets.

[0088]

[0089]

[0090] 3) Qualitative experimental results

[0091] This invention is in Figure 4 The image visualizes the registration results of baseline ColorPCR and the proposed method RR-Net, clearly showing the performance gap between them. ColorPCR exhibits alignment errors in some point clouds, while RR-Net successfully resolves this issue because the rotationally equivalent set color features extracted by RR-Net contain directional information about the local structure, achieving more accurate registration with a higher proportion of interior points. Figure 4 As shown, each row displays a registration case in a scene with significant rotation and low overlap, where ColorPCR performs slightly worse. RR-Net can recognize color and rotation in the blue chair (first row), headboard (second row), and sofa (third row). By leveraging rotational equivariance in color point cloud registration, RR-Net achieves robust registration in challenging scenes.

Claims

1. A rotation-resistant color point cloud registration network, characterized in that, It includes a feature extraction backbone, a geometric color superpoint matching module, and a point matching-registration module, which are arranged sequentially along the data flow transmission direction, wherein: The feature extraction backbone includes residual blocks and nonlinear layers. The residual blocks are configured to: downsample the input point cloud to obtain sparse point cloud features, and upsample the sparse point cloud features to obtain dense point cloud features. The nonlinear layers are configured to: extract the sparse point cloud features and fuse the dense point cloud features to obtain geometric color rotation and other similar features. The geometric color superpoint matching module is configured to: extract the color gradient embedding vector of the geometric color rotation isovariant feature in the superpoint domain according to the covariance analysis method; calculate the geometric relative position embedding vector of the geometric color rotation isovariant feature using geometric coordinates; and calculate the angle embedding vector of the geometric color rotation isovariant feature using the triplet of the superpoint; and construct a superpoint feature vector set based on the one-to-one correspondence between the geometric relative position embedding vector, the color gradient embedding vector, and the angle embedding vector. The point matching-registration module is configured to: use SVD to calculate the local changes of the superpoint feature vector set, and select the optimal solution to output the point correspondence.

2. The anti-rotation color point cloud registration network according to claim 1, characterized in that, The residual block includes a downsampling layer, a convolutional kernel, and a nearest neighbor upsampling layer, and the nearest neighbor upsampling layers are connected in a skip connection.

3. The anti-rotation color point cloud registration network according to claim 2, characterized in that, The convolution kernel is set to a 3×3 convolution layer.

4. The anti-rotation color point cloud registration network according to claim 3, characterized in that, The first two convolutional layers upsample the color information of the input point cloud.

5. The anti-rotation color point cloud registration network according to claim 2, characterized in that, The nonlinear layer includes two rotationally invariant layers; One of the rotation-invariant layers has its input connected to the output of the convolutional kernel for extracting the sparse point cloud features; The input of the remaining rotation-invariant layer is connected to the output of the nearest neighbor upsampling layer to extract the dense point cloud features.

6. The anti-rotation color point cloud registration network according to claim 5, characterized in that, The rotationally invariant layer is configured as follows: Point cloud features are learned from rotationally isotropic spatial statistics using the VN-MLP module; The size of the vector neurons is calculated by using an L2-Norm layer, and the learned point cloud features are converted into a rotation-invariant form to obtain the geometric color rotation-invariant features. The geometric color rotation equivariant feature vector is mapped to K dimensions using the MLP module to obtain the relevant vector; The correlation scores of the geometric color rotation isomorphic features are output using the Softmax operation.

Images