A multi-scale perception-based multi-view point cloud registration method

Abstract

The present application relates to a kind of multi-scale perception-based multi-view point cloud registration method.The method uses the way that multi-scale perception module is combined with attention mechanism, adopts pose graph optimization method, proposes a kind of multi-view point cloud registration method, mainly includes: selecting point cloud set in point cloud pair, down-sampling is carried out to point cloud, registration calculation is carried out two by two, in the calculation process, the multiscale local geometric structure information of each point is introduced by attention mechanism, point cloud overall geometric information and the structure information of corresponding point cloud, and point cloud feature set is obtained, the relative registration matrix and weight estimation coefficient of corresponding point cloud can be obtained by point cloud feature set, pose graph is established and optimized on this basis, finally the absolute registration matrix of each point cloud is obtained and each point cloud is converted and combined to obtain complete point cloud.

Description

Technical Field

[0001] This invention belongs to the field of machine vision, specifically a multi-view point cloud registration method based on multi-scale perception. Background Technology

[0002] Point cloud registration is the process of aligning two or more point cloud datasets to a common reference coordinate system. In the real world, data is often collected in a dispersed manner. For example, in the scanning and 3D reconstruction of callus tissue in plant cells, point cloud data of callus tissue acquired from different angles need to be registered together. This merges data from different perspectives, compensates for the deficiencies of a single viewpoint, and obtains a complete 3D representation for overall analysis. Furthermore, registration can correct data deviations caused by equipment errors or environmental changes, improving the accuracy and reliability of the 3D model.

[0003] For multi-view point cloud images acquired in scenarios such as plant cell callus tissue, point cloud registration is divided into pairwise view point cloud registration and multi-view point cloud registration, depending on the number of points to be registered. Pairwise view point cloud registration involves registering two sets of point clouds with a certain degree of overlap to a single coordinate system. Existing mature paradigms obtain corresponding points in the two sets of point clouds through feature mapping and matching, and rely on a large number of point pairs to calculate the registration matrix using optimization-based methods. Obtaining reliable point features is the foundation of point cloud registration. Manually designed point feature extraction methods rely too heavily on point cloud features and have poor generalization ability. With the development of deep learning, many feature extraction networks have been proposed, but their robustness to rotation is poor, which greatly affects the accuracy of matching point selection.

[0004] Meanwhile, multi-view point cloud registration is more difficult than two-view point cloud registration due to the complexity of the relationships between point clouds and the existence of accumulated errors. As the number of viewpoints increases, the overlap between adjacent point clouds may decrease, making it difficult to find the correct corresponding point pairs during the matching process, thus increasing the difficulty of registration. Multi-view point cloud registration involves multiple pairwise registrations of point clouds, and each registration may introduce errors. These errors accumulate as the registration process progresses, leading to significant deviations in the final point cloud model. Summary of the Invention

[0005] This invention provides a multi-view point cloud registration method based on multi-scale perception to overcome the problems of long calculation time and high cumulative registration error in multi-view point cloud registration.

[0006] The technical solution adopted by this invention to achieve the above objectives is: a multi-view point cloud registration method based on multi-scale perception, comprising the following steps:

[0007] Step 1: Select point cloud pairs from the point cloud set and downsample the point clouds;

[0008] Step 2: Perform pairwise registration on the point clouds in the point cloud set to obtain the relative registration matrix and weight estimation coefficients between each pair of point clouds;

[0009] Step 3: Register the multi-view point cloud formed by multiple point cloud pairs. Calculate the absolute registration matrix of each point cloud using the relative registration matrix between each pair of point clouds and the pose graph weight estimation coefficients.

[0010] Step 4: Register the point cloud to be registered based on the absolute registration matrix obtained in Step 3 to generate a complete object point cloud model.

[0011] The point cloud downsampling method is the farthest point sampling downsampling method.

[0012] The process of performing pairwise registration of point clouds in the set to obtain the relative registration matrix and weight estimation coefficients between each pair of point clouds includes the following steps:

[0013] Step 1: Select a pair of point clouds from the point cloud set. The point cloud pair is the two point clouds that need to be registered. For point cloud P and point cloud Q in the two point clouds to be registered, execute Step 2 and Step 3 respectively.

[0014] Step 2: After calculation by the encoder module, the local attention features of each point in the point cloud are obtained. The local attention features of each point are then updated by the self-attention module to obtain self-attention features that reference the features of other points. The self-attention features of all points are combined to form the point cloud feature set P. se Q se ;

[0015] Step 3: Combine the feature sets P corresponding to the two point clouds, which contain the self-attention features of each point. se Q se The cross-attention module calculates and obtains the updated cross-attention feature set of the two point clouds. At this time, the cross-attention feature set integrates the information of the point cloud itself and the information between the point clouds.

[0016] Step 4: Then, the corresponding points between the two point clouds are determined by feature matching, and the registration matrix is ​​calculated by the ransac method to obtain the relative registration matrix and weight estimation coefficients.

[0017] Step 2 is as follows:

[0018] The multi-scale perception module sets the point to be calculated together with its k nearest neighbors, calculates its local structure to obtain the rotation invariance of the features; multiple k values ​​correspond to multiple point cloud feature calculation networks, and the feature vectors calculated by multiple point cloud feature calculation networks are combined to form a feature matrix to obtain the point P in the point cloud. i Or Q iMulti-scale local geometric structure information

[0019] Local attention module, fusing point cloud midpoint P i Or Q i The features of the point itself and the features of its neighborhood points are introduced, and the point P is mapped by the multi-sensor module. i Or Q i Multi-scale local geometric structure information is used to calculate point P. i Or Q i Local attention features m p or m q Then, the updated point P is obtained through a feedforward network. i Or click Q i Corresponding feature vector or Connect coordinates N with... or The combined update set P or Q is iterated multiple times through the multi-scale perception module and the local attention module process to finally obtain the encoded sets P′ and Q′.

[0020] The self-attention module calculates the self-attention feature P' and Q' obtained from point clouds P and Q after local attention module computation. A point within these points is incorporated into the self-attention calculation through its geometric relationship with all other points in the current point cloud. After passing through a feedforward network, a self-attention feature P integrating overall geometric information is obtained. se Or Q se .

[0021] In the local attention module, the point P in the fused point cloud i Or Q i The features of the point itself and the features of its neighborhood points are introduced, and the point P is mapped by the multi-sensor module. i Or Q i Multi-scale local geometric structure information is used to calculate point P. i Or Q i Local attention features m p or m q The calculation is as follows:

[0022] For point cloud P or point cloud Q, the feature matrix G containing geometric information s ,Right now or With feature information matrix X s respectively with parameter matrix W G and W Q Multiplying them together yields the query matrix A containing the fused information. G and with the characteristic matrix C s With key value matrix W K The product of AC Adding them together yields matrix A. Matrix A and C s AND-value matrix W V The product of V and m is used as the information m for point P. i Local attention features m p Or click Q i Local attention features m q This is input to the feedforward network in the encoder module.

[0023] The self-attention module calculates as follows:

[0024] The geometric relationship between a point in a point cloud and all other points in the current point cloud is encoded using inter-point distance encoding, resulting in R and the feature matrix C. s As the self-attention input, R and the parameter matrix Multiplication yields matrix E se C s With query matrix Multiplying them together gives Qu se C s With key value matrix Multiply to get K se Following the basic attention mechanism, Qu se With K se Multiplication yields a matrix At the same time, Qu se With E se Multiplication yields a matrix containing information about the point cloud set. Will and Adding them together, we get matrix A. se and V se P is obtained after multiplication through a feedforward network. se Or Q se ;where V se By C s AND-value matrix Multiply them to get the result.

[0025] The registration of multi-view point clouds includes pose graph establishment and pose graph optimization; wherein, the pose graph is an undirected graph structure, the vertices of which represent each point cloud to be registered, and the edges are the weight coefficients of the relative registration matrix. The pose graph optimization is to calculate the optimal weight reference coefficients and the absolute registration matrix corresponding to each point cloud through optimization methods.

[0026] A multi-view point cloud registration system based on multi-scale perception includes:

[0027] The encoder module is used to obtain the rotation-invariant features of each point in the point cloud at multiple scales. Then, the features are used as input and embedded into the local attention module to calculate the feature vectors of each point in the point cloud, and the encoded sets P′ and Q′ are obtained.

[0028] The self-attention module calculates the self-attention feature P' and Q' obtained from point clouds P and Q after local attention module computation. A point within these points is incorporated into the self-attention calculation through its geometric relationship with all other points in the current point cloud. After passing through a feedforward network, a self-attention feature P integrating overall geometric information is obtained. se Or Q se ;

[0029] The cross-attention module is used to integrate self-attention features P se Q se The relative registration matrix is ​​obtained through cross-attention calculation. and weight estimation coefficient w ij ;

[0030] The multi-view point cloud registration module is used to calculate the absolute registration matrix of each point cloud by using the relative registration matrix and weight estimation coefficients between pairs of point clouds, and then register the point clouds to be registered to generate a complete object point cloud model.

[0031] The encoder module includes:

[0032] The multi-scale perception module consists of multiple point cloud feature computation networks with identical structures. Its structure comprises a first multi-layer perception layer, a rotation-invariant mapping module, a second multi-layer perception layer, and a max-pooling layer. The rotation-invariant mapping module groups the point to be computed with its k nearest neighbors and calculates their local structure to obtain the rotation invariance of the features. Multiple k values ​​correspond to multiple point cloud feature computation networks. The feature vectors calculated by these networks are combined to form a feature matrix to obtain the point P in the point cloud. i Or Q i Multi-scale local geometric structure information

[0033] Local attention module, used to fuse point P in point cloud i Or Q i The features of the point itself and the features of its neighborhood points are introduced, and the point P is mapped by the multi-sensor module. i Or Q i Multi-scale local geometric structure information is used to calculate point P. i Or Q i Local attention features m p or m q Then, the updated point P is obtained through a feedforward network. i Or click Q i Corresponding feature vector or Connect coordinates N with... or The combined update set P or Q is iterated multiple times through the multi-scale perception module and the local attention module process to finally obtain the encoded sets P′ and Q′.

[0034] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the multi-view point cloud registration method based on multi-scale perception.

[0035] The present invention has the following beneficial effects and advantages:

[0036] 1. This invention utilizes a multi-scale perception module to extract features from the multi-scale set structure of points, thereby improving the accuracy of point pairing.

[0037] 2. This invention extracts features of points based on rotation invariance, thereby reducing the sensitivity of features to point cloud rotation.

[0038] 3. This invention employs an attention mechanism to fuse the geometric structure information of points, the structural information of point clouds, and the structural information between point cloud pairs, thereby improving the success rate of point cloud matching. Attached Figure Description

[0039] Figure 1 The flowchart of the multi-view point cloud registration method based on multi-scale perception provided by this invention is shown below; wherein, 1. Point cloud sampling, 2. Pairwise point cloud registration, 2.1 Encoder module, 2.1.1 Multi-scale perception module, 2.1.2 Local attention module, 2.2 Self-attention module, 2.3 Cross-attention module, 2.4 Parameter calculation, 3. Multi-view point cloud registration, 3.1 Pose graph establishment, 3.2 Pose graph optimization, 4. Point cloud transformation;

[0040] Figure 2 Detailed structural diagram of the multi-view point cloud registration method based on multi-scale perception provided by this invention;

[0041] Figure 3 An encoder module diagram provided in this invention;

[0042] Figure 4 A diagram of a multi-scale sensing module provided in this invention;

[0043] Figure 5 A diagram illustrating the local attention calculation process provided in this invention.

[0044] Figure 6 A diagram of a self-attention module provided in this invention;

[0045] Figure 7 A diagram illustrating the self-attention calculation process provided in this invention.

[0046] Figure 8The cross-attention module diagram provided in this invention;

[0047] Figure 9 The cross-attention calculation process provided by this invention is illustrated in the diagram. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Images of plant cell callus tissue were acquired using multiple cameras to obtain point clouds from multiple perspectives, forming a point cloud set. The final callus tissue point cloud model was obtained through a multi-view point cloud registration method.

[0050] like Figure 1 As shown, a multi-view point cloud registration method based on multi-scale perception includes:

[0051] Step 1: Select point cloud pairs from the point cloud set and downsample the point clouds to reduce the computational scale;

[0052] Step 2: Perform pairwise registration on the point clouds in the set to obtain the relative registration matrix and weight estimation coefficients between each pair of point clouds;

[0053] Step 3: Register the multi-view point clouds. Calculate the absolute registration matrix of each point cloud using the relative registration matrix between each pair of point clouds and the weight estimation coefficients.

[0054] Step 4: Register the point cloud to be registered based on the absolute registration matrix obtained in Step 3 to generate a complete object point cloud model.

[0055] In this embodiment, the method for sampling the point cloud set to be registered in step one is the farthest point sampling method. Its core idea is to select a starting point from the dataset, and then iteratively select the point farthest from the currently selected point set until the required number of sampling points is reached. This method reduces the amount of data computation while ensuring that the sampling points are relatively uniformly distributed in space, thereby preserving the geometric structure of the dataset to the greatest extent.

[0056] In this embodiment, as Figure 2 As shown, the point cloud pairwise registration method described in step two includes the following steps:

[0057] Step 1: Select point cloud pair (P, Q) from the point cloud set. Point cloud pair (P, Q) are two point clouds that need to be registered.

[0058] Step 2: After calculation by the programmer module, the local attention features of each point in the point cloud are obtained and combined into a point cloud feature vector set P′. The local attention features of each point are then updated by the self-attention module to obtain self-attention features that reference the features of other points. The self-attention features of all points are then combined into a point cloud feature vector set P′. se ;

[0059] Step 3: Combine the feature sets (P) corresponding to the two point clouds, which contain the self-attention features of each point. se Q se The updated cross-attention feature set of the two point clouds is obtained by calculating through the cross-attention module. At this point, the cross-attention feature set integrates information from the point cloud itself and from the points between the points;

[0060] Step four: Then, corresponding points between the two point clouds are determined through feature matching, and the registration matrix {R} is obtained using the RANSAC method. ij The calculation and pose graph of} ij Weight estimation of}.

[0061] like Figure 2 As shown, the programmer module in the pairwise registration method consists of a multi-scale perception module and a local attention module. The multi-scale perception module obtains the rotation-invariant features of each point in the point cloud at multiple scales, and then uses these features as input to embed them into a traditional attention module to calculate the feature vectors of each point in the point cloud. The local attention module consists of attention calculation and a feedforward network.

[0062] Figure 3 The specific structure of the programmer module is as follows: the set P has the structure P:(N,C), where N represents the set of point cloud coordinates, and C is the corresponding feature set. The feature set is defined by points P in set P. i The combination of multiple neighborhood point sets yields the support set P. i s (Iterative update), and support point features (Initial value is P) i (Features of the nearest point) and P i The feature set of the k nearest neighbors Among them, the set P is supported. s Point P can be obtained through a multi-scale sensing module. i Local set structure information And through a linear mapping to G s , and set X is obtained through a shared linear mapping. s C sThe above information is processed through local attention calculation and a feedforward network to obtain the updated P. i The point corresponds to the feature vector x i , and the coordinates N and {x i The combined update of set P yields the encoded set P′. The same logic applies to set Q.

[0063] like Figure 4 As shown, in this embodiment, the programmer module in the pairwise registration method, the multi-scale perception module, is composed of multiple point cloud feature calculation networks with identical structures. Its structure consists of a multi-layer perception layer 1, a rotation-invariant mapping module, a multi-layer perception layer 2, and a max-pooling layer. The rotation-invariant mapping module can calculate the point P to be calculated. i Form a set with k nearest neighbors, calculate its local structure to obtain the rotationally invariant feature g. k In this process, multiple k values ​​correspond to multiple point cloud feature calculation networks. The feature vectors calculated by these networks are combined to form a feature matrix, which is used to obtain the feature vectors of point P. i Multi-scale local geometric structure information

[0064] Preferably, the rotation-invariant mapping module in the multi-scale sensing module can use spherical mapping, and its specific calculation process is as follows: Given the point P to be encoded i The set of points P that are the nearest neighbors of k i,j}:

[0065]

[0066] Where α, β, b are learnable parameters, ⊙ is the Hadamard product, and ∈ is a parameter that ensures the stability of division. Point P i And the average value of the features of the neighborhood points.

[0067] like Figure 5 As shown, in this embodiment, the programmer module in the pairwise registration method and the local attention module fuse the point cloud center point P. i The features of the point itself and the features of its neighborhood points are introduced, and the point P is mapped by the multi-sensor module. i Multi-scale local geometric structure information is calculated, and point P is obtained after multiple iterations. i The local attention feature. The calculation process of local attention is as follows: feature matrix G with geometric information. s With feature information matrix X s In the parameter matrix W respectively G and W Q Multiplying them together yields the query matrix A containing the fused information. G and with C s With key value matrix W KThe product of A C Adding them together yields matrix A. Matrix A and C s AND-value matrix W V The product of V and m is obtained by multiplying V and m, and then the product of m and m is obtained by the feedforward network. i Corresponding feature vector

[0068] like Figure 6 As shown, in step two of this embodiment, the pairwise registration method involves the self-attention module registering the point cloud P. i The geometric relationship between a point and all other points is encoded using inter-point distances, resulting in R. This R is then incorporated into the calculation of self-attention. After passing through a feedforward network, a point cloud feature P integrating the overall geometric information is obtained. se The attention calculation process is as follows: Figure 7 As shown: R and the characteristic matrix C s As a self-attention input, C and Multiplying them together gives E se C s With query matrix Multiplying them together gives Qu se C s With key value matrix Multiply to get K se Following the basic attention mechanism, Qu se With K se Multiplication yields a matrix At the same time, Qu se With E se Multiplication yields a matrix containing information about the point cloud set. Will and Adding them together, we get matrix A. se and V se P is obtained after multiplication through a feedforward network. se V se By C s AND-value matrix Multiply them to get the result.

[0069] The distance between points is encoded using Euclidean distance. and The distance is Distance Obtained through coherent position encoding δ d Similarly, to adjust the hyperparameters for distance sensitivity, the angle encoding between points in the k-neighborhood and the point x to be calculated is also computed. Distance Obtained through coherent position encoding δ aThe distance between optimal points is encoded as a hyperparameter used to adjust the angle sensitivity. r i,j W represents the elements that constitute matrix R. D W A This is the weight matrix.

[0070] like Figure 7 As shown, the input to the cross-attention module is set P. se With set Q se After calculation, we obtained Figure 8 , Figure 9 Demonstrates cross-attention computation The process is no different from the traditional attention mechanism.

[0071] Preferably, the loss function for the entire network is a circular loss.

[0072] like Figure 2 As shown, the multi-view point cloud registration method in step three of this embodiment includes two steps: pose graph establishment and pose graph optimization. The pose graph is an undirected graph structure, where vertices represent the point clouds to be registered, and edges are the weight coefficients of the relative registration matrix. Pose graph optimization involves calculating the optimal weight reference coefficients {w} using an optimization method. ij} and the absolute registration matrix {R} corresponding to each point cloud i}

[0073] Preferably, pose graph optimization is performed using the following method:

[0074] The goal of optimization Where {R i} can be calculated by matrix

[0075] The eigenvalues ​​are obtained. SO(3) denotes the three-dimensional special orthogonal group, ε denotes the set of point cloud pairs, and w Nj I3 represents the weight of the relative registration matrix of the point cloud pair (N, j), where I3 represents the 3rd order identity matrix and N represents the point cloud N.

[0076] When updating {R i After that, {w} needs to be updated. ij To obtain the parameters for the next iteration, first calculate... Then, {w} is calculated using gradient descent. ij}, specifically R represents the registration matrix residual. ij represents the relative registration matrix between point clouds, Δ represents the angle difference between two registration matrices, and n represents the number of iterations.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A multi-view point cloud registration method based on multi-scale perception, characterized in that, Includes the following steps: Step 1: Select point cloud pairs from the point cloud set and downsample the point clouds; Step 2: Perform pairwise registration on the point clouds in the point cloud set to obtain the relative registration matrix and weight estimation coefficients between each pair of point clouds; Step 3: Register the multi-view point cloud formed by multiple point cloud pairs. Calculate the absolute registration matrix of each point cloud using the relative registration matrix between each pair of point clouds and the pose graph weight estimation coefficients. Step 4: Register the point cloud to be registered based on the absolute registration matrix obtained in Step 3 to generate a complete object point cloud model.

2. The multi-view point cloud registration method based on multi-scale perception according to claim 1, characterized in that, The point cloud downsampling method is the farthest point sampling downsampling method.

3. The multi-view point cloud registration method based on multi-scale perception according to claim 1, characterized in that, The process of performing pairwise registration of point clouds in the set to obtain the relative registration matrix and weight estimation coefficients between each pair of point clouds includes the following steps: Step 1: Select a pair of point clouds from the point cloud set. The point cloud pair is the two point clouds that need to be registered. For point cloud P and point cloud Q in the two point clouds to be registered, execute Step 2 and Step 3 respectively. Step 2: After calculation by the encoder module, the local attention features of each point in the point cloud are obtained. The local attention features of each point are then updated by the self-attention module to obtain self-attention features that reference the features of other points. The self-attention features of all points are combined to form the point cloud feature set P. se Q se ; Step 3: Combine the feature sets P corresponding to the two point clouds, which contain the self-attention features of each point. se Q se The cross-attention module calculates and obtains the updated cross-attention feature set of the two point clouds. At this time, the cross-attention feature set integrates the information of the point cloud itself and the information between the point clouds. Step 4: Then, the corresponding points between the two point clouds are determined by feature matching, and the registration matrix is ​​calculated by the ransac method to obtain the relative registration matrix and weight estimation coefficients.

4. The multi-view point cloud registration method based on multi-scale perception according to claim 3, characterized in that, Step 2 is as follows: The multi-scale perception module sets the point to be calculated together with its k nearest neighbors, calculates its local structure to obtain the rotation invariance of the features; multiple k values ​​correspond to multiple point cloud feature calculation networks, and the feature vectors calculated by multiple point cloud feature calculation networks are combined to form a feature matrix to obtain the point P in the point cloud. i Or Q i Multi-scale local geometric structure information Local attention module, fusing point cloud midpoint P i Or Q i The features of the point itself and the features of its neighborhood points are introduced, and the point P is mapped by the multi-sensor module. i Or Q i Multi-scale local geometric structure information is used to calculate point P. i Or Q i Local attention features m p or m q Then, the updated point P is obtained through a feedforward network. i Or click Q i Corresponding feature vector or Connect coordinates N with... or The combined update set P or Q is iterated multiple times through the multi-scale perception module and the local attention module to finally obtain the encoded set P. ′ and Q ′ ; The self-attention module calculates the set P obtained from point clouds P and Q using the local attention module. ′ and Q ′ A point is used to calculate self-attention by incorporating its geometric relationship with all other points in the current point cloud. After passing through a feedforward network, a self-attention feature P that integrates the overall geometric information is obtained. se Or Q se .

5. The multi-view point cloud registration method based on multi-scale perception according to claim 4, characterized in that, In the local attention module, the point P in the fused point cloud i Or Q i The features of the point itself and the features of its neighborhood points are introduced, and the point P is mapped by the multi-sensor module. i Or Q i Multi-scale local geometric structure information is used to calculate point P. i Or Q i Local attention features m p or m q The calculation is as follows: For point cloud P or point cloud Q, the feature matrix G containing geometric information s ,Right now or With feature information matrix X s respectively with parameter matrix W G and W Q Multiplying them together yields the query matrix A of the fused information. G and with the characteristic matrix C s With key value matrix W K The product of A C Adding them together yields matrix A. Matrix A and C s AND-value matrix W V The product of V and m is used as the information m for point P. i Local attention features m p Or click Q i Local attention features m q This is input to the feedforward network in the encoder module.

6. The multi-view point cloud registration method based on multi-scale perception according to claim 4, characterized in that, The self-attention module calculates as follows: The geometric relationship between a point in a point cloud and all other points in the current point cloud is encoded using inter-point distance encoding, resulting in R and the feature matrix C. s As the self-attention input, R and the parameter matrix Multiplication yields matrix E se C s With query matrix Multiplying them together gives Qu se C s With key matrix Multiply to get K se Following the basic attention mechanism, Qu se With K se Multiplication yields a matrix At the same time, Qu se With E se Multiplication yields a matrix containing information about the point cloud set. Will and Adding them together, we get matrix A. se and V se P is obtained after multiplication through a feedforward network. se Or Q se ;where V se By C s AND-value matrix Multiply them to get the result.

7. The multi-view point cloud registration method based on multi-scale perception according to claim 1, characterized in that, The registration of multi-view point clouds includes pose graph establishment and pose graph optimization; wherein, the pose graph is an undirected graph structure, the vertices of which represent each point cloud to be registered, and the edges are the weight coefficients of the relative registration matrix. The pose graph optimization is to calculate the optimal weight reference coefficients and the absolute registration matrix corresponding to each point cloud through optimization methods.

8. A multi-view point cloud registration system based on multi-scale perception, characterized in that, include: The encoder module obtains rotation-invariant features of each point in the point cloud across multiple scales. These features are then used as input and embedded into the local attention module to calculate the feature vectors of each point in the point cloud, resulting in the encoded set P. ′ Q ′ ; The self-attention module, for the set P obtained after computation by the local attention module, ′ and Q ′ A point is used to calculate self-attention by incorporating its geometric relationship with all other points in the current point cloud. After passing through a feedforward network, a self-attention feature P that integrates the overall geometric information is obtained. se Or Q se ; The cross-attention module is used to integrate self-attention features P se Q se The relative registration matrix is ​​obtained through cross-attention calculation. and weight estimation coefficient w ij ; The multi-view point cloud registration module is used to calculate the absolute registration matrix of each point cloud by using the relative registration matrix and weight estimation coefficients between pairs of point clouds, and then register the point clouds to be registered to generate a complete object point cloud model.

9. A multi-view point cloud registration system based on multi-scale perception according to claim 8, characterized in that, The encoder module includes: The multi-scale perception module consists of multiple point cloud feature computation networks with identical structures. Its structure comprises a first multi-layer perception layer, a rotation-invariant mapping module, a second multi-layer perception layer, and a max-pooling layer. The rotation-invariant mapping module groups the point to be computed with its k nearest neighbors and calculates their local structure to obtain the rotation invariance of the features. Multiple k values ​​correspond to multiple point cloud feature computation networks. The feature vectors calculated by these networks are combined to form a feature matrix to obtain the point P in the point cloud. i Or Q i Multi-scale local geometric structure information Local attention module, used to fuse point P in point cloud i Or Q i The features of the point itself and the features of its neighborhood points are introduced, and the point P is mapped by the multi-sensor module. i Or Q i Multi-scale local geometric structure information is used to calculate point P. i Or Q i Local attention features m p or m q Then, the updated point P is obtained through a feedforward network. i Or click Q i Corresponding feature vector or Connect coordinates N with... or The combined update set P or Q is iterated multiple times through the multi-scale perception module and the local attention module to finally obtain the encoded set P. ′ and Q ′ .

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements a multi-view point cloud registration method based on multi-scale perception as described in any one of claims 1-7.

Images