Self-supervised registration method for multi-source remote sensing point clouds

Abstract

The invention belongs to the field of remote sensing point cloud registration, and particularly relates to a self-supervised registration method for multi-source remote sensing point clouds. The problem that the registration error is large when supervision information is lacked due to the fact that the prior art depends on the supervision information is solved. The self-supervised registration method for the multi-source remote sensing point cloud comprises the following steps: acquiring the multi-source remote sensing point cloud; constructing a mask self-encoding point cloud reconstruction network, and training the constructed mask self-encoding point cloud reconstruction network according to the multi-source remote sensing point cloud to obtain a trained mask self-encoding point cloud reconstruction network; obtaining a to-be-registered multi-source remote sensing point cloud and a template point cloud; reconstructing a network according to the to-be-registered multi-source remote sensing point cloud, the template point cloud and the trained self-mask point cloud to obtain a transformation relation of the to-be-registered multi-source remote sensing point cloud; transforming the to-be-registered multi-source remote sensing point clouds according to the transformation relation of the to-be-registered multi-source remote sensing point clouds to obtain registered multi-source remote sensing point clouds, and solving the problem of large registration error of the existing multi-source point cloud registration method.

Description

Technical Field

[0001] The invention belongs to the field of remote sensing point cloud registration, and in particular relates to a self-supervised registration method for multi-source remote sensing point clouds. Background Art

[0002] With the continuous development of three-dimensional detection technology, there are now a variety of means of acquiring remote sensing point clouds. The mainstream technologies include actively acquiring point clouds using lidar or generating point clouds using photogrammetry methods. Different ways of acquiring point clouds have their own advantages and limitations. In order to integrate and utilize the respective advantages of multi-source point clouds, it is first necessary to perform necessary registration processing on the multi-source point clouds. Most of the existing point cloud registration methods are designed for homologous point clouds acquired by the same sensor, without considering the differences in point cloud characteristics caused by different principles of multi-source point cloud acquisition. In addition, the deep learning-based method requires supervision information of the true value of the registration for pre-training, and the true value between the multi-source remote sensing point clouds actually collected is difficult to obtain. Therefore, the existing technology relies on supervision information, resulting in large registration errors when supervision information is lacking. Summary of the invention

[0003] The purpose of the present invention is to solve the problem that the existing technology relies on supervision information, resulting in large registration errors when supervision information is lacking. A self-supervised registration method for multi-source remote sensing point clouds is invented, which includes:

[0004] Step 1: Obtain multi-source remote sensing point cloud;

[0005] Step 2: Construct a masked self-encoding point cloud reconstruction network, and train the constructed masked self-encoding point cloud reconstruction network according to the multi-source remote sensing point cloud to obtain a trained masked self-encoding point cloud reconstruction network;

[0006] Step 3: Obtain the multi-source remote sensing point cloud and template point cloud to be registered; reconstruct the network according to the multi-source remote sensing point cloud and template point cloud to be registered and the masked self-encoder point cloud trained in step 2 to obtain the transformation relationship of the multi-source remote sensing point cloud to be registered;

[0007] Step 4: transform the multi-source remote sensing point cloud to be registered according to the transformation relationship of the multi-source remote sensing point cloud to obtain the registered multi-source remote sensing point cloud.

[0008] The beneficial effects of the present invention are:

[0009] 1. Aiming at the characteristic differences between 3D reconstructed point clouds and lidar point clouds, the present invention designs a multi-source point cloud self-supervised registration framework based on masked autoencoder features. Robust global feature extraction is performed on the distribution differences of multi-source point clouds through masked autoencoders and self-supervised learning methods, and multi-source point clouds are registered using feature alignment, thereby improving the registration accuracy of multi-source point clouds.

[0010] 2. The present invention can register multi-source remote sensing point clouds with large characteristic differences without known supervision information, thereby improving the registration accuracy of multi-source point clouds. In order to verify the performance of the algorithm proposed in the present invention, experiments were conducted on the lidar point clouds collected by drones and the three-dimensional reconstructed point clouds based on images. The experimental results verified the effectiveness of the method proposed in the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of the process of the multi-source point cloud self-supervised registration method of the present invention;

[0012] Figure 2 It is a schematic diagram of a registration experiment of a multi-source point cloud self-supervised registration method of the present invention. DETAILED DESCRIPTION

[0013] Specific implementation method 1: Combination Figure 1 The present invention describes a self-supervised registration method for multi-source remote sensing point clouds, comprising:

[0014] Step 1: Obtain multi-source remote sensing point cloud;

[0015] Step 2: Construct a masked self-encoding point cloud reconstruction network, and train the constructed masked self-encoding point cloud reconstruction network according to the multi-source remote sensing point cloud to obtain a trained masked self-encoding point cloud reconstruction network;

[0016] Step 3: Obtain the multi-source remote sensing point cloud and template point cloud to be registered; reconstruct the network according to the multi-source remote sensing point cloud and template point cloud to be registered and the masked self-encoder point cloud trained in step 2 to obtain the transformation relationship of the multi-source remote sensing point cloud to be registered;

[0017] Step 4: transform the multi-source remote sensing point cloud to be registered according to the transformation relationship of the multi-source remote sensing point cloud to obtain the registered multi-source remote sensing point cloud.

[0018] Aiming at the characteristic differences between 3D reconstructed point clouds and lidar point clouds, the present invention designs a multi-source point cloud self-supervised registration framework based on masked autoencoder features. Robust global feature extraction is performed on the distribution differences of multi-source point clouds through masked autoencoders and self-supervised learning, and multi-source point clouds are registered using feature alignment, thereby improving the registration accuracy of multi-source point clouds.

[0019] The present invention can register multi-source remote sensing point clouds with large characteristic differences without known supervision information, thereby improving the registration accuracy of multi-source point clouds. In order to verify the performance of the algorithm proposed in the present invention, experiments were conducted on the lidar point clouds collected by drones and the three-dimensional reconstruction point clouds based on images. The experimental results verified the effectiveness of the method proposed in the present invention.

[0020] Specific implementation method 2: The difference between this implementation method and specific implementation method 1 is that:

[0021] The multi-source remote sensing point cloud in step 1 includes: a laser radar point cloud and a three-dimensional reconstruction point cloud;

[0022] The laser radar used in the present invention is VELODYNE VLP16 LITE, and the camera for image acquisition is Micasenserededge-MX DUAL. The acquisition location is located in the Science Park of Harbin Institute of Technology;

[0023] The laser radar and camera used in the present invention can be selected according to the actual use.

[0024] The other steps and parameters are the same as those in the first specific implementation.

[0025] Specific implementation method three: This implementation method is different from the specific implementation method one in that:

[0026] The masked self-encoding point cloud reconstruction network in step 2 includes: a random mask module, an encoder network and a decoder network;

[0027] The masked self-encoding point cloud reconstruction network is trained according to the multi-source remote sensing point cloud to obtain a trained masked self-encoding point cloud reconstruction network; the specific process is:

[0028] Step 2.1: Take the multi-source remote sensing point cloud as the input point cloud P, input it into the random mask module, and obtain the masked point cloud P';

[0029] Step 2.2: Input the masked point cloud P' into the encoder network to obtain the global feature F';

[0030] Step 2.3: Input the global feature F' into the decoder network to obtain the reconstructed point cloud PC;

[0031] Step 2.4: Calculate the loss function of the reconstructed point cloud PC and the input point cloud P, and train the masked self-encoding point cloud reconstruction network according to the loss function. When the loss function is minimized, the training is completed and the trained masked self-encoding point cloud reconstruction network is obtained.

[0032] This step uses a self-supervised training method instead of relying on known transformation relationships for pre-training to extract local features of the point cloud.

[0033] The other steps and parameters are the same as those in the first to second embodiments.

[0034] Specific implementation method 4: This implementation method is different from specific implementation methods 1 to 4 in that:

[0035] In step 2.1, the multi-source remote sensing point cloud is used as the input point cloud P, and the random mask module is input to obtain the masked point cloud P'; The specific process is:

[0036] Step 2.1.1: Set the number of sampling points to N, where N is a positive integer, and randomly select a point in the input point cloud P as the initial sampling point P1;

[0037] Step 2.1.2: Calculate the distance between the unselected points in the input point cloud P and the initial sampling point P1, and select the point with the largest distance as the sampling point P2;

[0038] Step 2.1.3: Calculate the minimum distance from the unselected point in the input point cloud P to the existing sampling point set, and select the point with the largest minimum distance as the sampling point;

[0039] Step 2.1.4: Repeat step 2.1.3. When N sampling points are obtained, stop the execution and go to step 2.1.5. The nth sampling point obtained is denoted as Pn.

[0040] Step 2.1.5: N sampling points are combined into the farthest point sampling point cloud PS; the mask point cloud PK is determined according to the sampling point cloud PS;

[0041] Step 2.1.6: Mask the input point cloud P according to the mask point cloud PK obtained in step 2.1.5 to obtain the masked point cloud P'. The specific process is:

[0042] The above sampling points in the input point cloud P are masked and marked as invisible points (the rest of the points in P are marked as visible points, namely P').

[0043] The other steps and parameters are the same as those in the first to third embodiments.

[0044] Specific implementation method 5: The difference between this implementation method and specific implementation methods 1 to 4 is that the specific process of calculating the minimum distance from the unselected points in the input point cloud P to the existing sampling point set in step 2.1.3 is:

[0045] Calculate the distance from the unselected point to the nearest sampling point in the existing sampling point set as the minimum distance from the unselected point in the input point cloud P to the existing sampling point set;

[0046] Assume that the existing sampling point set includes: an initial sampling point P1 and a sampling point P2; then the distance from the oth point in the unselected points to the initial sampling point P1 is o1, and the distance from the oth point to the sampling point P2 is o2. If o1 is greater than o2, then the minimum distance from the oth point to the sampling point set is o2;

[0047] ,When calculating the minimum distance from the unselected point in the input point cloud P to each point in the existing sampling point set, the minimum value of the previously calculated sampling distance is retained as the distance from the point to the sampled point to avoid repeated calculation;

[0048] The specific process of determining the mask point cloud PK according to the sample point cloud PS in step 2.1.5 is as follows:

[0049] In the input point cloud P, find the nearest K points of each point in the farthest point sampling point cloud PS in turn as the mask point cloud PK;

[0050] For example, if PS has N points in total, find K nearest points for each point in PS in the input point cloud P, and find NK points in total. However, for different points, it is allowed to repeat when finding K nearest neighbor points, so the total number of points obtained in the end is less than or equal to NK. The other steps and parameters are the same as those in the first to fourth embodiments.

[0051] Specific implementation method 6: This implementation method is different from specific implementation methods 1 to 5 in that:

[0052] The encoder network structure in step 2.2 includes: an MLP layer and a maximum pooling layer in sequence;

[0053] The MLP layer includes an input layer, a first hidden layer, a second hidden layer, a third hidden layer, a fourth hidden layer and an output layer in sequence; the first hidden layer, the second hidden layer, the third hidden layer and the fourth hidden layer are all convolutional layers;

[0054] The convolution kernel size of the first hidden layer is 1×3; the convolution kernel sizes of the second hidden layer, the third hidden layer, and the fourth hidden layer are all 1×1

[0055] In step 2.2, the masked point cloud P' is input into the encoder network to obtain the global feature F'; the specific process is:

[0056] The masked point cloud P' is used as the input of the encoder network, with a dimension of N'×3. The encoder network consists of four layers of MLP and one maximum pooling layer. The number of convolution kernel channels of MLP is 64, 64, 128, and 1024 respectively. The masked point cloud P' passes through four layers of MLP in sequence and outputs the intermediate features of N'×1024 dimensions. The maximum pooling layer takes the maximum value of the intermediate features extracted by the fourth MLP layer, and finally obtains 1024 global features F'.

[0057] ; Other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0058] Specific implementation method 7: This implementation method is different from specific implementation methods 1 to 6 in that:

[0059] The decoder network structure in step 2.3 includes a first fully connected layer, a second fully connected layer, a third fully connected layer and a fourth fully connected layer in sequence;

[0060] The activation functions of the first fully connected layer, the second fully connected layer, the third fully connected layer and the fourth fully connected layer are all LeakyReLU;

[0061] In step 2.3, the global feature F' is input into the decoder network to obtain the reconstructed point cloud PC; the specific process is:

[0062] The global feature F' is used as the input of the decoder. The global feature F' passes through the decoder in turn through 4 layers of fully connected layers. The dimension of the last layer prediction output is N×3. As the reconstructed point cloud PC

[0063] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0064] Specific implementation eight: This implementation differs from specific implementations one to seven in that:

[0065] The loss function calculation formula of the reconstructed point cloud PC and the input point cloud P in step 2.4 is expressed as:

[0066]

[0067] Where x represents a point in the input point cloud P, y represents a point in the reconstructed point cloud PC; Pnum and represent the number of points in the input point cloud P, and PCnum represents the number of points in the reconstructed point cloud PC; express.

[0068] Based on the minimization of the loss function, the masked self-encoding point cloud reconstruction network is self-supervised trained through the back-propagation algorithm to optimize the network weights. The network weights when the loss function is minimized are used as the trained masked self-encoding point cloud reconstruction network. Benefiting from the masked reconstruction training method, the trained network has the ability to extract robust features of the point cloud as a whole, and can improve the registration ability of multi-source point clouds with local missing points.

[0069] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0070] Specific implementation method 9: This implementation method is different from specific implementation methods 1 to 8 in that:

[0071] In the step 3, the multi-source remote sensing point cloud to be registered and the template point cloud are obtained; the transformation relationship of the multi-source remote sensing point cloud to be registered is obtained according to the multi-source remote sensing point cloud to be registered and the template point cloud and the masked self-encoding point cloud trained in step 2. The specific process is:

[0072] Step 3.1: Obtain the multi-source remote sensing point cloud S and template point cloud T to be registered;

[0073] The multi-source point cloud refers to point cloud data obtained by two or more methods for the same scene, and the point cloud data obtained by one of the methods can be arbitrarily designated as the template point cloud T, and the other as the point cloud to be registered S;

[0074] Step 3.2: Set the mask rate of the random mask module in the trained masked autoencoder point cloud reconstruction network to 0;

[0075] Step 3.3: Input the multi-source remote sensing point cloud S to be registered into the random mask module and encoder network in the trained mask self-encoder point cloud reconstruction network in turn to obtain the intermediate feature G(S);

[0076] The template point cloud T is sequentially input into the random mask module and encoder network in the trained mask self-encoder point cloud reconstruction network to obtain the intermediate feature G(T);

[0077] Step 3.4: Construct a registration matrix based on the intermediate features G(S) and G(T); the registration matrix is ​​expressed as follows:

[0078] G(S)=G(γ -1 ·T) (2)

[0079] γ represents the transformation relationship of the multi-source remote sensing point cloud to be registered;

[0080] Step 3.5: Linearize the registration matrix to obtain the linearized registration matrix, which is expressed as:

[0081]

[0082] φ represents, ξ represents the transformation increment;

[0083] Step 3.6: Calculate the Jacobian matrix of the linearized registration matrix, which is expressed as:

[0084]

[0085] J represents the analytical derivative of the Jacobian matrix;

[0086] The analytical derivative of the Jacobian matrix refers to the derivative of each element in the Jacobian matrix, usually expressed as a partial derivative;

[0087] Step 3.7: Use differential gradient approximation for each column of the Jacobian matrix to obtain an approximated Jacobian matrix, where the i-th column J of the approximated Jacobian matrix is i The formula is:

[0088]

[0089] t i It is expressed as an infinitesimal variable of γ, so in actual use, ti is set to a small fixed value to obtain better results. i is the exponential mapping result of the transformation matrix; exp() represents exponential mapping; X i is the exponential mapping result of the transformation matrix, and the transformation matrix is ​​solved by the Jacobian matrix, which is a processing process well known to those skilled in the art.

[0090] Step 3.8: Calculate based on the approximated Jacobian matrix J′ and get the transformation increment ξ, which can be expressed as:

[0091] ξ=J + [G(S)-G(T)] (6)

[0092] Among them J + represents the plus-sign generalized inverse matrix of the approximated Jacobian matrix J′;

[0093] The process of solving the plus-sign generalized inverse matrix of the approximated Jacobian matrix J′ is a calculation process well known to those skilled in the art.

[0094] Step 3.9: Set the threshold and the maximum number of iterations; when the transformation increment ξ obtained in step 3.8 is less than the specified threshold or reaches the maximum number of iterations, the transformation increments obtained in each round of iterations are multiplied to obtain the multiplied transformation increments, and then the multiplied transformation increments are exponentially mapped to obtain the transformation relationship γ;

[0095] When the transformation increment ξ obtained in step 3.8 is not less than the specified threshold and has not reached the maximum number of iterations, the multi-source remote sensing point cloud S to be registered is transformed into a point cloud S' according to the transformation increment ξ; it can be expressed as:

[0096] S'=ξ·S (7)

[0097] Input the point cloud S' into the encoder network in the trained masked self-encoder point cloud reconstruction network to obtain the intermediate feature G(S') and then return to S3.4 for iteration.

[0098] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0099] This step optimizes and solves the gradient direction of the feature difference between the point cloud to be registered and the template point cloud, combining the optimization algorithm with the deep learning algorithm, thus improving the accuracy of multi-source point cloud registration.

[0100] Specific implementation method 10: This implementation method is different from specific implementation methods 1 to 9 in that:

[0101] In step 4, the multi-source remote sensing point cloud to be registered is transformed according to the transformation relationship of the multi-source remote sensing point cloud to be registered, so as to obtain the registered multi-source remote sensing point cloud. The specific process is as follows:

[0102] According to the transformation relationship γ of the multi-source remote sensing point cloud to be registered, the point cloud S to be registered is rigidly transformed to obtain the registered point cloud S out , expressed as:

[0103] S out =γ·S (8)

[0104] The rigid transformation refers to a combination of rotation and translation transformation, which is a definition well known to those skilled in the art.

[0105] The other steps and parameters are the same as those in Specific Implementations 1 to 9.

[0106] In combination with the multi-source point cloud registration method designed by the present invention, a registration experiment was carried out on the lidar point cloud collected by the drone and the image-based 3D reconstruction point cloud dataset. The registration results are shown in Figure 2 As shown:

[0107] The laser radar used is VELODYNE VLP16 LITE, and the camera used for image acquisition is Micasenserededge-MXDUAL. The acquisition location is located in the Science Park of Harbin Institute of Technology. The nearest neighbor point root mean square error is used as the evaluation index for registration. The registration accuracy of this method reaches 0.35m, which is higher than the accuracy of existing methods.

[0108] The above only describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above specific implementation methods. Although the present invention has been disclosed as above in the preferred embodiments, it is not used to limit the present invention. Any technician familiar with the profession can make some changes or modify the technical contents disclosed above into equivalent embodiments with equivalent changes without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A self-supervised registration method for multi-source remote sensing point clouds, characterized in that: The following steps are involved: Step 1: Obtain multi-source remote sensing point cloud; Step 2: Construct a masked self-encoding point cloud reconstruction network, and train the constructed masked self-encoding point cloud reconstruction network according to the multi-source remote sensing point cloud to obtain a trained masked self-encoding point cloud reconstruction network; Step 3: Obtain the multi-source remote sensing point cloud and template point cloud to be registered; reconstruct the network according to the multi-source remote sensing point cloud and template point cloud to be registered and the masked self-encoder point cloud trained in step 2 to obtain the transformation relationship of the multi-source remote sensing point cloud to be registered; Step 4: transform the multi-source remote sensing point cloud to be registered according to the transformation relationship of the multi-source remote sensing point cloud to obtain the registered multi-source remote sensing point cloud.

2. The self-supervised registration method of multi-source remote sensing point cloud according to claim 1, characterized in that: The multi-source remote sensing point cloud in step 1 includes: a lidar point cloud and a three-dimensional reconstruction point cloud.

3. The self-supervised registration method of multi-source remote sensing point cloud according to claim 2, characterized in that: The masked self-encoding point cloud reconstruction network in step 2 includes: a random mask module, an encoder network and a decoder network; The masked self-encoding point cloud reconstruction network is trained according to the multi-source remote sensing point cloud to obtain a trained masked self-encoding point cloud reconstruction network; the specific process is: Step 2.1: Take the multi-source remote sensing point cloud as the input point cloud P, input it into the random mask module, and obtain the masked point cloud P'; Step 2.2: Input the masked point cloud P' into the encoder network to obtain the global feature F'; Step 2.3: Input the global feature F' into the decoder network to obtain the reconstructed point cloud PC; Step 2.4: Calculate the loss function of the reconstructed point cloud PC and the input point cloud P, and train the masked self-encoder point cloud reconstruction network according to the loss function. The training is completed when the loss function is minimized, and the trained masked self-encoder point cloud reconstruction network is obtained.

4. The self-supervised registration method of multi-source remote sensing point cloud according to claim 3, characterized in that: In step 2.1, the multi-source remote sensing point cloud is used as the input point cloud P, and the random mask module is input to obtain the masked point cloud P'; The specific process is: Step 2.1.1: Set the number of sampling points to N, where N is a positive integer, and randomly select a point in the input point cloud P as the initial sampling point P1; Step 2.1.2: Calculate the distance between the unselected points in the input point cloud P and the initial sampling point P1, and select the point with the largest distance as the sampling point P2; Step 2.1.3: Calculate the minimum distance from the unselected point in the input point cloud P to the existing sampling point set, and select the point with the largest minimum distance as the sampling point; Step 2.1.4: Repeat step 2.1.

3. When N sampling points are obtained, stop the execution and go to step 2.1.

5. The nth sampling point obtained is denoted as Pn. Step 2.1.5: N sampling points are combined into the farthest point sampling point cloud PS; Determine the mask point cloud PK according to the sampled point cloud PS; Step 2.1.6: Mask the input point cloud P according to the mask point cloud PK obtained in step 2.1.5 to obtain the masked point cloud P'.

5. The self-supervised registration method of multi-source remote sensing point cloud according to claim 4, characterized in that: The specific process of determining the mask point cloud PK according to the sample point cloud PS in step 2.1.5 is as follows: In the input point cloud P, the nearest K points of each point in the farthest point sampling point cloud PS are searched in turn as the mask point cloud PK.

6. The self-supervised registration method of multi-source remote sensing point cloud according to claim 5, characterized in that: The encoder network structure in step 2.2 includes: an MLP layer and a maximum pooling layer in sequence; The MLP layer includes an input layer, a first hidden layer, a second hidden layer, a third hidden layer, a fourth hidden layer and an output layer in sequence; the first hidden layer, the second hidden layer, the third hidden layer and the fourth hidden layer are all convolutional layers; The convolution kernel size of the first hidden layer is 1×3; the convolution kernel sizes of the second hidden layer, the third hidden layer and the fourth hidden layer are all 1×1.

7. The self-supervised registration method of multi-source remote sensing point cloud according to claim 6, characterized in that: The decoder network structure in step 2.3 includes a first fully connected layer, a second fully connected layer, a third fully connected layer and a fourth fully connected layer in sequence; The activation functions of the first fully connected layer, the second fully connected layer, the third fully connected layer and the fourth fully connected layer are all LeakyReLU.

8. The self-supervised registration method of multi-source remote sensing point cloud according to claim 7, characterized in that: The loss function calculation formula of the reconstructed point cloud PC and the input point cloud P in step 2.4 is expressed as: Where x represents a point in the input point cloud P, y represents a point in the reconstructed point cloud PC; Pnum and represent the number of points in the input point cloud P, and PCnum represents the number of points in the reconstructed point cloud PC; Represents the square of the L2 norm.

9. The self-supervised registration method of multi-source remote sensing point cloud according to claim 8, characterized in that: In the step 3, the multi-source remote sensing point cloud to be registered and the template point cloud are obtained; the transformation relationship of the multi-source remote sensing point cloud to be registered is obtained according to the multi-source remote sensing point cloud to be registered and the template point cloud and the masked self-encoding point cloud trained in step 2. The specific process is: Step 3.1: Obtain the multi-source remote sensing point cloud S and template point cloud T to be registered; Step 3.2: Set the mask rate of the random mask module in the trained masked autoencoder point cloud reconstruction network to 0; Step 3.3: Input the multi-source remote sensing point cloud S to be registered into the random mask module and encoder network in the trained mask self-encoder point cloud reconstruction network in turn to obtain the intermediate feature G(S); Then the template point cloud T is sequentially input into the random mask module and encoder network in the trained mask self-encoder point cloud reconstruction network to obtain the intermediate feature G(T); Step 3.4: Construct a registration matrix based on the intermediate features G(S) and G(T); The registration matrix is ​​expressed as: G(S)=G(γ -1 ·T) (2) Among them, γ is the transformation relationship of the multi-source remote sensing point cloud to be registered; Step 3.5: Linearize the registration matrix to obtain the linearized registration matrix. The linearized registration matrix is ​​expressed as follows: Where ξ represents the transformation increment; Step 3.6: Calculate the Jacobian matrix J of the linearized registration matrix, The Jacobian matrix of the linearized registration matrix is ​​expressed as follows: Step 3.7: Use differential gradient approximation for each column of the Jacobian matrix to obtain the approximate Jacobian matrix J′. The i-th column of the approximated Jacobian matrix J′ is represented by J i ′,J i The formula of ′ is: t i The variable represented by γ, X i is an intermediate variable; exp() represents exponential mapping; Step 3.8: Calculate based on the approximated Jacobian matrix J′ and get the transformation increment ξ, which can be expressed as: ξ=J + [G(S)-G(T)] (6) Among them J + represents the plus-sign generalized inverse matrix of the approximated Jacobian matrix J′; Step 3.9: Set the threshold and the maximum number of iterations; when the transformation increment ξ obtained in step 3.8 is less than the specified threshold or reaches the maximum number of iterations, the transformation increments obtained in each round of iteration are multiplied to obtain the multiplied transformation increments, and then the multiplied transformation increments are exponentially mapped to obtain the transformation relationship γ of the multi-source remote sensing point cloud to be registered; When the transformation increment ξ obtained in step 3.8 is not less than the specified threshold and has not reached the maximum number of iterations, According to the transformation increment ξ, the multi-source remote sensing point cloud S to be registered is transformed into a point cloud S'; it can be expressed as: S'=ξ·S (7) Input the point cloud S' into the encoder network in the trained masked self-encoder point cloud reconstruction network to obtain the intermediate feature G(S') and then return to S3.4 for iteration.

10. The self-supervised registration method of multi-source remote sensing point cloud according to claim 9, characterized in that: In step 4, the multi-source remote sensing point cloud to be registered is transformed according to the transformation relationship of the multi-source remote sensing point cloud to be registered, so as to obtain the registered multi-source remote sensing point cloud. The specific process is as follows: According to the transformation relationship γ of the multi-source remote sensing point cloud to be registered, the point cloud S to be registered is rigidly transformed to obtain the registered point cloud S out , expressed as: S out =γ·S (8).

Images