A point cloud registration method and system based on a rotation equivariant network

Abstract

The application discloses a point cloud registration method and system based on a rotation equivariant network, and the method comprises the following steps: obtaining point clouds to be registered; processing each point cloud to be registered by using a first network based on point rotation equivariance and translation invariance, obtaining a reference direction and a key point score of each point cloud; extracting key points according to the key point score, and extracting a rotation invariant and SO(2) equivariant feature for each key point according to the reference direction and a second network based on patches; based on a preset matching strategy, generating point correspondences with a high inlier rate according to the features; and solving a relative pose by using a corresponding grouping algorithm according to the point correspondences, thereby completing point cloud registration. The embodiment of the application constructs an efficient and universal feature learning framework, can generate point correspondences with a high inlier rate by using rotation invariant and SO(2) equivariant features, and can be widely applied to the technical field of computers.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly to a point cloud registration method and system based on rotation equivariant network. BACKGROUND

[0002] Noun explanation:

[0003] Point cloud: a set of unordered 3D points, which is a discrete representation of the surface of a three-dimensional object, reflecting the real geometry of the object.

[0004] Point cloud registration: for two point clouds with overlapping regions, the purpose of point cloud registration is to estimate an optimal transformation matrix, so that the two point clouds can be aligned.

[0005] Point-based network: a feature learning network that outputs the features of each point.

[0006] Patch: a local surface.

[0007] Patch-based network: a feature learning network that outputs the features of each patch.

[0008] Inlier rate: the ratio of correct point correspondences in a series of point correspondences.

[0009] Rotation equivariant: refers to the variable that changes with the rotation of the point cloud, and the change can be used to deduce the rotation amount.

[0010] Translation invariant: refers to the variable that does not change with the translation of the point cloud.

[0011] SO(2): a subgroup of SO(3) rotation group, which only contains rotations in the XoY plane.

[0012] Point cloud registration plays a crucial role in three-dimensional reconstruction and robot navigation. Given two or more partially overlapping 3D scans, the purpose of point cloud registration is to align these consecutive point clouds to a common coordinate system. An ideal registration framework not only requires accurate and efficient alignment of geometric shapes, but also can be generalized to unknown scenes. However, due to limited data quality (e.g. existence of noise, non-uniform density, different viewing angles and different data distribution), it is challenging to achieve a satisfactory balance between efficiency, accuracy and generalization.

[0013] Existing registration techniques are mainly divided into correspondence-based registration methods and correspondence-free registration methods. By establishing a series of reliable point correspondences, correspondence-based methods usually have better registration performance than correspondence-free methods, especially for large-scale scene point cloud registration. However, these correspondence-based registration methods still cannot meet the actual application requirements, because they are either not accurate and efficient enough, or lack good generalization.

[0014] In the field of point cloud registration, the following problems exist:

[0015] 1. There is no unified, efficient and general feature learning framework. In order to ensure efficiency, existing methods usually use point-based networks to extract features, but such features are difficult to generalize. In order to improve generalization, existing methods turn to use patch-based networks to extract features, but such methods are inefficient. Therefore, the current registration methods are either inefficient or lack generalization.

[0016] 2. There is no effective and general matching mechanism. Most of the correspondence-based registration frameworks directly establish point correspondences through nearest neighbor feature matching, but such method usually results in a low inlier ratio. When the inlier ratio is low, the quality of point cloud registration is poor. SUMMARY

[0017] Therefore, the embodiments of the present application provide an efficient point cloud registration method and system based on rotation equivariant network, which can generate point correspondences with high inlier ratio.

[0018] An aspect of the embodiments of the present application provides a point cloud registration method based on rotation equivariant network, comprising:

[0019] obtaining point clouds to be registered;

[0020] processing each of the point clouds to be registered by using a first network based on point rotation equivariant and translation invariant, to obtain reference direction and key point score of each point cloud;

[0021] extracting key points according to the key point score, and extracting rotation invariant and SO(2) equivariant features for each key point according to the reference direction and a second network based on patch;

[0022] generating point correspondences with high inlier ratio according to the features based on a preset matching strategy;

[0023] solving a relative pose by a correspondence grouping algorithm according to the point correspondences, to complete point cloud registration.

[0024] Optionally, the processing of each of the point clouds to be registered by using the first network based on point rotation equivariant and translation invariant to obtain reference direction and key point score of each point cloud comprises:

[0025] extracting a feature matrix rotation equivariant and translation invariant from the point clouds to be registered;

[0026] constructing a point-based rotation equivariant network with equivariant convolution as a basic operation sub to perform equivariant convolution operation, to learn deep rotation equivariant features;

[0027] According to the feature matrix and the rotation equivariant feature, a reference direction and a key point score of each point are estimated.

[0028] Optionally, the key point extraction according to the key point score and the extraction of rotation-invariant and SO(2) equivariant features for each key point according to the reference direction and the patch-based second network comprise:

[0029] The key point scores of each point are sorted, and the top K points with the highest scores are determined as key points.

[0030] A local patch centered on the key point is searched, each local patch is rotated to align the local patch with its reference direction, and a rotation-invariant and SO(2) equivariant feature for each key point is generated by using a patch-based network.

[0031] Optionally, the generating of point correspondences with a high inlier rate according to the features based on the preset matching strategy comprises:

[0032] An initial point correspondence is generated by matching using rotation-invariant features.

[0033] A rigid transformation for each group of point correspondences is estimated by constructing a matching cost and a cost aggregation using SO(2) equivariant features.

[0034] Point correspondences with a high inlier rate are determined based on geometric constraints according to the rigid transformation.

[0035] Optionally, the estimating of a rigid transformation for each group of point correspondences by constructing a matching cost and a cost aggregation using SO(2) equivariant features comprises:

[0036] A pair of SO(2) equivariant feature maps matched by each group of point correspondences is obtained according to an initial matching relationship.

[0037] A regression network is constructed to estimate a rotation matrix according to the properties of SO(2) equivariant features.

[0038] A rigid transformation between each group of point correspondences is derived according to the rotation matrix.

[0039] Optionally, the constructing of a regression network to estimate a rotation matrix according to the properties of SO(2) equivariant features comprises:

[0040] A matching cost is obtained by horizontally rotating the first equivariant feature in a pair of SO(2) equivariant feature maps and subtracting the second equivariant feature.

[0041] The matching cost is input into a 3D convolutional neural network for cost aggregation, and the probability of each rotation is calculated by a softmax function.

[0042] The probability and rotation angle are multiplied and summed to obtain the rotation angle difference between the two isovariant features, and the rotation matrix corresponding to the rotation angle difference is determined.

[0043] Another aspect of this invention provides a point cloud registration system based on a rotationally equivariant network, comprising:

[0044] The first module is used to acquire the point cloud to be registered;

[0045] The second module is used to process each of the point clouds to be registered using a first network that is rotationally equivariant and translationally invariant based on points, and to obtain the reference direction and key point score of each point cloud.

[0046] The third module is used to extract key points based on the key point scores, and extract rotation-invariant and SO(2) equivalent features for each key point based on the reference direction and the patch-based second network.

[0047] The fourth module is used to generate point correspondences with a high interior point rate based on the features according to a preset matching strategy.

[0048] The fifth module is used to solve the relative pose based on the point correspondence using a corresponding grouping algorithm to complete point cloud registration.

[0049] Another aspect of the present invention provides an electronic device, including a processor and a memory;

[0050] The memory is used to store programs;

[0051] The processor executes the program to implement the method described above.

[0052] Another aspect of this invention provides a computer-readable storage medium storing a program that is executed by a processor to implement the methods described above.

[0053] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0054] The embodiments of the present invention first acquire point clouds to be registered; each point cloud to be registered is processed using a first network based on rotational equivalence and translational invariance to obtain the reference orientation and keypoint score of each point cloud; keypoints are extracted based on the keypoint scores, and rotation-invariant and SO(2) equivalence features are extracted for each keypoint based on the reference orientation and a patch-based second network; based on a preset matching strategy, point correspondences with high inlier rates are generated based on the features; based on the point correspondences, the relative poses are solved using a correspondence grouping algorithm to complete the point cloud registration. The embodiments of the present invention introduce a point-based network based on rotational equivalence and translational invariance to predict the reference orientation and keypoint score of each point, and construct an efficient and general feature learning framework that can generate point correspondences with high inlier rates using rotation-invariant and SO(2) equivalence features. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A flowchart illustrating the overall steps of an embodiment of the present invention;

[0057] Figure 2 A flowchart for obtaining point-by-point reference direction and key point score provided in an embodiment of the present invention;

[0058] Figure 3 A flowchart illustrating the generation of point correspondences with a high interior point rate using a coarse-to-fine matching strategy provided in this embodiment of the invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0060] To address the problems existing in the prior art, one aspect of this invention provides a point cloud registration method based on a rotationally equivariant network, comprising:

[0061] Obtain the point cloud to be registered;

[0062] Each point cloud to be registered is processed using a first network that is rotationally equivariant and translationally invariant based on points to obtain the reference orientation and key point score for each point cloud.

[0063] Keypoints are extracted based on the keypoint scores, and rotation-invariant and SO(2) equivalent features are extracted for each keypoint based on the reference direction and the patch-based second network.

[0064] Based on a preset matching strategy, point correspondences with a high interior point rate are generated according to the features.

[0065] Based on the point correspondence, the relative pose is solved by the corresponding grouping algorithm to complete the point cloud registration.

[0066] Optionally, the step of processing each of the point clouds to be registered using a point-based rotation-equal and translation-invariant first network to obtain the reference orientation and keypoint score for each point cloud includes:

[0067] Extract rotationally equivalent and translationally invariant feature matrices from the point cloud to be registered;

[0068] We construct a point-based rotational equivariant network using equivariant convolution as the basic operator to perform equivariant convolution operations and learn deep-level rotational equivariant features.

[0069] Based on the feature matrix and the rotational isomorphic features, the reference orientation and keypoint score of each point are estimated.

[0070] Optionally, the keypoint extraction based on the keypoint score, and the extraction of rotation-invariant and SO(2) equivariant features for each keypoint based on the reference orientation and the patch-based second network, includes:

[0071] Sort the keypoint scores for each point and identify the top K points with the highest scores as keypoints;

[0072] Find local patches centered on keypoints, rotate each local patch to align it with its own reference direction, and use a patch-based network to generate a rotation-invariant and SO(2) equivalent feature for each keypoint.

[0073] Optionally, generating point correspondences with a high interior ratio based on the features according to a preset matching strategy includes:

[0074] Matching is performed using rotation-invariant features to generate initial point correspondences;

[0075] By utilizing the equivariant characteristics of SO(2), a rigid body transformation is estimated for each set of points by constructing matching costs and cost aggregation.

[0076] Based on the rigid body transformation, point correspondences with high interior point ratios are determined according to geometric constraints.

[0077] Optionally, the step of utilizing the equivariant features of SO(2) to estimate a rigid body transformation for each set of points by constructing matching costs and cost aggregation includes:

[0078] Based on the initial matching relationship, obtain a pair of SO(2) isovariant feature maps corresponding to each set of points;

[0079] Based on the properties of the SO(2) isovariant features, a regression network is constructed to estimate the rotation matrix;

[0080] The rigid body transformations between each set of points are derived based on the rotation matrix.

[0081] Optionally, the step of constructing a regression network to estimate the rotation matrix based on the properties of the SO(2) isovariant features includes:

[0082] After horizontally rotating the first isovariant feature in a pair of SO(2) isovariant feature maps, subtracting it from the second isovariant feature to obtain a matching cost;

[0083] The matching cost is input into a 3D convolutional neural network for cost aggregation, and the probability of each rotation is calculated using the softmax function;

[0084] The probability and rotation angle are multiplied and summed to obtain the rotation angle difference between the two isovariant features, and the rotation matrix corresponding to the rotation angle difference is determined.

[0085] Another aspect of this invention provides a point cloud registration system based on a rotationally equivariant network, comprising:

[0086] The first module is used to acquire the point cloud to be registered;

[0087] The second module is used to process each of the point clouds to be registered using a first network that is rotationally equivariant and translationally invariant based on points, and to obtain the reference direction and key point score of each point cloud.

[0088] The third module is used to extract key points based on the key point scores, and extract rotation-invariant and SO(2) equivalent features for each key point based on the reference direction and the patch-based second network.

[0089] The fourth module is used to generate point correspondences with a high interior point rate based on the features according to a preset matching strategy.

[0090] The fifth module is used to solve the relative pose based on the point correspondence using a corresponding grouping algorithm to complete point cloud registration.

[0091] Another aspect of the present invention provides an electronic device, including a processor and a memory;

[0092] The memory is used to store programs;

[0093] The processor executes the program to implement the method described above.

[0094] Another aspect of this invention provides a computer-readable storage medium storing a program that is executed by a processor to implement the methods described above.

[0095] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0096] The specific implementation process and details of the present invention will be described in detail below with reference to the accompanying drawings:

[0097] The purpose of this invention is to provide an efficient, accurate, and universal point cloud registration method, which has the following characteristics:

[0098] 1. This invention introduces a point-based rotation-equalized and translation-invariant network to predict point-by-point reference orientation and key point scores.

[0099] 2. This invention ingeniously combines point-based and patch-based networks to provide an efficient and universal feature learning framework.

[0100] 3. This invention utilizes the characteristics of rotation invariance and SO(2) equivariance to propose a coarse-to-fine matching strategy, aiming to generate point correspondences with a high interior point rate.

[0101] refer to Figure 1 The overall implementation process of this invention includes the following steps:

[0102] Step 1: Use a point-based rotation-invariant and translation-invariant network to process each point cloud, obtain the reference direction and key point score for each point, thereby improving the representation power and learning efficiency of the features.

[0103] To estimate robust reference orientation and keypoint scores, the reference orientation typically needs to be rotationally equivariant, while the keypoint scores must be rotationally invariant. The key to obtaining the required reference orientation and keypoints lies in constructing a neural network to learn point-by-point rotationally equivariant and translationally invariant features. Existing methods either have extremely high space and time complexity or rely on translation-sensitive global point coordinates. Therefore, these methods cannot extract point-by-point rotationally equivariant and translationally invariant features. To address this, this invention proposes a point-based rotationally equivariant and translationally invariant network to obtain point-by-point reference orientation and keypoint scores. Figure 2 As shown, the detailed steps are as follows: Step 1-Step 3:

[0104] Step 1: Extract the rotation-equivalent and translation-invariant feature matrix. To ensure the rotation equivariance and translation invariance of the network, this embodiment of the invention first studies the following three geometric properties of the point cloud:

[0105] (1) Relative coordinates: p ji =p j -p i

[0106] (2) Neighborhood center:

[0107] (3) Initial direction: n i It is the covariance matrix The eigenvector corresponding to the smallest eigenvalue.

[0108] Where, N i Point p i It supports all adjacent points within a radius R. Based on the above three geometric properties, this embodiment of the invention constructs a rotationally equivariant and translationally invariant feature matrix.

[0109] Step 2: Learning Deep-Level Rotational Equivariance Features. To further extract point-by-point deep-level rotational equivariance features, this embodiment of the invention constructs a point-based rotational equivariance network using existing equivariant convolution as the basic operator. Specifically, at layer l, point p... i The isovariant convolution operation at a point can be represented as:

[0110]

[0111] Where W is a weight matrix, and EquivariantConv corresponds to a rotationally equivariant operator. Since It is SO(3) rotationally equivariant and translationally invariant, therefore the entire convolutional network also possesses the same rotational equivariance and translational invariance. The entire point-based rotationally equivariant network adopts a hierarchical feature learning architecture. Compared with existing rotationally equivariant networks, the proposed point-based rotationally equivariant network is very lightweight, thus possessing high efficiency. Furthermore, the proposed method has both rotational equivariance and translational invariance, making it the first rotationally equivariant network applicable to scene-level tasks.

[0112] Step 3: Estimate the reference orientation and keypoint score for each point. In this embodiment of the invention, the isovariant features from the last convolutional layer are input into two independent branches. One branch estimates the point-by-point reference orientation, and the other branch estimates the keypoint score. The estimated reference orientation provides a robust local reference coordinate system for each point, thereby improving the representational power of subsequent feature descriptors. The estimated keypoint score can be used to extract fewer but better keypoints, thus improving the efficiency of the entire feature learning framework.

[0113] Step 2: Extract keypoints based on keypoint scores, and extract rotation-invariant and SO(2) equivalent features for each keypoint using reference orientation and a patch-based network. Detailed steps are as follows:

[0114] Step 1: Extract key points. Sort the key point scores for each point, and consider the top K points with the highest scores as key points.

[0115] Step 2: Extract rotation-invariant and SO(2) equivalent features for each keypoint. First, find local patches centered on the keypoints, then rotate each local patch to align it with its own reference direction (where keypoint p...). i The corresponding rotation matrix is ​​R i This eliminates rotational variations along the Z-axis. Finally, an existing patch-based network is used to generate a rotationally invariant and SO(2) isovariant feature for each keypoint.

[0116] Step 3: Generate point correspondences with a high interior ratio using a coarse-to-fine matching strategy. Specific detailed steps are as follows: Figure 3 As shown:

[0117] Step 1: Use rotation-invariant features for matching to generate initial point correspondences. Perform nearest neighbor matching on the rotation-invariant features of point clouds P and Q to obtain a series of initial point correspondences Ω. init .

[0118] Step 2: Utilizing the isovariant characteristics of SO(2), a rigid body transformation is estimated for each set of points by constructing matching costs and cost aggregation. Given a series of initial matching relations Ω... init Each set of points corresponds to {pi ,q j Each corresponds to a pair of SO(2) isovariant feature maps. Where H, W, and D represent the height, width, and feature dimension of the feature map, respectively. Based on the properties of SO(2) isovariant features, and The only difference between them is the rotation on SO(2). Therefore, this embodiment of the invention constructs a regression network to estimate the rotation matrix: firstly, ... Rotate horizontally W times, with each rotation angle being W. After rotation and Subtracting them, we construct a matching cost V∈R. H×W×W×D Subsequently, the matching cost V is input into a 3D convolutional neural network (3DCNN) for cost aggregation: C:R H×W×W×D →R W The probability of each rotation is calculated using the softmax function. The probabilities are multiplied by the rotation angles and summed to obtain the rotation angle difference θ between the two isovariant features, and the corresponding rotation matrix is:

[0119]

[0120] Combining Step 2 in Step 2, the rigid body transformation between each set of points can be derived as follows:

[0121] Step 3: Based on geometric constraints, a series of point correspondences with high interior point ratios are obtained. Since correct point correspondences all have similar rigid body transformations, while incorrect point correspondences do not satisfy this condition, this embodiment of the invention employs a geometric constraint to find these correct point correspondences. This embodiment of the invention first finds an optimal rigid body transformation using the following formula:

[0122]

[0123] Where 1(·) represents the indicator function, ‖·‖ represents the Euclidean distance, and τ is a distance threshold. Subsequently, point correspondences satisfying the following geometric constraints are considered exact point correspondences:

[0124]

[0125] Step 4: Solve for the relative pose using the corresponding grouping algorithm. The detailed steps are as follows:

[0126] Step 1: Use existing correspondence grouping algorithms (such as RANSAC) to perform fine point correspondence Ω. fine Pose estimation is performed to output the final transformation matrix.

[0127] In summary, the point cloud registration method implemented in this invention provides an efficient and general feature learning framework by introducing a point-based rotationally equivariant network and combining it with a patch-based network. Subsequently, a feature matcher is designed using rotation-invariant and SO(2) equivariant features, which can establish accurate point correspondences and thus help estimate an accurate relative pose.

[0128] Compared with the prior art, the present invention has the following advantages:

[0129] 1. High registration efficiency: A point-based rotation-equivariant and translation-invariant network is used to extract key points and their reference directions, which greatly reduces the feature extraction time.

[0130] 2. High registration accuracy: A coarse-to-fine matching strategy is used to extract point correspondences with a high inlier rate, which helps to estimate more accurate relative pose.

[0131] 3. Strong generalization ability: By combining point-based and patch-based networks to extract feature descriptors, the generalization ability of features is improved.

[0132] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0133] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0134] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0135] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0136] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0137] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0138] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0139] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0140] The above is a detailed description of the preferred embodiments of the present invention, but the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A point cloud registration method based on a rotationally equivariant network, characterized in that, include: Obtain the point cloud to be registered; Each point cloud to be registered is processed using a first network based on point rotation equivariance and translation invariance to obtain the reference direction and key point score for each point cloud. Keypoints are extracted based on the keypoint scores, and rotation-invariant and SO(2) equivariant features are extracted for each keypoint based on the reference direction and the patch-based second network; where S0(2) represents a subgroup of the SO(3) rotation group; Based on a preset matching strategy, point correspondences with a high interior point rate are generated according to the features. Based on the point correspondence, the relative pose is solved by the corresponding grouping algorithm to complete the point cloud registration.

2. The point cloud registration method based on a rotationally equivariant network according to claim 1, characterized in that, The process of using a point-based rotation-equal and translation-invariant first network to process each of the point clouds to be registered, obtaining the reference orientation and keypoint score for each point cloud, includes: Extract rotationally equivalent and translationally invariant feature matrices from the point cloud to be registered; We construct a point-based rotational equivariant network using equivariant convolution as the basic operator to perform equivariant convolution operations and learn deep-level rotational equivariant features. Based on the feature matrix and the rotational isomorphic features, the reference orientation and keypoint score of each point are estimated.

3. The point cloud registration method based on a rotationally equivariant network according to claim 1, characterized in that, The keypoint extraction based on the keypoint score, and the extraction of rotation-invariant and SO(2) equivariant features for each keypoint based on the reference direction and the patch-based second network, include: Sort the keypoint scores for each point and identify the top K points with the highest scores as keypoints; Find local patches centered on keypoints, rotate each local patch to align it with its own reference direction, and use a patch-based network to generate a rotation-invariant and SO(2) equivalent feature for each keypoint.

4. The point cloud registration method based on a rotationally equivariant network according to claim 1, characterized in that, The method of generating point correspondences with a high interior point ratio based on the preset matching strategy and the features includes: Matching is performed using rotation-invariant features to generate initial point correspondences; By utilizing the equivariant characteristics of SO(2), a rigid body transformation is estimated for each set of points by constructing matching costs and cost aggregation. Based on the rigid body transformation, point correspondences with high interior point ratios are determined according to geometric constraints.

5. The point cloud registration method based on a rotationally equivariant network according to claim 4, characterized in that, The method of utilizing the equivariant characteristics of SO(2) to estimate a rigid body transformation for each set of points by constructing matching costs and cost aggregation includes: Based on the initial matching relationship, obtain a pair of SO(2) isovariant feature maps corresponding to each set of points; Based on the properties of the SO(2) isovariant features, a regression network is constructed to estimate the rotation matrix; The rigid body transformations between each set of points are derived based on the rotation matrix.

6. The point cloud registration method based on a rotationally equivariant network according to claim 5, characterized in that, The method of constructing a regression network to estimate the rotation matrix based on the properties of the SO(2) isovariant characteristics includes: After horizontally rotating the first isovariant feature in a pair of SO(2) isovariant feature maps, subtracting it from the second isovariant feature to obtain a matching cost; The matching cost is input into a 3D convolutional neural network for cost aggregation, and the probability of each rotation is calculated using the softmax function; The probability and rotation angle are multiplied and summed to obtain the rotation angle difference between the two isovariant features, and the rotation matrix corresponding to the rotation angle difference is determined.

7. A point cloud registration system based on a rotationally equivariant network, characterized in that, include: The first module is used to acquire the point cloud to be registered; The second module is used to process each of the point clouds to be registered using a first network that is rotationally equivariant and translationally invariant based on points, and to obtain the reference direction and key point score of each point cloud. The third module is used to extract key points based on the key point scores, and extract rotation-invariant and SO(2) equivalent features for each key point based on the reference direction and the patch-based second network; where S0(2) represents a subgroup of the SO(3) rotation group; The fourth module is used to generate point correspondences with a high interior point rate based on the features according to a preset matching strategy. The fifth module is used to solve the relative pose based on the point correspondence using a corresponding grouping algorithm to complete point cloud registration.

8. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images