Neural field guided point cloud registration method and device, electronic equipment and storage medium

Abstract

The invention discloses a neural field guided point cloud registration method and device, electronic equipment and a storage medium, and belongs to the technical field of point cloud data processing, and the method comprises the following steps: building a neural network for guiding point cloud registration by using a neural field, inputting a source point cloud, and outputting rotation angles and displacements of x, y and z axes; expressing the target point cloud in the form of a neural field by using a small neural network; matrix orthogonalization constraint setting is carried out on the registered neural network; introducing the constructed target point cloud neural field to construct a loss function for the registration neural network; training a registration neural network and predicting a rotation angle and displacement by taking source point cloud data as input, obtaining a rotation matrix by utilizing matrix orthogonalization constraint, and continuously approaching an optimal transformation matrix through back propagation under the guidance of a loss function; and performing registration of the source point cloud and the target point cloud by using the optimal transformation matrix. According to the method, the neural field is introduced into point cloud registration, so that rapid and accurate point cloud registration is realized.

Description

Technical Field

[0001] The invention relates to a neural field-guided point cloud registration method, device, electronic equipment and storage medium, and belongs to the technical field of point cloud data processing. Background Art

[0002] Point cloud, as the main data for representing 3D scenes, has been applied in many industrial fields. Due to the limitation of the sensor scanning range, multiple scans are essential for large 3D scenes, and point cloud registration is the key to achieve data fusion. Most existing registration methods are formulated to minimize the geometric projection error between two point clouds to estimate the transformation matrix. Usually, it is necessary to search for corresponding points between point clouds and then perform transformation estimation based on these points. Through multiple iterations, the optimal transformation with the lowest geometric projection error can be obtained.

[0003] Recently, many deep learning-based registration algorithms have been proposed. For these methods, neural networks are used to learn robust features for accurate corresponding point search. PointNet is one of the backbone networks used for feature learning. Although learnable features make corresponding point search more robust, they are not stable enough in unseen scenes with different distribution from the training data. Therefore, a method combining deep learning and optimization is proposed. The main idea of ​​this method is to use deep neural networks to solve the transformation equations in the traditional registration problem. PointNetLK combines the Lucas-Kanade algorithm with a neural network to obtain an accurate Jacobian matrix through backpropagation. This matrix is ​​then used to solve the final transformation matrix.

[0004] However, both techniques have significant limitations that restrict their use in practical applications. Although classical methods are fast and accurate, they are overly sensitive to initial positions, noise, and the complexity and completeness of shapes, causing the algorithms to easily fall into local optimal solutions. On the other hand, learning-based methods are usually difficult to generalize to scenarios that do not conform to the distribution of the training dataset. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a neural field-guided point cloud registration method, device, electronic device and storage medium, which can achieve fast and accurate point cloud registration.

[0006] The technical solution adopted by the present invention to solve the technical problem is:

[0007] In a first aspect, an embodiment of the present invention provides a neural field guided point cloud registration method, comprising the following steps:

[0008] Step 1: Establish a neural network that uses neural fields to guide point cloud registration. The input of the registration neural network is the source point cloud, and the output is the rotation angle θ of the x, y, and z axes. x ,θ y ,θ z With displacement t x ,t y ,t z ;

[0009] Step 2, using a small neural network to represent the target point cloud in the form of a neural field;

[0010] Step 3, setting matrix orthogonalization constraints for the registered neural network;

[0011] Step 4, introducing the constructed target point cloud neural field to construct a loss function for the registration neural network;

[0012] Step 5: Using the source point cloud data as input, train the registration neural network, predict the rotation angle and displacement, obtain the rotation matrix using the matrix orthogonalization constraint, and continuously approach the optimal transformation matrix through back propagation under the guidance of the loss function;

[0013] Step 6: Use the optimal transformation matrix to align the source point cloud with the target point cloud.

[0014] As a possible implementation of this embodiment, the step of establishing a neural network for guiding point cloud registration using a neural field includes the following steps:

[0015] Use a multilayer perceptron neural network to fit the signed distance field SDF:

[0016]

[0017] Where f(x) is the shortest signed distance from the query point x to the surface S of the 3D shape O, which is used to indicate whether the query point x is inside or outside the shape O;

[0018] The surface S of a 3D shape O is implicitly represented as:

[0019]

[0020] Where x represents a point in space, represents the spatial dimension, f(x) represents the function where f(x) = 0 on the point cloud surface, and S represents the distance from the spatial point to the three-dimensional shape surface;

[0021] The space where the surface S of the 3D shape O is located is uniformly sampled and used as training data;

[0022] The parameters θ of the multilayer perceptron are optimized by minimizing the error Loss1 between the true distance and the predicted distance:

[0023]

[0024] Among them, τ is the gradient similarity weight factor, d is the true value of the signed distance, and f θ (x i ) is the predicted signed distance, I0 is the zero level set of the signed distance field, is the gradient of the signed distance predicted by the query point on the zero level set I0, n i is the true value of the normal of the query point;

[0025] If there is a point cloud normal, τ = 1, then the gradient of the query point on the SDF zero level set I0 and the corresponding normal are used as additional supervision signals, otherwise τ = 0.

[0026] As a possible implementation of this embodiment, the registered neural network is composed of a small PointNet architecture. After performing the maximum pooling operation, a global feature vector of size 1024 is obtained and input into a decoder composed of multiple fully connected layers; the last fully connected layer outputs a vector containing 6 parameters: θ x ,θ y ,θ z、 t x ,t y and t z ,θ x ,θ y and θ z、 represents the rotation angle around the x, y and z axes, t x ,t y and t z Represents translation along the x, y, and z axes.

[0027] As a possible implementation of this embodiment, the step of setting a matrix orthogonalization constraint on the registered neural network includes the following steps:

[0028] Predict the rotation angles (θ) around the x, y, and z axes x ,θ y ,θ z );

[0029] Through R=R z (θ z )×R y (θ y )×R x (θ x ) calculates the rotation matrix, R x ,R y ,R z are the rotation matrices around the x, y, and z axes respectively.

[0030] As a possible implementation of this embodiment, the loss function is:

[0031]

[0032] Among them, T(x) is the rotation and displacement transformation of point x, f θ (T(x i )) is the neural field at the query point T(x i ), is the gradient after rotation and translation of query point x, n i is the normal of the query point.

[0033] In a second aspect, an embodiment of the present invention provides a neural field guided point cloud registration device, comprising:

[0034] The network building module is used to build a neural network that uses the neural field to guide point cloud registration. The input of the registered neural network is the source point cloud, and the output is the rotation angle θ of the x, y, and z axes. x ,θ y ,θ z With displacement t x ,t y ,t z ;

[0035] A form representation module is used to represent the target point cloud in the form of a neural field using a small neural network;

[0036] A constraint setting module, used to set matrix orthogonal constraints for the registered neural network;

[0037] A loss function setting module is used to introduce the constructed target point cloud neural field to construct a loss function for the registration neural network;

[0038] The network training module is used to train the registration neural network with the source point cloud data as input, predict the rotation angle and displacement, obtain the rotation matrix using the matrix orthogonalization constraint, and continuously approach the optimal transformation matrix through back propagation under the guidance of the loss function;

[0039] The point cloud registration module is used to register the source point cloud with the target point cloud using the optimal transformation matrix.

[0040] As a possible implementation of this embodiment, the process of establishing a neural network using a neural field to guide point cloud registration is:

[0041] Use a multilayer perceptron neural network to fit the signed distance field SDF:

[0042]

[0043] Where f(x) is the shortest signed distance from the query point x to the surface S of the 3D shape O, which is used to indicate whether the query point x is inside or outside the shape O;

[0044] The surface S of a 3D shape O is implicitly represented as:

[0045]

[0046] Where x represents a point in space, represents the spatial dimension, f(x) represents the function where f(x) = 0 on the point cloud surface, and S represents the distance from the spatial point to the three-dimensional shape surface;

[0047] The space where the surface S of the 3D shape O is located is uniformly sampled and used as training data;

[0048] The parameters θ of the multilayer perceptron are optimized by minimizing the error Loss1 between the true distance and the predicted distance:

[0049]

[0050] Among them, τ is the gradient similarity weight factor, d is the true value of the signed distance, and f θ (x i ) is the predicted signed distance, I0 is the zero level set of the signed distance field, is the gradient of the signed distance predicted by the query point on the zero level set I0, n i is the true value of the normal of the query point;

[0051] If there is a point cloud normal, τ = 1, then the gradient of the query point on the SDF zero level set I0 and the corresponding normal are used as additional supervision signals, otherwise τ = 0.

[0052] As a possible implementation of this embodiment, the registered neural network is composed of a small PointNet architecture. After performing the maximum pooling operation, a global feature vector of size 1024 is obtained and input into a decoder composed of multiple fully connected (FC) layers; the last FC layer outputs a vector containing 6 parameters: θ x ,θ y ,θ z、 t x ,t y and t z ,θ x ,θ y and θ z、 represents the rotation angle around the x, y and z axes, t x ,t y and t z Represents translation along the x, y, and z axes.

[0053] As a possible implementation of this embodiment, the specific process of setting the matrix orthogonalization constraint for the registered neural network is as follows:

[0054] Predict the rotation angles (θ) around the x, y, and z axes x ,θ y ,θ z );

[0055] Through R=R z (θ z )×R y (θ y )×R x (θ x ) calculates the rotation matrix, R x ,R y ,R z are the rotation matrices around the x, y, and z axes respectively.

[0056] As a possible implementation of this embodiment, the loss function is:

[0057]

[0058] Among them, T(x) is the rotation and displacement transformation of point x, f θ (T(x i )) is the neural field at the query point T(x i ), is the gradient after rotation and translation of query point x, n i is the normal of the query point.

[0059] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of any of the above-mentioned neural field-guided point cloud registration methods.

[0060] In a fourth aspect, an embodiment of the present invention provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned neural field-guided point cloud registration methods are executed.

[0061] The beneficial effects of the technical solution of the embodiment of the present invention are as follows:

[0062] A neural field guided point cloud registration method according to the technical solution of an embodiment of the present invention comprises the following steps: Step 1, establishing a neural network (PC2NF) for point cloud registration guided by a neural field, with the input being a source point cloud and the output being a rotation angle θ of the x, y, and z axesx ,θ y ,θ z With displacement t x ,t y ,t z ; Step 2, using a small neural network to represent the target point cloud in the form of a neural field; Step 3, setting matrix orthogonalization constraints for the registered neural network; Step 4, constructing a loss function for the registration neural network. The target point cloud neural field constructed in step 2 is introduced to construct a loss function; Step 5, training the registration neural network. With the source point cloud data as input, the registration neural network is trained to predict the rotation angle and displacement, and the rotation matrix is ​​obtained using the matrix orthogonalization constraint. Under the guidance of the loss function, the optimal transformation matrix is ​​continuously approached through back propagation; Step 6, the source point cloud and the target point cloud are registered using the optimal transformation matrix. The present invention introduces the neural field into point cloud registration, converts the registration into an optimization process from point cloud to neural field, and realizes fast and accurate point cloud registration. Based on the neural field, the present invention uses the powerful ability of deep learning for modeling and optimization, thereby improving the accuracy and robustness of point cloud registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a flow chart of a neural field guided point cloud registration method according to an exemplary embodiment;

[0064] Figure 2 is a schematic structural diagram of a neural field-guided point cloud registration device according to an exemplary embodiment;

[0065] Figure 3 The figure is a PC2NF network architecture diagram according to an exemplary embodiment. DETAILED DESCRIPTION

[0066] In order to more clearly illustrate the technical features of the solution of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0067] like Figure 1 As shown, a neural field guided point cloud registration method provided by an embodiment of the present invention includes the following steps:

[0068] Step 1: Establish a neural network that uses neural fields to guide point cloud registration. The input of the registration neural network is the source point cloud, and the output is the rotation angle θ of the x, y, and z axes. x ,θ y ,θ z With displacement t x ,t y ,t z ;

[0069] Step 2, using a small neural network to represent the target point cloud in the form of a neural field;

[0070] Step 3, setting matrix orthogonalization constraints for the registered neural network;

[0071] Step 4, introducing the constructed target point cloud neural field to construct a loss function for the registration neural network;

[0072] Step 5: Using the source point cloud data as input, train the registration neural network, predict the rotation angle and displacement, obtain the rotation matrix using the matrix orthogonalization constraint, and continuously approach the optimal transformation matrix through back propagation under the guidance of the loss function;

[0073] Step 6: Use the optimal transformation matrix to align the source point cloud with the target point cloud.

[0074] As a possible implementation of this embodiment, the step of establishing a neural network for guiding point cloud registration using a neural field includes the following steps:

[0075] Use a multilayer perceptron neural network to fit the signed distance field SDF:

[0076]

[0077] Where f(x) is the shortest signed distance from the query point x to the surface S of the 3D shape O, which is used to indicate whether the query point x is inside or outside the shape O;

[0078] The surface S of a 3D shape O is implicitly represented as:

[0079]

[0080] Where x represents a point in space, represents the spatial dimension, f(x) represents the function where f(x) = 0 on the point cloud surface, and S represents the distance from the spatial point to the three-dimensional shape surface;

[0081] The space where the surface S of the 3D shape O is located is uniformly sampled and used as training data;

[0082] The parameters θ of the multilayer perceptron are optimized by minimizing the error Loss1 between the true distance and the predicted distance:

[0083]

[0084] Among them, τ is the gradient similarity weight factor, d is the true value of the signed distance, and f θ (x i ) is the predicted signed distance, I0 is the zero level set of the signed distance field, is the gradient of the signed distance predicted by the query point on the zero level set I0, n i is the true value of the normal of the query point;

[0085] If there is a point cloud normal, τ = 1, then the gradient of the query point on the SDF zero level set I0 and the corresponding normal are used as additional supervision signals, otherwise τ = 0.

[0086] As a possible implementation of this embodiment, the registered neural network is composed of a small PointNet architecture. After performing the maximum pooling operation, a global feature vector of size 1024 is obtained and input into a decoder composed of multiple fully connected layers; the last fully connected layer outputs a vector containing 6 parameters: θ x ,θ y ,θ z、 t x ,t y and t z ,θ x ,θ y and θ z、 represents the rotation angle around the x, y and z axes, t x ,t y and t z Represents translation along the x, y, and z axes.

[0087] As a possible implementation of this embodiment, the step of setting a matrix orthogonalization constraint on the registered neural network includes the following steps:

[0088] Predict the rotation angles (θ) around the x, y, and z axes x ,θ y ,θ z );

[0089] Through R=R z (θ z )×R y (θ y )×R x (θ x ) calculates the rotation matrix, R x ,R y ,R z are the rotation matrices around the x, y, and z axes respectively.

[0090] As a possible implementation of this embodiment, the loss function is:

[0091]

[0092] Among them, T(x) is the rotation and displacement transformation of point x, f θ (T(x i )) is the neural field at the query point T(x i ), is the gradient after rotation and translation of query point x, ni is the normal of the query point.

[0093] like Figure 2 As shown, an embodiment of the present invention provides a neural field guided point cloud registration device, comprising:

[0094] The network building module is used to build a neural network that uses the neural field to guide point cloud registration. The input of the registered neural network is the source point cloud, and the output is the rotation angle θ of the x, y, and z axes. x ,θ y ,θ z With displacement t x ,t y ,t z ;

[0095] A form representation module is used to represent the target point cloud in the form of a neural field using a small neural network;

[0096] A constraint setting module, used to set matrix orthogonal constraints for the registered neural network;

[0097] A loss function setting module is used to introduce the constructed target point cloud neural field to construct a loss function for the registration neural network;

[0098] The network training module is used to train the registration neural network with the source point cloud data as input, predict the rotation angle and displacement, obtain the rotation matrix using the matrix orthogonalization constraint, and continuously approach the optimal transformation matrix through back propagation under the guidance of the loss function;

[0099] The point cloud registration module is used to register the source point cloud with the target point cloud using the optimal transformation matrix.

[0100] As a possible implementation of this embodiment, the process of establishing a neural network using a neural field to guide point cloud registration is:

[0101] Use a multilayer perceptron neural network to fit the signed distance field SDF:

[0102]

[0103] Where f(x) is the shortest signed distance from the query point x to the surface S of the 3D shape O, which is used to indicate whether the query point x is inside or outside the shape O;

[0104] The surface S of a 3D shape O is implicitly represented as:

[0105]

[0106] Where x represents a point in space, represents the spatial dimension, f(x) represents the function where f(x) = 0 on the point cloud surface, and S represents the distance from the spatial point to the three-dimensional shape surface;

[0107] The space where the surface S of the 3D shape O is located is uniformly sampled and used as training data;

[0108] The parameters θ of the multilayer perceptron are optimized by minimizing the error Loss1 between the true distance and the predicted distance:

[0109]

[0110] Among them, τ is the gradient similarity weight factor, d is the true value of the signed distance, and f θ (x i ) is the predicted signed distance, I0 is the zero level set of the signed distance field, is the gradient of the signed distance predicted by the query point on the zero level set I0, n i is the true value of the normal of the query point;

[0111] If there is a point cloud normal, τ = 1, then the gradient of the query point on the SDF zero level set I0 and the corresponding normal are used as additional supervision signals, otherwise τ = 0.

[0112] As a possible implementation of this embodiment, the registered neural network is composed of a small PointNet architecture. After performing the maximum pooling operation, a global feature vector of size 1024 is obtained and input into a decoder composed of multiple fully connected (FC) layers; the last FC layer outputs a vector containing 6 parameters: θ x ,θ y ,θ z、 t x ,t y and t z ,θ x ,θ y and θ z、 represents the rotation angle around the x, y and z axes, t x ,t y and t z Represents translation along the x, y, and z axes.

[0113] As a possible implementation of this embodiment, the specific process of setting the matrix orthogonalization constraint for the registered neural network is as follows:

[0114] Predict the rotation angles (θ) around the x, y, and z axes x ,θ y ,θ z );

[0115] Through R=R z (θz )×R y (θ y )×R x (θ x ) calculates the rotation matrix, R x ,R y ,R z are the rotation matrices around the x, y, and z axes respectively.

[0116] As a possible implementation of this embodiment, the loss function is:

[0117]

[0118] Among them, T(x) is the rotation and displacement transformation of point x, f θ (T(x i )) is the neural field at the query point T(x i ), is the gradient after rotation and translation of query point x, n i is the normal of the query point.

[0119] A neural field is a field parameterized by a neural network. In visual computing, a neural field can be constructed by sampling points and their corresponding distances to the surface and then inputting them into a neural network. This paper first uses a neural field to represent the shape, which is a learnable signed distance field. The design of PC2NF is first introduced in detail, and then the registration mechanism between the neural field and the source point cloud is explained.

[0120] A. Neural fields in implicit surface representation.

[0121] Compared with explicit representations (including meshes, voxels, and point clouds), implicit surface representations provide many useful properties. Mainstream methods include occupancy fields and signed distance fields (SDFs). The present invention uses SDF representation, which is expressed as where f(x) is the shortest signed distance from a query point x to the surface S of a 3D shape O, and the sign indicates whether x is inside or outside the shape O. Therefore, the surface S of the shape O can be implicitly represented as the zero level set of f: f(x)=0.

[0122]

[0123] The present invention follows the paradigm of using neural networks to represent continuous neural SDFs. Unlike DeepSDF, which encodes the shape as a latent code z, the present invention uses a multi-layer perceptron (MLP) neural network to fit an accurate and continuous SDF. This network takes query points as input and predicts signed distances. In order to train the neural network to learn an accurate signed distance field, in addition to the surface sampling points, a uniform sampling of the space in which it is located is required as training data. The parameters θ of the MLP are optimized by minimizing the error between the true distance and the predicted distance. In addition, to improve accuracy, if a point cloud normal exists, the gradient of the query point on the zero level set I0 of the SDF and the corresponding normal are used as additional supervisory signals, and τ=1 if the normal exists, otherwise τ=0.

[0124]

[0125] The second term in this function encourages the gradient to be close to the supplied normal N.

[0126] B. Registration in neural fields.

[0127] like Figure 3 As shown, in the PC2NF network architecture of the present invention, the target point cloud P T Represented in the form of neural fields. The registration neural network consists of a small PointNet architecture, and after performing a max pooling operation, a global feature vector of size 1024 is obtained. This feature is then fed into a decoder consisting of multiple fully connected (FC) layers. The last FC layer outputs a vector containing 6 parameters: the first three parameters represent the rotation angles around the x, y, and z axes, and the last three parameters represent the translation along the x, y, and z axes.

[0128] C. Matrix orthogonalization constraint.

[0129] Different from the direct prediction of the rotation matrix or quaternion method, the present invention estimates the rotation matrix in two steps. First, the rotation angles (θ x ,θ y ,θ z ). Then, by R = R z (θ z )×R y (θ y )×R x (θ x) calculates the rotation matrix. There are multiple benefits to this approach: 1) It avoids the problem of quaternion error accumulation that usually occurs after multiple transformation operations, resulting in inaccurate rotation. Usually to alleviate this problem, a periodic normalization operation is performed once the error exceeds a set threshold; 2) It avoids the need for a singular value decomposition (SVD) operation. The method of the present invention ensures orthogonality and det(R)=1, where det(R) is the determinant of the matrix R. Therefore, compared to directly estimating the parameters of the rotation matrix, the method of the present invention avoids complex SVD operations and achieves better convergence during training.

[0130] D. Training loss.

[0131] In the neural signed distance field, for the input point set I, each point can obtain a signed distance value. If the point is on the surface, this value is zero, and the goal of the loss function of the present invention is to guide the network to minimize the sum of the signed distances of all points. In addition, since there is a gradient field in the neural distance field, the gradient constraint is introduced as a regularization term.

[0132]

[0133] The value of the gradient similarity weight factor τ is in the range [0,1]. The first term in the loss function represents the absolute signed distance of all points, which encourages the network to obtain accurate rotation angles and translations so that all points are located on the zero level set of the SDF. The second term in the loss function is the gradient regularization term.

[0134] The present invention designs a neural network PC2NF for point cloud registration guided by neural field to simulate the registration behavior of the source point cloud in the neural field learned in the target point cloud. The network rotates and translates the source point cloud in the neural field to obtain the best position and direction, when the signed distance of the point cloud should be the lowest and the gradient similarity should be the highest. The PC2NF network uses the neural field learned from the target point cloud for registration. The neural field is continuous to facilitate dynamic optimization of the transformation matrix, thereby improving the registration performance and accuracy.

[0135] Since it is difficult to ensure the orthogonality of the predicted rotation matrix, the present invention introduces a matrix orthogonality constraint optimization method to avoid error accumulation and high computational cost caused by quaternion conversion or singular value decomposition (SVD) operations, and achieve better convergence of the neural network.

[0136] The method is highly robust to noisy and residual point clouds and achieves competitive results on noise-free point clouds. In addition, compared with feature learning-based methods, the method does not require training on large-scale data sets.

[0137] An electronic device provided by an embodiment of the present invention includes a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of any neural field-guided point cloud registration method as described above.

[0138] Specifically, the above-mentioned memory and processor can be general-purpose memory and processor, which are not specifically limited here. When the processor runs the computer program stored in the memory, the above-mentioned neural field-guided point cloud registration method can be executed.

[0139] Those skilled in the art will appreciate that the structure of the electronic device does not limit the electronic device and may include more or fewer components than shown in the figure, or combine or split certain components, or arrange the components differently.

[0140] In some embodiments, the electronic device may also include a touch screen that can be used to display a graphical user interface (e.g., a startup interface of an application) and receive user operations on the graphical user interface (e.g., startup operations on an application). The specific touch screen may include a display panel and a touch panel. The display panel may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. The touch panel may collect the user's contact or non-contact operations on or near it, and generate pre-set operation instructions, for example, the user uses any suitable object such as a finger, stylus, or accessories on or near the touch panel. In addition, the touch panel may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation and posture, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into information that the processor can process, and then sends it to the processor, and can receive and execute commands from the processor. In addition, the touch panel can be implemented by various types such as resistive, capacitive, infrared and surface acoustic wave, and any technology developed in the future can also be used to implement the touch panel. Further, the touch panel can cover the display panel, and the user can operate on or near the touch panel covered on the display panel according to the graphical user interface displayed on the display panel. After the touch panel detects the operation on or near it, it is transmitted to the processor to determine the user input, and then the processor provides corresponding visual output on the display panel in response to the user input. In addition, the touch panel and the display panel can be implemented as two independent components or integrated.

[0141] Corresponding to the method for starting the above-mentioned application, an embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is run by a processor, the steps of any of the above-mentioned neural field-guided point cloud registration methods are executed.

[0142] The application startup device provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application, its implementation principle and the technical effect produced are the same as those in the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0143] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0144] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0145] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, each functional module in the embodiments provided in the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0147] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0148] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A neural field guided point cloud registration method, characterized in that: The steps include: Step 1: Establish a neural network that uses neural fields to guide point cloud registration. The input of the registration neural network is the source point cloud, and the output is the rotation angle θ of the x, y, and z axes. x ,θ y ,θ z With displacement t x ,t y ,t z ; Step 2, using a small neural network to represent the target point cloud in the form of a neural field; Step 3, setting matrix orthogonalization constraints for the registered neural network; Step 4, introducing the constructed target point cloud neural field to construct a loss function for the registration neural network; Step 5: Using the source point cloud data as input, train the registration neural network, predict the rotation angle and displacement, obtain the rotation matrix using the matrix orthogonalization constraint, and continuously approach the optimal transformation matrix through back propagation under the guidance of the loss function; Step 6: Use the optimal transformation matrix to align the source point cloud with the target point cloud.

2. The neural field guided point cloud registration method according to claim 1, characterized in that: The method of establishing a neural network for guiding point cloud registration using a neural field comprises the following steps: Use a multilayer perceptron neural network to fit the signed distance field SDF: Where f(x) is the shortest signed distance from the query point x to the surface S of the 3D shape O, which is used to indicate whether the query point x is inside or outside the shape O; The surface S of a 3D shape O is implicitly represented as: Where x represents a point in space, represents the spatial dimension, f(x) represents the function where f(x) = 0 on the point cloud surface, and S represents the distance from the spatial point to the three-dimensional shape surface; The space where the surface S of the 3D shape O is located is uniformly sampled and used as training data; The parameters θ of the multilayer perceptron are optimized by minimizing the error Loss1 between the true distance and the predicted distance: Among them, τ is the gradient similarity weight factor, d is the true value of the signed distance, and f θ (x i ) is the predicted signed distance, I0 is the zero level set of the signed distance field, is the gradient of the signed distance predicted by the query point on the zero level set I0, n i is the true value of the normal of the query point; If there is a point cloud normal, τ = 1, then the gradient of the query point on the SDF zero level set I0 and the corresponding normal are used as additional supervision signals, otherwise τ = 0.

3. The neural field guided point cloud registration method according to claim 2, characterized in that: The registered neural network consists of a small PointNet architecture. After performing the maximum pooling operation, a global feature vector of size 1024 is obtained and input into a decoder composed of multiple fully connected layers; the last fully connected layer outputs a vector containing 6 parameters: θ x ,θ y ,θ z、 t x ,t y and t z ,θ x ,θ y and θ z、 represents the rotation angle around the x, y and z axes, t x ,t y and t z Represents translation along the x, y, and z axes.

4. The neural field guided point cloud registration method according to claim 3, characterized in that: The matrix orthogonalization constraint setting for the registered neural network includes the following steps: Predict the rotation angles (θ) around the x, y, and z axes x ,θ y ,θ z ); Through R=R z (θ z )×R y (θ y )×R x (θ x ) calculates the rotation matrix, R x ,R y ,R z are the rotation matrices around the x, y, and z axes respectively.

5. The neural field guided point cloud registration method according to claim 4, characterized in that: The loss function is: Among them, T(x) is the rotation and displacement transformation of point x, f θ (T(x i )) is the neural field at the query point T(x i ), is the gradient after rotation and translation of query point x, n i is the normal of the query point.

6. A neural field guided point cloud registration device, characterized in that: include: The network building module is used to build a neural network that uses the neural field to guide point cloud registration. The input of the registered neural network is the source point cloud, and the output is the rotation angle θ of the x, y, and z axes. x ,θ y ,θ z With displacement t x ,t y ,t z ; A form representation module is used to represent the target point cloud in the form of a neural field using a small neural network; A constraint setting module, used to set matrix orthogonal constraints for the registered neural network; A loss function setting module is used to introduce the constructed target point cloud neural field to construct a loss function for the registration neural network; The network training module is used to train the registration neural network with the source point cloud data as input, predict the rotation angle and displacement, obtain the rotation matrix using the matrix orthogonalization constraint, and continuously approach the optimal transformation matrix through back propagation under the guidance of the loss function; The point cloud registration module is used to register the source point cloud with the target point cloud using the optimal transformation matrix.

7. The neural field guided point cloud registration device according to claim 6, characterized in that: The process of establishing a neural network using a neural field to guide point cloud registration is as follows: Use a multilayer perceptron neural network to fit the signed distance field SDF: Where f(x) is the shortest signed distance from the query point x to the surface S of the 3D shape O, which is used to indicate whether the query point x is inside or outside the shape O; The surface S of a 3D shape O is implicitly represented as: Where x represents a point in space, represents the spatial dimension, f(x) represents the function where f(x) = 0 on the point cloud surface, and S represents the distance from the spatial point to the three-dimensional shape surface; The space where the surface S of the 3D shape O is located is uniformly sampled and used as training data; The parameters θ of the multilayer perceptron are optimized by minimizing the error Loss1 between the true distance and the predicted distance: Among them, τ is the gradient similarity weight factor, d is the true value of the signed distance, and f θ (x i ) is the predicted signed distance, I0 is the zero level set of the signed distance field, is the gradient of the signed distance predicted by the query point on the zero level set I0, n i is the true value of the normal of the query point; If there is a point cloud normal, τ = 1, then the gradient of the query point on the SDF zero level set I0 and the corresponding normal are used as additional supervision signals, otherwise τ = 0.

8. The neural field guided point cloud registration device according to claim 7, characterized in that: The loss function is: Among them, T(x) is the rotation and displacement transformation of point x, f θ (T(x i )) is the neural field at the query point T(x i ), is the gradient after rotation and translation of query point x, n i is the normal of the query point.

9. An electronic device, characterized in that: It includes a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the neural field-guided point cloud registration method as described in any one of claims 1-5.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, executes the steps of the neural field-guided point cloud registration method as described in any one of claims 1 to 5.

Images