Neural Field Guided Point Cloud Registration Method, Device, Electronic Device and Storage Medium

Abstract

A method, device, electronic device and storage medium for point cloud registration guided by a neural field according to the present invention belong to the technical field of point cloud data processing. The method includes the following steps: establishing a neural network for point cloud registration guided by a neural field, with the input being the source point cloud and the output being the rotation angles and displacements of the x, y, and z axes; representing the target point cloud in the form of a neural field using a small neural network; setting matrix orthogonality constraints for the registration neural network; introducing the constructed neural field of the target point cloud to construct a loss function for the registration neural network; using the source point cloud data as the input to train the registration neural network, predicting the rotation angles and displacements, obtaining the rotation matrix using matrix orthogonality constraints, and continuously approaching the optimal transformation matrix through backpropagation under the guidance of the loss function; using the optimal transformation matrix to perform registration of the source point cloud and the target point cloud. The present invention introduces a neural field into point cloud registration, achieving fast and accurate point cloud registration.

Description

Technical Field

[0001] The present invention relates to a neural field-guided point cloud registration method, apparatus, electronic device and storage medium, belonging to the technical field of point cloud data processing. Background Art

[0002] Point cloud, as the main data for 3D scene representation, has been applied in many industrial fields. Due to the limitation of the sensor scanning range, for large 3D scenes, multiple scans are essential, and point cloud registration is the key to achieving data fusion. Most existing registration methods are formulated to minimize the geometric projection error between two point clouds to estimate the transformation matrix. Usually, corresponding points need to be searched between the point clouds, and then the transformation is estimated based on these points. Through multiple iterations, the optimal transformation with the lowest geometric projection error can be obtained.

[0003] Recently, many deep learning-based registration algorithms have been proposed. For these methods, neural networks are used to learn robust features for accurate corresponding point search. PointNet is used as one of the feature learning backbone networks. Although the learnable features make the corresponding point search more robust, they are not stable enough in unseen scenes with different distributions from the training data. Therefore, a method combining deep learning and optimization has been proposed. The main idea of this method is to use a deep neural network to solve the transformation equation in the traditional registration problem. PointNetLK combines the Lucas-Kanade algorithm with a neural network to obtain an accurate Jacobian matrix through backpropagation. Then this matrix is used to solve the final transformation matrix.

[0004] However, both of these two techniques have significant limitations that restrict their use in practical applications. Although the classical methods are fast and accurate, they are too sensitive to the initial position, noise, and the complexity and integrity of the shape, resulting in the algorithm being prone to falling into local optimal solutions. On the other hand, the learning-based methods are usually difficult to generalize to scenes that do not conform to the distribution of the training data set. Summary of the Invention

[0005] To solve the above problems, the present invention proposes a neural field-guided point cloud registration method, apparatus, electronic device and storage medium, which can achieve fast and accurate point cloud registration.

[0006] The technical solution adopted by the present invention to solve its technical problems is as follows:

[0007] In a first aspect, a neural field-guided point cloud registration method provided by an embodiment of the present invention includes the following steps:

[0008] Step 1: Establish a neural network for point cloud registration guided by the neural field. The input of the registration neural network is the source point cloud, and the output is the rotation angles θ x 、θ y 、θ z and the displacement t x 、t y 、t z ;

[0009] Step 2: Represent the target point cloud in the form of a neural field using a small neural network;

[0010] Step 3: Set the matrix orthogonality constraint for the registration neural network;

[0011] Step 4: Introduce the constructed neural field of the target point cloud to construct a loss function for the registration neural network;

[0012] Step 5: Use the source point cloud data as the input to train the registration neural network, predict the rotation angles and displacement, obtain the rotation matrix using the matrix orthogonality constraint, and continuously approximate the optimal transformation matrix through backpropagation under the guidance of the loss function;

[0013] Step 6: Register the source point cloud and the target point cloud using the optimal transformation matrix.

[0014] As a possible implementation of this embodiment, establishing the neural network for point cloud registration guided by the neural field includes the following steps:

[0015] Use a multi-layer perceptron neural network to fit the signed distance field SDF:

[0016]

[0017] where f(x) is the shortest signed distance from the query point x to the surface S of the 3D shape O, used to represent whether the query point x is inside or outside the shape O;

[0018] Implicitly represent the surface S of the 3D shape O as:

[0019]

[0020] where x represents the spatial point, represents the spatial dimension, f(x) represents the function where f(x) = 0 on the surface of the point cloud, and S represents the distance from the spatial point to the surface of the three-dimensional shape;

[0021] Uniformly sample the space where the surface S of the 3D shape O is located and use it as the training data;

[0022] Optimize the parameters θ of the multi-layer perceptron by minimizing the error Loss1 between the true distance and the predicted distance:

[0023]

[0024] Among them, τ is the gradient similarity weight factor, d is the signed distance ground truth, and f θ (x i ) is the predicted signed distance, I0 is the zero level set of the signed distance field, is the gradient of the predicted signed distance at the query point on the zero level set I0, and n i is the ground truth normal of the query point;

[0025] If there is a point cloud normal, τ = 1, then the gradient of the query point on the SDF zero level set I0 and the corresponding normal are used as additional supervision signals, otherwise τ = 0.

[0026] As a possible implementation of this embodiment, the registered neural network consists of a small PointNet architecture. After performing the max pooling operation, a global feature vector of size 1024 is obtained and input into a decoder composed of multiple fully connected layers; the last fully connected layer outputs a vector containing 6 parameters: θ x , θ y , θ z、 t x , t y , and t z , θ x , θ y , and θ z、 represent the rotation angles around the x, y, and z axes, and t x , t y , and t z represent the translations along the x, y, and z axes.

[0027] As a possible implementation of this embodiment, the setting of matrix orthogonality constraint for the registered neural network includes the following steps:

[0028] Predict the rotation angles (θ x , θ y , θ z ) around the x, y, and z axes;

[0029] Calculate the rotation matrix through R = R z (θ z ) × R y (θ y ) × R x (θ x ), where R x , R y , R z are the rotation matrices around the x, y, and z axes respectively.

[0030] As a possible implementation of this embodiment, the loss function is:

[0031]

[0032] where T(x) is the rotation and displacement transformation of point x, and f θ (T(x i )) is the signed distance value of the neural field at the query point T(x i ), is the gradient after rotation and translation of the query point x, and n i is the normal of the query point.

[0033] In a second aspect, a point cloud registration device guided by a neural field provided by an embodiment of the present invention includes:

[0034] A network establishment module, configured to establish a neural network for guiding point cloud registration using a neural field. The input of the registration neural network is the source point cloud, and the output is the rotation angles θ x 、θ y 、θ z of the x, y, and z axes and the displacements t x 、t y 、t z ;

[0035] A form representation module, configured to represent the target point cloud in the form of a neural field using a small neural network;

[0036] A constraint setting module, configured to perform matrix orthogonality constraint setting on the registration neural network;

[0037] A loss function setting module, configured to introduce the constructed neural field of the target point cloud to construct a loss function for the registration neural network;

[0038] A network training module, configured to use the source point cloud data as input to train the registration neural network, predict the rotation angle and displacement, obtain the rotation matrix using the matrix orthogonality constraint, and continuously approximate the optimal transformation matrix through backpropagation under the guidance of the loss function;

[0039] A point cloud registration module, configured to register the source point cloud and the target point cloud using the optimal transformation matrix.

[0040] As a possible implementation of this embodiment, the process of establishing a neural network for guiding point cloud registration using a neural field is:

[0041] Use a multi-layer perceptron neural network to fit the signed distance field SDF:

[0042]

[0043] Among them, f(x) is the shortest signed distance from the query point x to the surface S of the 3D shape O, which is used to indicate whether the query point x is inside or outside the shape O;

[0044] The surface S of the 3D shape O is implicitly represented as:

[0045]

[0046] where x represents a spatial point, represents the spatial dimension, f(x) represents the function where f(x) = 0 on the point cloud surface, and S represents the distance from the spatial point to the surface of the three-dimensional shape;

[0047] The space where the surface S of the 3D shape O is located is uniformly sampled and used as training data;

[0048] The parameters θ of the multi-layer perceptron are optimized by minimizing the error Loss1 between the true distance and the predicted distance:

[0049]

[0050] where τ is the gradient similarity weight factor, d is the true signed distance, f θ (x i ) is the predicted signed distance, I0 is the zero level set of the signed distance field, is the gradient of the predicted signed distance of the query point on the zero level set I0, and n i is the true normal of the query point;

[0051] If there is a point cloud normal and τ = 1, then the gradient of the query point on the SDF zero level set I0 and the corresponding normal are used as additional supervision signals, otherwise τ = 0.

[0052] As a possible implementation of this embodiment, the registered neural network consists of a small PointNet architecture. After performing the max pooling operation, a global feature vector of size 1024 is obtained and input into a decoder composed of multiple fully connected (FC) layers; the last FC layer outputs a vector containing 6 parameters: θ x 、θ y 、θ z、 t x 、t y and t z ,θ x 、θ y and θ z、 represent the rotation angles around the x, y, and z axes, and t x 、t y and t z represent the translations along the x, y, and z axes.

[0053] As a possible implementation of this embodiment, the specific process of setting the matrix orthogonality constraint for the registered neural network is as follows:

[0054] Predict the rotation angles (θ x , θ y , θ z ) around the x, y, and z axes;

[0055] Calculate the rotation matrix through R = R z (θ z ) × R y (θ y ) × R x (θ x ), where R x , R y , R z are the rotation matrices around the x, y, and z axes respectively.

[0056] As a possible implementation of this embodiment, the loss function is:

[0057]

[0058] where T(x) is the rotation and displacement transformation of point x, and f θ (T(x i )) is the signed distance value of the neural field at the query point T(x i ), is the gradient after rotation and translation of the query point x, and n i is the normal of the query point.

[0059] In a third aspect, an electronic device provided by an embodiment of the present invention includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of any of the above-mentioned neural field-guided point cloud registration methods.

[0060] In a fourth aspect, a storage medium provided by an embodiment of the present invention stores a computer program, and when the computer program is run by a processor, it executes the steps of any of the above-mentioned neural field-guided point cloud registration methods.

[0061] The beneficial effects of the technical solutions of the embodiments of the present invention are as follows:

[0062] A neural field-guided point cloud registration method for the technical solution of the embodiment of the present invention includes the following steps: Step 1, establish a neural network (PC2NF) for guiding point cloud registration using a neural field, with the input being the source point cloud and the output being the rotation angles θ of the x, y, and z axesx 、 θ y 、 θ z and displacement t x 、 t y 、 t z ; Step 2, represent the target point cloud in the form of a neural field using a small neural network; Step 3, set matrix orthogonality constraints for the registration neural network; Step 4, construct a loss function for the registration neural network. Introduce the neural field of the target point cloud constructed in Step 2 to construct the loss function; Step 5, train the registration neural network. Using the source point cloud data as the input, train the registration neural network to predict the rotation angles and displacements, obtain the rotation matrix using matrix orthogonality constraints, and under the guidance of the loss function, continuously approximate the optimal transformation matrix through backpropagation; Step 6, use the optimal transformation matrix to register the source point cloud and the target point cloud. The present invention introduces a neural field into point cloud registration, transforms the registration into an optimization process from point cloud to neural field, and realizes fast and accurate point cloud registration. The present invention is based on a neural field, uses the powerful capabilities of deep learning for modeling and optimization, and improves the accuracy and robustness of point cloud registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a flowchart of a neural field-guided point cloud registration method shown according to an exemplary embodiment;

[0064] Figure 2 is a schematic structural diagram of a neural field-guided point cloud registration device shown according to an exemplary embodiment;

[0065] Figure 3 is a PC2NF network architecture diagram shown according to an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0066] To more clearly illustrate the technical features of the solution of the present invention, the present invention will be described in detail below through specific embodiments and in conjunction with its accompanying drawings.

[0067] As Figure 1 shown, a neural field-guided point cloud registration method provided by an embodiment of the present invention includes the following steps:

[0068] Step 1, establish a neural network for guiding point cloud registration using a neural field. The input of the registration neural network is the source point cloud, and the output is the rotation angles θ x 、 θ y 、 θ z and displacement t x 、 t y 、 t z ;

[0069] Step 2, represent the target point cloud in the form of a neural field using a small neural network;

[0070] Step 3, set the matrix orthogonality constraint for the registered neural network;

[0071] Step 4, introduce the constructed target point cloud neural field to construct a loss function for the registered neural network;

[0072] Step 5, use the source point cloud data as input, train the registered neural network, predict the rotation angle and displacement, obtain the rotation matrix using the matrix orthogonality constraint, and continuously approximate the optimal transformation matrix through backpropagation under the guidance of the loss function;

[0073] Step 6, use the optimal transformation matrix to register the source point cloud and the target point cloud.

[0074] As a possible implementation of this embodiment, the establishment of the neural network for guiding point cloud registration using the neural field includes the following steps:

[0075] Use a multi-layer perceptron neural network to fit the signed distance field SDF:

[0076]

[0077] where f(x) is the shortest signed distance from the query point x to the surface S of the 3D shape O, used to indicate whether the query point x is inside or outside the shape O;

[0078] Implicitly represent the surface S of the 3D shape O as:

[0079]

[0080] where x represents a spatial point, represents the spatial dimension, f(x) represents the function where f(x)=0 on the point cloud surface, and S represents the distance from the spatial point to the surface of the three-dimensional shape;

[0081] Uniformly sample the space where the surface S of the 3D shape O is located and use it as training data;

[0082] Optimize the parameters θ of the multi-layer perceptron by minimizing the error Loss1 between the true distance and the predicted distance:

[0083]

[0084] where τ is the gradient similarity weight factor, d is the signed distance truth value, f θ (x i ) is the predicted signed distance, I0 is the zero level set of the signed distance field, is the gradient of the predicted signed distance of the query point on the zero level set I0, and n i is the normal truth value of the query point;

[0085] If there is a point cloud normal and τ = 1, the gradient of the query point on the SDF zero-level set I0 and the corresponding normal are used as additional supervision signals; otherwise, τ = 0.

[0086] As a possible implementation of this embodiment, the registered neural network consists of a small PointNet architecture. After performing the max pooling operation, a global feature vector of size 1024 is obtained and input into a decoder composed of multiple fully connected layers; the last fully connected layer outputs a vector containing 6 parameters: θ x , θ y , θ z、 t x , t y and t z , θ x , θ y and θ z、 represent the rotation angles around the x, y, and z axes, and t x , t y and t z represent the translations along the x, y, and z axes.

[0087] As a possible implementation of this embodiment, the setting of matrix orthogonality constraints for the registered neural network includes the following steps:

[0088] Predict the rotation angles around the x, y, and z axes (θ x , θ y , θ z );

[0089] Calculate the rotation matrix through R = R z (θ z ) × R y (θ y ) × R x (θ x ), where R x , R y , R z are the rotation matrices around the x, y, and z axes respectively.

[0090] As a possible implementation of this embodiment, the loss function is:

[0091]

[0092] where T(x) is the rotation and displacement transformation of point x, and f θ (T(x i )) is the signed distance value of the neural field at the query point T(x i ), is the gradient of the query point x after rotation and translation, and ni The normal vector of the query point.

[0093] As Figure 2 shown, an apparatus for point cloud registration guided by a neural field provided by an embodiment of the present invention includes:

[0094] A network establishment module, configured to establish a neural network for guiding point cloud registration by using a neural field. The input of the neural network for registration is the source point cloud, and the output is the rotation angles θ x 、θ y 、θ z and displacement t x 、t y 、t z ;

[0095] A form representation module, configured to represent the target point cloud in the form of a neural field by using a small neural network;

[0096] A constraint setting module, configured to perform matrix orthogonality constraint setting on the neural network for registration;

[0097] A loss function setting module, configured to introduce the constructed neural field of the target point cloud to construct a loss function for the neural network for registration;

[0098] A network training module, configured to use the source point cloud data as input to train the neural network for registration, predict the rotation angle and displacement, obtain the rotation matrix by using matrix orthogonality constraint, and continuously approximate the optimal transformation matrix through backpropagation under the guidance of the loss function;

[0099] A point cloud registration module, configured to perform registration of the source point cloud and the target point cloud by using the optimal transformation matrix.

[0100] As a possible implementation manner of this embodiment, the process of establishing the neural network for guiding point cloud registration by using a neural field is:

[0101] Using a multi-layer perceptron neural network to fit the signed distance field SDF:

[0102]

[0103] where f(x) is the shortest signed distance from the query point x to the surface S of the 3D shape O, and is used to represent whether the query point x is inside or outside the shape O;

[0104] Implicitly representing the surface S of the 3D shape O as:

[0105]

[0106] where x represents a spatial point, represents the spatial dimension, f(x) represents the function where f(x) = 0 on the point cloud surface, and S represents the distance from the spatial point to the three-dimensional shape surface;

[0107] The space where the surface S of the 3D shape O is located is uniformly sampled and used as training data;

[0108] The parameters θ of the multilayer perceptron are optimized by minimizing the error Loss1 between the true distance and the predicted distance:

[0109]

[0110] Among them, τ is the gradient similarity weight factor, d is the true value of the signed distance, and f θ (x i ) is the predicted signed distance, I0 is the zero level set of the signed distance field, is the gradient of the signed distance predicted by the query point on the zero level set I0, n i is the true value of the normal of the query point;

[0111] If there is a point cloud normal, τ = 1, then the gradient of the query point on the SDF zero level set I0 and the corresponding normal are used as additional supervision signals, otherwise τ = 0.

[0112] As a possible implementation of this embodiment, the registered neural network is composed of a small PointNet architecture. After performing the maximum pooling operation, a global feature vector of size 1024 is obtained and input into a decoder composed of multiple fully connected (FC) layers; the last FC layer outputs a vector containing 6 parameters: θ x ,θ y ,θ z、 t x ,t y and t z ,θ x ,θ y and θ z、 represents the rotation angle around the x, y and z axes, t x ,t y and t z Represents translation along the x, y, and z axes.

[0113] As a possible implementation of this embodiment, the specific process of setting the matrix orthogonalization constraint for the registered neural network is as follows:

[0114] Predict the rotation angles (θ) around the x, y, and z axes x ,θ y ,θ z );

[0115] Through R=R z (θz ) × R y (θ y ) × R x (θ x ) calculates the rotation matrix, R x , R y , R z are the rotation matrices around the x, y, and z axes respectively.

[0116] As a possible implementation of this embodiment, the loss function is:

[0117]

[0118] where T(x) is the rotation and displacement transformation of point x, and f θ (T(x i )) is the signed distance value of the neural field at the query point T(x i ), is the gradient after rotation and translation of the query point x, and n i is the normal of the query point.

[0119] A neural field is a field parameterized by a neural network. In visual computing, a neural field can be constructed by sampling points and their corresponding distances to the surface and then inputting them into a neural network. The present invention first uses a neural field to represent shapes, which is a learnable signed distance field. First, the design of PC2NF will be introduced in detail, and then the registration mechanism between the neural field and the source point cloud will be elaborated.

[0120] A. Neural field in implicit surface representation.

[0121] Compared with explicit representations (including meshes, voxels, point clouds), implicit surface representations offer many useful properties. Mainstream methods include occupancy fields and signed distance fields (SDF). The present invention adopts the SDF representation, expressed as where f(x) is the shortest signed distance from the query point x to the surface S of the 3D shape O, and the sign indicates whether x is inside or outside the shape O. Therefore, the surface S of the shape O can be implicitly represented as the zero level set of f: f(x) = 0.

[0122]

[0123] The present invention follows the paradigm of using a neural network to represent a continuous neural SDF. Different from DeepSDF that encodes shapes as latent codes z, the present invention uses a multi-layer perceptron (MLP) neural network to fit an accurate and continuous SDF. This network takes query points as inputs and predicts signed distances. To train the neural network to learn an accurate signed distance field, in addition to surface sampling points, uniform sampling of the space where they are located is also required as training data. The parameters θ of the MLP are optimized by minimizing the error between the true distance and the predicted distance. In addition, to improve accuracy, if point cloud normals exist, the gradient of the query points on the SDF zero-level set I0 and the corresponding normals are used as additional supervision signals, where τ = 1 if normals exist, otherwise τ = 0.

[0124]

[0125] The second term in this function encourages the gradient to be close to the provided normal N.

[0126] B. Registration in neural fields.

[0127] As Figure 3 shown, in the PC2NF network architecture of the present invention, the target point cloud P T is represented in the form of a neural field. The registration neural network consists of a small PointNet architecture. After performing a max-pooling operation, a global feature vector of size 1024 is obtained. Then, this feature is input into a decoder composed of multiple fully connected (FC) layers. The last FC layer outputs a vector containing 6 parameters: the first three parameters represent the rotation angles around the x, y, and z axes, and the last three parameters represent the translations along the x, y, and z axes.

[0128] C. Matrix orthogonality constraint.

[0129] Different from the method of directly predicting rotation matrices or quaternions, the present invention estimates the rotation matrix in two steps. First, the rotation angles around the x, y, and z axes (θ x , θ y , θ z ) are predicted. Then, through R = R z (θ z ) × R y (θ y ) × R x (θ x)Calculate the rotation matrix. This method has several advantages: 1) It avoids the problem of quaternion error accumulation that usually occurs after multiple transformation operations, resulting in inaccurate rotation. Usually, to alleviate this problem, once the error exceeds the set threshold, a regular normalization operation is performed; 2) It avoids the need for singular value decomposition (SVD) operations. The method of the present invention ensures orthogonality and det(R)=1, where det(R) is the determinant of matrix R. Therefore, compared with directly estimating the parameters of the rotation matrix, the method of the present invention avoids complex SVD operations and achieves better convergence during the training process.

[0130] D. Training loss.

[0131] In the neural signed distance field, for the input point set I, each point can obtain a signed distance value. If the point is on the surface, this value is zero. The goal of the loss function of the present invention is to guide the network to minimize the sum of the signed distances of all points. In addition, due to the existence of a gradient field in the neural distance field, a gradient constraint is introduced as a regularization term.

[0132]

[0133] The value of the gradient similarity weight factor τ is in the range of [0,1]. The first term in the loss function represents the absolute signed distance of all points, which encourages the network to obtain accurate rotation angles and translations so that all points are on the zero level set of the SDF. The second term in the loss function is the gradient regularization term.

[0134] The present invention designs a neural network PC2NF for point cloud registration guided by a neural field to simulate the registration behavior of the source point cloud in the neural field learned from the target point cloud. This network rotates and translates the source point cloud in the neural field to obtain the best position and orientation, at which time the signed distance of the point cloud should be the lowest and the gradient similarity should be the highest. The PC2NF network uses the neural field learned from the target point cloud for registration. The neural field is continuous to facilitate dynamic optimization of the transformation matrix, thereby improving the registration performance and accuracy.

[0135] Since it is difficult to ensure the orthogonality of the predicted rotation matrix, the present invention introduces a matrix orthogonality constraint optimization method, which avoids error accumulation and high computational cost caused by quaternion conversion or singular value decomposition (SVD) operations, and achieves better convergence of the neural network.

[0136] The present invention has high robustness to noisy and defective point clouds and achieves competitive results on noise-free point clouds. In addition, compared with feature learning-based methods, the method of the present invention does not require training on a large-scale dataset.

[0137] An electronic device provided by an embodiment of the present invention includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the device runs, the processor communicates with the memory through the bus. The processor executes the machine-readable instructions to perform the steps of the point cloud registration method guided by any of the above neural fields.

[0138] Specifically, the above-mentioned memory and processor can be general-purpose memory and processor, and no specific limitation is made here. When the processor runs the computer program stored in the memory, it can execute the above-mentioned point cloud registration method guided by the neural field.

[0139] Those skilled in the art can understand that the structure of the electronic device does not constitute a limitation on the electronic device, and it may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange different components.

[0140] In some embodiments, the electronic device may further include a touch screen, which can be used to display a graphical user interface (for example, the startup interface of an application) and receive user operations on the graphical user interface (for example, the startup operation for an application). Specifically, the touch screen may include a display panel and a touch panel. The display panel can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), etc. The touch panel can collect contact or non-contact operations of the user on or near it and generate preset operation instructions. For example, the user uses a finger, a stylus, or any suitable object or attachment to operate on the touch panel or near the touch panel. In addition, the touch panel can include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the touch orientation and posture of the user, and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into information that the processor can process, then sends it to the processor, and can receive the command sent by the processor and execute it. In addition, various types such as resistive, capacitive, infrared, and surface acoustic wave can be used to implement the touch panel, and any technology developed in the future can also be used to implement the touch panel. Further, the touch panel can cover the display panel. The user can operate on or near the touch panel covering the display panel according to the graphical user interface displayed on the display panel. After the touch panel detects the operation on or near it, it is transmitted to the processor to determine the user input. Subsequently, the processor provides a corresponding visual output on the display panel in response to the user input. In addition, the touch panel and the display panel can be implemented as two independent components or integrated.

[0141] Corresponding to the startup method of the above application program, an embodiment of the present invention further provides a storage medium, on which a computer program is stored, and when the computer program is run by a processor, it executes the steps of any of the above-mentioned neural field-guided point cloud registration methods.

[0142] The startup device of the application program provided by the embodiments of the present application can be specific hardware on the device or software or firmware installed on the device. The implementation principle and the technical effects generated by the device provided by the embodiments of the present application are the same as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference can be made to the corresponding content in the foregoing method embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices, and units described above can all refer to the corresponding processes in the above method embodiments, and will not be repeated here.

[0143] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] In the embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of modules is only a logical function division, and there can be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces. The indirect coupling or communication connection of devices or modules can be in electrical, mechanical, or other forms.

[0145] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they can be located in one place, or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, the functional modules in the embodiments provided by the present application can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module.

[0147] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices generate means for implementing the functions specified in one or more of the flows Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0148] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means for implementing the functions specified in one or more of the flows Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more of the flows Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that it is still possible to modify the specific embodiments of the present invention or make equivalent substitutions without departing from the spirit and scope of the present invention. Any modification or equivalent substitution that does not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A method for neural field-guided point cloud registration, characterized in that, It includes the following steps: Step 1, establish a neural network for guiding point cloud registration using a neural field. The input of the registration neural network is the source point cloud, and the output is the rotation angles of the x, y, and z axes θ x 、 θ y 、 θ z and the displacement t x 、 t y 、 t z ; Step 2, using a small neural network to represent the target point cloud in the form of a neural field; Step 3, setting matrix orthogonality constraints for the registered neural network; Step 4, introducing the constructed neural field of the target point cloud to construct a loss function for the registered neural network; Step 5, using the source point cloud data as input, training the registered neural network, predicting the rotation angle and displacement, obtaining the rotation matrix using matrix orthogonality constraints, and under the guidance of the loss function, continuously approaching the optimal transformation matrix through backpropagation; Step 6, using the optimal transformation matrix to register the source point cloud and the target point cloud; The process of establishing a neural network for guiding point cloud registration using a neural field includes the following steps: Using a multi-layer perceptron neural network to fit the signed distance field SDF: Among them, ( x ) is the shortest signed distance from the query point x to 3D the surface of the O shape S , used to indicate whether the query point x is inside or outside the O shape; The surface S of a 3D shape O is implicitly represented as: Among them, x represents a spatial point, represents the spatial dimension, represents on the point cloud surface function of = 0, represents the distance from a spatial point to the surface of a three-dimensional shape; For a 3D shape O the space where the surface S lies is evenly sampled and used as training data; By minimizing the error between the true distance and the predicted distance to optimize the parameters of the multi-layer perceptron θ : Among them, is the gradient similarity weight factor, d is the signed distance ground truth, is the predicted signed distance, is the zero level set of the signed distance field, is the zero level set is the gradient of the predicted signed distance at the query point on is the ground truth normal vector of the query point; If there is point cloud normal and τ = 1, then use the gradient of the query point on the SDF zero level set and the corresponding normal as additional supervision signals, otherwise τ = 0. ​ 2. The neural field-guided point cloud registration method according to claim 1, wherein The registered neural network consists of a small PointNet architecture. After performing the maximum pooling operation, a global feature vector of size 1024 is obtained and input into a decoder composed of multiple fully connected layers; the last fully connected layer outputs a vector containing 6 parameters: θ x , θ y , θ z、 t x , t y and t z , θ x , θ y and θ z、 represents the rotation angle around the x, y and z axes, t x , t y and t z Represents translation along the x, y, and z axes.

3. The neural field-guided point cloud registration method according to claim 2, wherein The setting of matrix orthogonality constraints for the registered neural network includes the following steps: Predict the rotation angles around the x, y, and z axes ( θ x , θ y , θ z ); By R = R z ( θ z ) ×R y ( θ y ) ×R x ( θ x ) calculate the rotation matrix, R x ,R y ,R z which are the rotation matrices about the x, y, and z axes respectively.

4. The method for registering point clouds guided by a neural field according to claim 3, wherein, The loss function is: Among them, is the rotation and displacement transformation of the point x , is the signed distance value of the neural field at the query point , is the gradient of the query point x after rotation and translation, is the normal of the query point.

5. A neural field-guided point cloud registration device, characterized in that, It includes: A network establishment module, which is used to establish a neural network for guiding point cloud registration by a neural field. The input of the registration neural network is the source point cloud, and the output is the rotation angles of the x, y, and z axes θ x 、 θ y 、 θ z and displacement t x 、 t y 、 t z ; A form representation module for using a small neural network to represent the target point cloud in the form of a neural field; A constraint setting module for setting matrix orthogonality constraints for the registered neural network; A loss function setting module for introducing the constructed neural field of the target point cloud to construct a loss function for the registered neural network; A network training module for using the source point cloud data as input, training the registered neural network, predicting the rotation angle and displacement, obtaining the rotation matrix using matrix orthogonality constraints, and under the guidance of the loss function, continuously approaching the optimal transformation matrix through backpropagation; A point cloud registration module for registering the source point cloud and the target point cloud using the optimal transformation matrix; The process of establishing a neural network for guiding point cloud registration using a neural field is: Using a multi-layer perceptron neural network to fit the signed distance field SDF: Among them, ( x ) is the shortest signed distance from the query point x to 3D the surface of the O shape S for indicating whether the query point x is inside or outside the O shape; The surface S of a 3D shape O is implicitly represented as: Among them, x represents a spatial point, represents the spatial dimension, represents on the point cloud surface function of = 0, represents the distance from a spatial point to the surface of a three-dimensional shape; Uniformly sample the space where the surface S of the 3D shape O is located and use it as training data; By minimizing the error between the true distance and the predicted distance to optimize the parameters of the multi-layer perceptron θ : Among them, is the gradient similarity weight factor, d is the signed distance ground truth, is the predicted signed distance, is the zero level set of the signed distance field, is the zero level set is the gradient of the predicted signed distance at the query point on is the ground truth normal of the query point; If there is a point cloud normal and τ = 1, then the gradient of the query point on the SDF zero level set and the corresponding normal are used as additional supervision signals; otherwise, τ = 0. ​ 6. The neural field-guided point cloud registration device according to claim 5, wherein The loss function is: Among them, is the rotation and displacement transformation of the point x , is the signed distance value of the neural field at the query point , is the gradient after rotation and translation of the query point x, is the normal of the query point.

7. An electronic device, characterized in that, It includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of the neural field-guided point cloud registration method as described in any one of claims 1-4.

8. A storage medium, characterized in that, A computer program is stored on the storage medium, and when the computer program is run by the processor, it performs the steps of the neural field-guided point cloud registration method as described in any one of claims 1-4.

Images