A method, system, device and medium for constructing a local coordinate system of a point cloud
Through the EE-Net network model, a robust and repeatable local reference coordinate system was constructed, which solved the problem of poor local coordinate system construction in the existing technology and improved the point cloud Data processing efficiency and robustness.
Patent Information
- Application Number
- CN202210338386.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-01
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2042-04-01
AI Technical Summary
The existing three-dimensional point cloud local reference coordinate system construction method has symbol fuzzy problems and limited robustness to high noise and grid resolution changes, resulting in poor local coordinate system construction and cannot meet the needs.
By creating the 3D features of the point cloud local block and inputting it into the EE-Net network model, we obtain the weight coefficients of each point pair of the predicted point cloud local block to build the local reference coordinate system, and finally build the point cloud local reference based on these weight coefficients Coordinate system.
A more robust and repeatable local reference coordinate system is realized, which can keep the rigid body motion of the local block of the point cloud in the processed three-dimensional point cloud data unchanged, improving the efficiency and resource utilization of the system.
Smart Images

Figure CN114648582B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer data processing, and in particular to a method, system, device and medium for constructing a point cloud local coordinate system. Background Art
[0002] Object recognition is an active research area in computer vision technology, and this problem has been extensively studied in recent decades. There are many applications, including navigation, surveillance, automation, biometrics, surgery, and education. The goal of object recognition is to correctly identify objects in a scene and recover their pose (i.e., position and orientation). In addition to object recognition in 2D images, 3D object recognition has been widely studied in the past two decades due to the availability of low-cost LiDAR and high-speed computing devices. Existing 3D object recognition algorithms can be roughly divided into two categories, namely global feature-based algorithms and local feature-based algorithms. Global feature-based algorithms construct a set of features that encode the geometric properties of the entire 3D object. Examples of these algorithms include geometric 3D moments, shape distributions, and covariance functions. However, these algorithms require a complete 3D model and are therefore very sensitive to occlusion and clutter. Therefore, they are more suitable for 3D shape retrieval rather than object recognition and surface registration. In contrast, local feature-based algorithms define a set of features that encode the local neighborhood features of feature points and achieve point-to-point correspondence between surfaces by computing and comparing local feature descriptors of key points. Algorithms based on local features are more robust to occlusion and clutter. Therefore, local methods are more effective and robust for matching incomplete 3D surfaces, especially in object recognition and surface registration.
[0003] LRF (local reference coordinate system) is independent of the global coordinate system. It is a canonical coordinate system built on a 3D local surface. It is a useful geometric clue for 3D point clouds and has three orthogonal axes. However, existing LRF methods, based on covariance analysis (CA) or point space distribution (PSD), often have sign ambiguity problems in constructing local coordinate systems, or have limited robustness to high noise and changes in grid resolution, which makes the constructed local coordinate systems less effective and unable to meet the needs. Summary of the invention
[0004] In view of the above problems, the present invention provides a method, system, device and medium for constructing a local coordinate system of a point cloud, aiming to solve at least one of the above problems.
[0005] A method for constructing a point cloud local coordinate system comprises the following steps:
[0006] Make 3D features of local blocks of point cloud;
[0007] Input the 3D features into the EE-Net network model to obtain the weight coefficient of each point pair of the predicted point cloud local block to construct the local reference coordinate system;
[0008] According to each point pair of the predicted point cloud local block, a weight coefficient of the local reference coordinate system is constructed to obtain the predicted weight result;
[0009] According to the results of the predicted weights, a local reference coordinate system of the point cloud is constructed.
[0010] A system for constructing a local coordinate system of a point cloud, comprising the following computer program modules interconnected and communicating with each other:
[0011] 3D feature production module, used to produce 3D features of local blocks of point clouds;
[0012] The weight coefficient acquisition module is used to input the 3D features into the EE-Net network model to obtain the weight coefficient of each point pair of the predicted point cloud local block to construct the local reference coordinate system;
[0013] The weight result acquisition module is used to construct the weight coefficient of the local reference coordinate system according to each point pair of the predicted point cloud local block to obtain the predicted weight result;
[0014] The coordinate system construction module is used to construct the local reference coordinate system of the point cloud according to the results of the prediction weights.
[0015] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a method for constructing a point cloud local coordinate system as described in any one of the first aspects is implemented.
[0016] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, a method for constructing a point cloud local coordinate system described in any one of the first aspects is performed.
[0017] The beneficial effects of the present invention are:
[0018] The method, system, device and medium for constructing a local coordinate system of a point cloud provided by the present invention directly use the three-dimensional data obtained from the processed three-dimensional point cloud as the input of the network. At the same time, by designing a network structure, it can also ensure that the rigid body motion of the local block of the point cloud remains unchanged, thereby designing a more robust and repeatable local reference coordinate system, which is convenient and efficient and occupies less system resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.
[0020] Figure 1 It is a schematic diagram of the process of constructing a point cloud local coordinate system provided by an embodiment of the present invention;
[0021] Figure 2 is a schematic diagram of the EE-Net network model structure provided by an embodiment of the present invention;
[0022] Figure 3 is a schematic diagram of the structure of a twin neural network provided by an embodiment of the present invention;
[0023] Figure 4 Schematic diagram of the T-Net model provided by an embodiment of the present invention;
[0024] Figure 5 It is a schematic diagram of the SE module of the attention mechanism provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0025] The specific embodiments of the present invention are further described below in conjunction with the accompanying drawings. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0026] See also Figure 1 , a flow chart of a method for constructing a point cloud local coordinate system provided by an embodiment of the present invention, the method comprising the following steps:
[0027] S101, creating 3D features of local blocks of the point cloud.
[0028] This application uses the following two classic and authoritative databases for experiments:
[0029] BR database: The model is the complete point cloud data, and the scene is the point cloud data after the model is rotated and translated;
[0030] UWAOR database, the model is a complete point cloud data, the scene is a mixture of multiple models rotated, and has characteristics such as occlusion and missing.
[0031] The specific steps for making 3D features of local blocks of point cloud are as follows: First, generate a pair of model scene local block pairs, and obtain cloud model scene local block pairs based on the key feature points corresponding to the model and the scene. Obtain a local block in the model with the key feature point as the center and the size of 0.15 times the radius of the entire model. Then obtain the rotation matrix, and match the corresponding key feature points in the specific scene file according to the rotation matrix. If the same corresponding feature point exists in the scene file, a point cloud local block with the key point as the center and the size of 0.15 times the radius of the entire model is generated. The above method can generate a pair of model scene local blocks.
[0032] Generate 3D features based on local blocks of point cloud. Specifically, let the key feature point of the local block of point cloud be p, and the neighboring point of a key point be q i , for each neighboring point q i Calculate the distance from the key feature point p to the neighboring point q i The direction vector is then normalized to obtain the required 3D features. This method can obtain distinguishable and lossless data features.
[0033] S102, inputting the 3D features into the EE-Net network model to obtain the weight coefficient of each point pair of the predicted point cloud local block to construct the local reference coordinate system.
[0034] The generated 3D features are used as the input of the EE-Net network model, and the output is the weight of each point pair in the predicted point cloud local block to construct the local reference coordinate system. The specific structure of the EE-Net network model is realized by combining the Siamese-Net, T-Net, and SE attention mechanism. The EE-Net network model structure is as follows Figure 2 shown.
[0035] Twin network: The twin neural network is a coupled architecture based on two artificial neural networks. It consists of two sub-networks. Each sub-network receives an input, maps it to a high-dimensional feature space, and outputs the corresponding representation. By calculating the distance between the two representations, such as the Euclidean distance, the similarity of the two inputs can be compared. In this paper, the twin network structure is combined with the chamfer loss to compare the similarity of two local blocks of point clouds in the new local reference coordinate system, thereby iteratively optimizing the network parameters. The twin neural network structure is as follows: Figure 3 As shown;
[0036] T-Net network: It can make the learned features have the characteristics of rigid body motion invariance. The specific implementation principle is as follows: Processing the principle of rotation and translation transformation of point cloud data. After the point cloud data itself is rotated and translated, a new set of point cloud data will be obtained. That is, the same point cloud object will have different data representations after rotation and translation. The rigid body motion change characteristics of this point cloud data can be processed in the following way. First, learn the posture information of the point cloud itself to learn a DxD rotation matrix (D represents the feature dimension, here it is 3) that is most conducive to the network classification or segmentation. The principle is to adjust the transformation matrix by controlling the final loss function loss. The network structure does not care about what transformation is actually done in the end, as long as it is conducive to the final result. Intuitively, it is to learn a rotation matrix to rotate the input data to an angle that is more conducive to extracting target features, such as turning the object to the front.
[0037] Introduce the attention mechanism SE module:
[0038] See also Figure 5 Schematic diagram of the attention mechanism SE module shown in the embodiment;
[0039] The SE module first compresses the feature map obtained by convolution to obtain the global features at the channel level, then excites the global features to learn the relationship between channels and obtain the weights of different channels, and finally multiplies the original feature map to obtain the final features. In essence, it hopes that the model can automatically learn the importance of different channel features by paying attention to the relationship between channels. This attention mechanism allows the model to pay more attention to the channel features with the most information and suppress those unimportant features.
[0040] The compression operation refers to the fact that since the convolution only operates in a local space, it is difficult to obtain enough information to extract the relationship between channels. This is more serious for the early layers in the network because the receptive field is relatively small. The compression operation encodes the entire spatial feature on a channel into a global feature, which is achieved by global flat pooling.
[0041] The excitation operation means that after the compression operation obtains the global description features, another operation is needed to obtain the relationship between channels. This operation needs to meet two principles: first, it needs to be able to learn the nonlinear relationship between each channel; second, the learned relationship is not mutually exclusive, because multiple features are allowed here, rather than one-hot encoding. In order to reduce the complexity and generalization ability of the model, a structure containing two fully connected layers is used here, where the first fully connected layer plays a role in dimensionality reduction, and then the ReLU activation function is used for processing, and the last fully connected layer restores the original dimension.
[0042] Finally, the learned activation values of each channel are multiplied by the original features through proportional operations: the weight coefficients of each channel are obtained, so that the model can be more discriminative of the features of each channel.
[0043] S103. Construct a weight coefficient of a local reference coordinate system according to each point pair of the predicted point cloud local block to obtain a predicted weight result.
[0044] The EE-Net network model structure is as follows Figure 2 As shown. The input of the network model is the 3D feature of the local block of the point cloud, and the output is the weight of each point in the local block for constructing the local reference coordinate system. In the first hidden layer, the dimension is increased by first passing the input through the T-Net network to learn a 3x3 rotation matrix that aligns the point cloud surface, and then multiplying the input data with the rotation matrix to obtain features with rotation invariance. A one-dimensional convolution is then used to increase the dimension of a 2-dimensional feature to 64 dimensions to obtain richer feature information. The obtained features are then normalized and the results are nonlinearly mapped.
[0045] The second hidden layer is upgraded in dimension and the SE attention mechanism is introduced. Specifically, a one-dimensional convolution is used to upgrade the 64-dimensional features to 128 dimensions to obtain richer feature information. The obtained features are then normalized and the results are nonlinearly mapped. The SE attention mechanism is then used to allow the model to pay more attention to the channel features with the most information and suppress those unimportant channel features. The 128-dimensional features are then upgraded to 1024-dimensional features, normalized, nonlinearly mapped, and processed with the SE attention mechanism. A pooling layer is then used to obtain 1024 global features of the entire local block.
[0046] A concatenation operation is performed to concatenate the 1024 global features of the local block and the 64-dimensional features in the first hidden layer to obtain a matrix of size N×1088.
[0047] Then, the third hidden layer is used to reduce the dimension to obtain the prediction weight. Specifically, the 1088-dimensional feature is first compressed to 518 dimensions through a one-dimensional convolution, and then the obtained feature is normalized, and the normalized result is nonlinearly mapped. This similar operation is repeated until the dimension is reduced to a 1-dimensional feature, that is, an Nx1-dimensional feature is obtained.
[0048] Finally, the Hausdorff distance loss is used to quantify the quality of the current network. Then, the parameters are updated through back propagation, the parameters of the network model are automatically optimized, and repeated iterations are performed to combine the weight coefficients to finally obtain the predicted weight result.
[0049] The parameters of the network model are set as follows: the network is trained using a batch size of 256 local surfaces, and the adaptive learning rate adam optimizer is used, with an initial learning rate of 10 -4 , it decays by 5% after each epoch, and the epoch size is set to 20.
[0050] S104: Construct a local reference coordinate system of the point cloud according to the result of the prediction weight.
[0051] The steps for calculating the z-axis are as follows: let the key feature point of the local block of the point cloud be p, and calculate the tangent plane normal Z(p) of the key feature point p of the local block as the z-axis. Since the normal vector has sign ambiguity, its sign ambiguity can be removed by the following formula.
[0052]
[0053] where q i represents the i-th neighbor of key point p
[0054] The x-axis calculation steps are as follows: the 3D features produced in step S101 are used as the input of the EE-Net network model. In this model, the same architecture and basic weights are shared by the twin network, and the chamfer distance is used as the loss function. The training prediction obtains each neighboring point q i The weight w for constructing the x-axis i Then the x-axis is constructed by weighted sum. The specific steps are as follows: First, calculate the neighboring point q i The projection vector projected onto the tangent plane s of the z-axis is given by the following formula:
[0055] v i =Pq i -(Pq i ·Z(p))·Z(p)
[0056] In the formula, Z(p) represents the unit vector of the z-axis, and then all projection vectors are integrated in the form of weighted vector sum, where the weight is the output of the EE-Net network model. The formula is as follows:
[0057]
[0058] Get the x-axis X(p) of the local coordinate system; the y-axis is the cross product of the x-axis and the z-axis
[0059] The embodiment of the present invention directly uses the three-dimensional data obtained from the processed three-dimensional point cloud as the input of the network. At the same time, by designing a network structure, it can also ensure that the rigid body motion of the local block of the point cloud remains unchanged, thereby designing a more robust and repeatable local reference coordinate system.
[0060] A system for constructing a local coordinate system of a point cloud, the system comprising the following computer program modules interconnected and communicating with each other:
[0061] 3D feature production module, used to produce 3D features of local blocks of point clouds;
[0062] The weight coefficient acquisition module is used to input the 3D features into the EE-Net network model to obtain the weight coefficient of each point pair of the predicted point cloud local block to construct the local reference coordinate system;
[0063] The weight result acquisition module is used to construct the weight coefficient of the local reference coordinate system according to each point pair of the predicted point cloud local block to obtain the predicted weight result;
[0064] The coordinate system construction module is used to construct the local reference coordinate system of the point cloud according to the results of the prediction weights.
[0065] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a method for constructing a point cloud local coordinate system as described in any of the above embodiments is implemented.
[0066] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, a method for constructing a point cloud local coordinate system as described in any of the above embodiments is performed.
[0067] The method, system, device and medium for constructing a local coordinate system of a point cloud provided by the present invention directly use the three-dimensional data obtained from the processed three-dimensional point cloud as the input of the network. At the same time, by designing a network structure, it can also ensure that the rigid body motion of the local block of the point cloud remains unchanged, thereby designing a more robust and repeatable local reference coordinate system, which is convenient and efficient and occupies less system resources.
[0068] It should be particularly pointed out that in other embodiments of the present invention, within the scope of the above description, different steps, parameters, modules and models can be specifically selected to achieve the technical effects of the present invention, so they will not be listed one by one.
[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.
Claims
1. A method for constructing a local coordinate system of a point cloud, characterized in that: include: Make 3D features of local blocks of point cloud; Input the 3D features into the EE-Net network model to obtain the weight coefficient of each point pair of the predicted point cloud local block to construct the local reference coordinate system; According to each point pair of the predicted point cloud local block, a weight coefficient of the local reference coordinate system is constructed to obtain the predicted weight result; According to the results of the predicted weights, a local reference coordinate system of the point cloud is constructed; The EE-Net network model is realized by combining the Siamese-Net network, the T-Net network, and the SE attention mechanism; The step of inputting the 3D features into the EE-Net network model to obtain the weight coefficients of each point pair of the predicted point cloud local block to construct the local reference coordinate system includes: The 3D features are received through the twin network, and the similarities of the two point cloud local blocks in the new local reference coordinate system are compared, so as to iteratively optimize the network parameters; Then, the T-Net network learns the pose information of the point cloud itself to learn a DxD rotation matrix that is beneficial for the network to classify or segment; Finally, the SE attention mechanism is introduced to suppress unimportant features in the rotation matrix, and then the weight coefficient of each point for constructing the LRF local coordinate system is obtained through learning through the designed network model; The step of constructing a weight coefficient of a local reference coordinate system based on each point pair of the predicted point cloud local block to obtain a result of the predicted weight includes: The local block 3D features of the point cloud are input into the EE-Net network model, and a rotation matrix is obtained through T-net network learning. In the first hidden layer, a one-dimensional convolution is used to increase the 2-dimensional features of the matrix to 64 dimensions. In the second hidden layer, a one-dimensional convolution is used to upgrade the 64-dimensional features to 128 dimensions to obtain feature information, normalize the obtained feature information, and then perform nonlinear mapping on the results. Then, the attention mechanism is used to suppress unimportant channel features. Then, the 128-dimensional features are upgraded to 1024-dimensional features, and normalization, nonlinear mapping and attention mechanism processing are continued. Finally, a pooling layer is used to obtain 1024 global features of the entire local block. Perform a concatenation operation to concatenate the 1024 global features of the local block and the 64-dimensional features in the first hidden layer to obtain a size of N 1088 matrix; Then, the third hidden layer is used to reduce the dimension to obtain the prediction weight. Specifically, the 1088-dimensional feature is first compressed to 518 dimensions through a one-dimensional convolution, and then the obtained feature is normalized, and the normalized result is nonlinearly mapped. This similar operation is repeated until the dimension is reduced to a 1-dimensional feature, that is, an Nx1-dimensional feature is obtained. Finally, the Hausdorff distance loss is used to quantify the quality of the current network, and the parameters are updated through back propagation. The parameters of the network model are automatically optimized, and repeated iterations are performed to finally obtain the result of the predicted weight.
2. A method for constructing a point cloud local coordinate system according to claim 1, characterized in that: The step of making the 3D features of the local block of the point cloud includes: A pair of model scene local blocks is generated, and the 3D features of the point cloud local blocks are obtained according to the model scene local blocks.
3. A method for constructing a point cloud local coordinate system according to claim 2, characterized in that: The generating a pair of model scene local blocks comprises: First, based on the key feature points corresponding to the model and the scene, a local block centered on the key feature point and 0.15 times the radius of the entire model is obtained in the model; Then obtain the rotation matrix, and match the corresponding key feature points in the specific scene file according to the rotation matrix. If the same corresponding feature point exists in the scene file, a point cloud local block with the key point as the center and a size of 0.15 times the radius of the entire model is generated.
4. A method for constructing a point cloud local coordinate system according to claim 3, characterized in that: The step of obtaining the 3D features of the local block of the point cloud according to the local block of the model scene includes: Assume that the key feature point of the local block of the point cloud is p, and the neighboring point of a key point is q i , for each neighboring point q i Calculate the distance from the key feature point p to the neighboring point q i The direction vector is then normalized to obtain the 3D feature.
5. A system for constructing a local coordinate system of a point cloud, characterized in that: The method is used to implement a method for constructing a point cloud local coordinate system as claimed in any one of claims 1 to 4, comprising: 3D feature production module, used to produce 3D features of local blocks of point clouds; The weight coefficient acquisition module is used to input the 3D features into the EE-Net network model to obtain the weight coefficient of each point pair of the predicted point cloud local block to construct the local reference coordinate system; The weight result acquisition module is used to construct the weight coefficient of the local reference coordinate system according to each point pair of the predicted point cloud local block to obtain the predicted weight result; The coordinate system construction module is used to construct the local reference coordinate system of the point cloud according to the results of the prediction weights.
6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, a method for constructing a point cloud local coordinate system as described in any one of claims 1 to 4 is implemented.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a method for constructing a point cloud local coordinate system is performed as described in any one of claims 1 to 4.
Citation Information
Patent Citations
3D feature extraction method and system based on TOLDI descriptor and medium
CN112001261A
Dense correspondence prediction method based on non-rigid point cloud
CN112750198A
3D object local surface description method and device adopting auxiliary shaft and storage medium
CN113033270A
Three-dimensional object point cloud classification method based on attention mechanism
CN113569979A
Point cloud registration optimization method based on normal distribution transformation hypothesis verification
CN113920180A
