A pig body point cloud registration method based on overlapping area prediction

Abstract

The application discloses a pig body point cloud registration method based on an overlapping area prediction, and steps are as follows: obtaining and preprocessing an original pig body point cloud; constructing a pig body point cloud registration network; inputting the preprocessed original pig body point cloud into the pig body point cloud registration network, predicting the probability of an overlapping area and extracting point features; establishing a point pair relationship based on the predicted overlapping area and the point features, and calculating a position transformation relationship to complete registration. The application uses a neural network to estimate the overlapping part of the pig body point cloud and extract point features, calculates the position transformation relationship based on the output data, obtains a complete surface smooth pig body point cloud, and improves the speed and precision of the pig body multi-view point cloud registration.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-view three-dimensional point cloud processing, and more particularly to a pig body point cloud registration method based on overlapping area prediction. BACKGROUND

[0002] Point clouds collected from different perspectives are in different coordinate systems, and point cloud registration aims to transform point cloud data obtained from different perspective coordinate systems to a unified coordinate system. Two-view point cloud registration aims to register two sets of point clouds with certain overlapping areas to a coordinate system. Accurate measurement of body size requires reconstructing high-quality point clouds. High-quality point clouds require high-precision multi-view point cloud registration, and the surface should be smooth with fewer noise points. If the point cloud registration accuracy is not high, and there are many noise points on the overlapping surface of multiple frames of point clouds, the surface is not smooth, and the error of the calculated body size is large. In recent years, with the rapid development of information technology, machine learning, computer vision, digital image processing, cloud computing, big data, and electronic information have been widely applied in industry, agriculture, manufacturing, medical care, and education. In recent years, point cloud registration algorithms based on deep learning have been proposed, and their performance and speed have exceeded those of traditional ICP algorithms. Currently, pig body point cloud registration mostly uses static calibration methods, which have the disadvantage of weak anti-interference ability to the collection environment. SUMMARY

[0003] To solve the problems of weak anti-interference ability to the collection environment and tedious data processing in multi-view three-dimensional pig body point cloud registration, the present application provides a pig body point cloud registration method based on overlapping area prediction. This method effectively improves the accuracy of the registration process of the multi-view pig body point cloud registration network and improves the anti-interference ability of the pig body point cloud registration network to the environment.

[0004] To achieve the above-mentioned purposes of the present application, the technical solutions adopted are as follows:

[0005] A pig body point cloud registration method based on overlapping area prediction, the method comprising the following steps: obtaining and preprocessing the original pig body point cloud; constructing a pig body point cloud registration network; inputting the preprocessed original pig body point cloud into the pig body point cloud registration network to predict the probability of the overlapping area and extract the features of the points; establishing a point pair relationship based on the predicted overlapping area and the features of the points, and calculating the position transformation relationship to complete the registration.

[0006] Preferably, the preprocessing process of the original pig body point cloud is as follows: first, perform voxel down-sampling on the input two frames of original pig body point cloud; obtain two frames of down-sampled pig body point cloud with uniform surface distribution, and find corresponding points in the two frames of down-sampled pig body point cloud to form point pairs, which are used as data for calculating the error of the pig body point cloud registration network.

[0007] Preferably, the pig body point cloud registration network is a two-stream encoder-decoder network, comprising an encoder composed of a feature extraction module ResKPConv, a cross-overlap region attention module, and a decoder composed of a one-dimensional convolutional layer Conv1D. The encoder inputs the preprocessed original pig body point cloud for encoding and then outputs feature information. The cross-overlap region attention module inputs the two feature information outputs from the encoder to find the overlapping region between the two feature information and outputs the corresponding feature information. The decoder decodes the feature information output by the cross-overlap region attention module to obtain the final feature information.

[0008] Preferably, the encoder includes several feature extraction modules ResKPConv, and the feature extraction module ResKPConv includes a kernel convolution submodule KPConv. The specific working process of the kernel convolution submodule KPConv is as follows: the kernel convolution submodule KPConv performs convolution on the input data, adds the result of the convolution with the input data and aggregates them, and then outputs feature information through an activation function.

[0009] Preferably, the cross-overlapping region attention module includes a graph convolution submodule and a cross-attention submodule; the graph convolution submodule enhances the neighborhood information of each point in the feature information output by the encoder before it is input into the cross-attention submodule; the cross-attention submodule receives the feature information output by the two graph convolution submodules, uses it to find the overlapping region between the feature information output by the two graph convolution submodules, and outputs the corresponding feature information; then the graph convolution submodule further enhances the neighborhood information of each point in the feature information output by the cross-attention submodule.

[0010] Preferably, the graph convolution submodule includes several graph convolution layers, and the specific calculation formula for the graph convolution submodule is as follows:

[0011] y GNN =j β (cat[ (0) y, (1) y, (2) y])

[0012] Among them, y GNN To enhance the feature information after the domain information of each point; j β This is a sub-component in the graph convolution submodule used to enhance the generalization of the pig body point cloud registration network, including linear layers, normalization, and the LeakyReLU activation function; (0) y represents the feature information of the input graph convolutional submodule; (1) y is (0) y is the feature information output by the first graph convolutional layer; (2) y is (1)y is the feature information output by the second layer of graph convolutional layers; cat[] represents the aggregation and splicing of the feature information of the input graph convolutional submodules.

[0013] Preferably, the specific calculation formula of the feature information output by the first layer of graph convolutional layers and the feature information output by the second layer of graph convolutional layers is as follows:

[0014] (k+1) y i = maxh β (cat[ (k) y j - (k) y i , (k) y i ])

[0015] wherein (k) y i and( k) y j are the feature information of points i and j in the feature information of the input graph convolutional layer, points i and j have a neighboring relationship, (k+1) y i is the feature information of point i in the feature information output by the graph convolutional layer; k is the layer number of the passed graph convolutional layer, h β is a subcomponent in the graph convolutional submodule, used to enhance the generalization of the pig body point cloud registration network, and contains a linear layer and a LeakyReLU activation function, max() represents a maximum value function, and cat[] represents the aggregation and splicing of the feature information of the input graph convolutional layer.

[0016] Preferably, the cross-attention submodule comprises four parallel multi-attention subcomponents and an MLP subcomponent; the four parallel multi-attention subcomponents extract similar parts in the feature information of the two input four parallel multi-attention subcomponents as similar encodings, and then learn and perform nonlinear mapping through an MLP subcomponent full connection network to output the feature information.

[0017] Preferably, the decoder decodes the feature information output by the cross-overlapping region attention module, outputs the feature descriptor of each point, and refines the probability of the overlapping region of each point in the point cloud after downsampling and the topological feature encoding.

[0018] Preferably, the registration process is as follows: selecting points whose first dimension value of the neighborhood feature information decoded by the decoder is greater than a threshold value, establishing a point pair relationship based on the predicted region and the point feature information of the selected points, estimating the position transformation relationship, and completing the registration; wherein the first dimension is the probability of the overlapping region, and the threshold value is 80%-90%.

[0019] The beneficial effects of the present application are as follows:

[0020] This invention utilizes a neural network to prioritize estimating overlapping portions of pig body point clouds and extracting point features. Based on the network output data, it can calculate positional transformation relationships to obtain a complete, smooth pig body point cloud. This invention effectively solves the accuracy problem in the registration process of multi-view pig body point cloud registration networks and improves the network's resistance to environmental interference. Attached Figure Description

[0021] Figure 1 This is a flowchart of a pig point cloud registration method based on overlapping region prediction as described in Embodiment 1 of the present invention.

[0022] Figure 2 This is a schematic diagram of point cloud acquisition at different locations as described in Embodiment 1 of the present invention.

[0023] Figure 3 This is an overall flowchart of the point cloud registration network model described in Embodiment 1 of the present invention.

[0024] Figure 4 This is an overall effect diagram of the pig body registration completed as described in Embodiment 1 of the present invention.

[0025] Figure 5 This is a cross-sectional view of the pig body after registration as described in Embodiment 1 of the present invention. Detailed Implementation

[0026] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0027] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0028] Example 1

[0029] like Figure 1 As shown, a pig point cloud registration method based on overlapping region prediction is described, comprising the following steps:

[0030] S1: Acquire and preprocess the raw pig body point cloud;

[0031] S2: constructing a pig body point cloud registration network;

[0032] S3: inputting the preprocessed original pig body point cloud into the pig body point cloud registration network to predict the probability of the overlapping region and extract the feature of the point;

[0033] S4: establishing a point pair relationship based on the predicted overlapping region and the feature of the point, and calculating a position transformation relationship to complete registration.

[0034] The preprocessing process of the original pig body point cloud is as follows: firstly, voxel down-sampling is performed on the input two frames of original pig body point cloud; two frames of down-sampled pig body point cloud with uniform surface distribution are obtained, and corresponding points in the two frames of down-sampled pig body point cloud are found to form a point pair, which is used as data for calculating the error of the pig body point cloud registration network. The multi-view pig body local three-dimensional point cloud data is as shown in Figure 2 .

[0035] The overall flowchart of the point cloud registration network model is as shown in Figure 3 , the pig body point cloud registration network is a dual-stream encoding and decoding network, which includes an encoder composed of a feature extraction module ResKPConv, a cross-overlapping region attention module, and a decoder composed of a one-dimensional convolution layer Conv1D; the encoder encodes the preprocessed original pig body point cloud, and then outputs feature information; the cross-overlapping region attention module inputs two feature information output by the encoder to find the overlapping region between the two feature information, and outputs the corresponding feature information; the decoder decodes the feature information output by the cross-overlapping region attention module to obtain the final feature information.

[0036] The encoder includes a plurality of feature extraction modules ResKPConv, and the feature extraction module ResKPConv includes a kernel point convolution submodule KPConv; the specific working process of the kernel point convolution submodule KPConv is as follows: the kernel point convolution submodule KPConv performs convolution on the input data, adds and aggregates the results of the convolution, and then outputs feature information through an activation function; the specific calculation formula is as follows:

[0037] y'=LeakyReLU(y+KPConv(y))

[0038] Where LeakyReLU() is an activation function, KPConv() is a convolution operation in the kernel point convolution submodule KPConv, y is the input data of the feature extraction module ResKPConv, and y' is the output feature information.

[0039] The cross-overlap region attention module includes a graph convolution submodule and a cross-attention submodule. Before the feature information output by the encoder is input into the cross-attention submodule, the graph convolution submodule enhances the neighborhood information of each point in the feature information output by the encoder. The cross-attention submodule receives the feature information output by the two graph convolution submodules and uses it to find the overlapping region between the feature information output by the two graph convolution submodules and outputs the corresponding feature information. Then, the graph convolution submodule further enhances the neighborhood information of each point in the feature information output by the cross-attention submodule.

[0040] The graph convolution submodule includes several graph convolution layers, and the specific calculation formula for the graph convolution submodule is as follows:

[0041] y GNN =j β (cat[ (0) y, (1) y, (2) y])

[0042] Among them, y GNN To enhance the feature information after the domain information of each point; j β This is a sub-component within the graph convolution submodule, used to enhance the generalization of the pig body point cloud registration network, including linear layers, normalization, and the LeakyReLU activation function; 0) y represents the feature information of the input graph convolutional submodule; (1) y is (0) y is the feature information output by the first graph convolutional layer; (2) y is (1) y represents the feature information output by the second graph convolutional layer; cat[] represents the aggregation and splicing of the feature information of the input graph convolutional submodule.

[0043] The specific calculation formulas for the feature information output by the first graph convolutional layer and the feature information output by the second graph convolutional layer are as follows:

[0044] (k+1) y i =maxh β (cat[ (k) y j - (k) y i , (k) y i ])

[0045] in (k) y i and (k) y j The input graph convolutional layer contains feature information about points i and j, where points i and j are neighbors. (k+1) y iis the feature information of point i in the feature information output by the graph convolution layer; k is the layer number of the passed graph convolution layer, h β is a subcomponent in the graph convolution submodule, used to enhance the generalization of the pig body point cloud registration network, and contains a linear layer and a LeakyReLU activation function, max() represents a maximum value function, and cat[] represents aggregation and splicing of the feature information of the input graph convolution layer.

[0046] The cross-attention submodule includes four parallel multi-attention subcomponents and an MLP subcomponent; the four parallel multi-attention subcomponents extract similar parts in the feature information of the two input four parallel multi-attention subcomponents as similar encodings, and then learn and nonlinearly map through the MLP subcomponent full connection network to output the feature information; the specific calculation formula is as follows:

[0047]

[0048] wherein represents the feature of point i in the first feature information output by the cross-attention module, represents the feature of point i in the first feature information output by the graph convolution submodule, MLP() represents a full connection network, s i is the feature of point i in the s feature space, m i← is the output of the input first feature information through the four parallel multi-attention subcomponents; s i The specific calculation formula is as follows: wherein W s is a weight matrix; m i← The specific calculation formula is as follows:

[0049] m i← =∑a ij v j

[0050] wherein v j is the representation of point j in the second feature information in the v feature space, a ij is a weight of a bipartite graph formed by the mutual connection between point i in the first feature information and point j in the second feature information, and the weight calculation formula is as follows: k j is the encoding of the second feature information in the v feature space k, v j ,k j The specific calculation formula is as follows:

[0051]

[0052]

[0053] wherein Wk , W v and W s is a weight matrix, is a feature of a point j in the second feature information output by the graph convolution submodule.

[0054] The decoder decodes the feature information output by the cross-overlapping area attention module, outputs the feature descriptor of each point, and refines the probability and topological feature encoding of the overlapping area of each point in the down-sampled point cloud.

[0055] The registration process is: selecting points whose first dimension value of each point of the neighborhood feature information decoded by the decoder is greater than a threshold, establishing a point pair relationship based on the prediction area and the point feature information of the selected points, estimating the position transformation relationship, and completing the registration; wherein the first dimension is the probability of the overlapping area, and the threshold is 80%-90%.

[0056] The overall effect diagram of the pig body after registration is as shown in Figure 4 , and the cross-section is as shown in Figure 5 .

[0057] Obviously, the above embodiments of the present application are only examples for clearly illustrating the present application, and are not intended to limit the embodiments of the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. A pig body point cloud registration method based on overlapping region prediction, characterized by: The method comprises the following steps: obtaining and preprocessing the original pig body point cloud; constructing a pig body point cloud registration network; inputting the preprocessed original pig body point cloud into the pig body point cloud registration network to predict the probability of the overlapping area and extract the point features; establishing a point pair relationship based on the predicted overlapping area and the point features, and calculating the position transformation relationship to complete the registration; The pig body point cloud registration network is a dual-stream coding and decoding network, comprising an encoder composed of a feature extraction module ResKPConv, a cross-overlapping area attention module and a decoder composed of a one-dimensional convolution layer Conv1D; the encoder encodes the preprocessed original pig body point cloud and then outputs feature information; The cross-overlapping area attention module inputs two encoder output feature information, which is used to find the overlapping area between the two feature information, and outputs the corresponding feature information; The decoder decodes the feature information output by the cross-overlapping area attention module to obtain the final feature information. The cross-overlapping area attention module comprises a graph convolution submodule and a cross-attention submodule; the graph convolution submodule strengthens the domain information of each point in the feature information output by the encoder before the feature information is input into the cross-attention submodule; the cross-attention submodule receives the feature information output by the two graph convolution submodules, which is used to find the overlapping area between the feature information output by the two graph convolution submodules, and outputs the corresponding feature information; then the graph convolution submodule strengthens the domain information of each point in the feature information output by the cross-attention submodule.

2. The pig body point cloud registration method based on overlapping region prediction according to claim 1, characterized in that: The preprocessing process of the original pig body point cloud is as follows: firstly, voxel down-sampling is performed on the input two frames of original pig body point cloud; two frames of down-sampled pig body point cloud with uniform surface distribution are obtained, and corresponding points in the two frames of down-sampled pig body point cloud are found to form point pairs, which are used as data for calculating the error of the pig body point cloud registration network.

3. The pig body point cloud registration method based on overlapping region prediction according to claim 1, characterized in that: The encoder comprises a plurality of feature extraction modules ResKPConv, and each feature extraction module ResKPConv comprises a kernel point convolution submodule KPConv; the specific working process of the kernel point convolution submodule KPConv is as follows: the kernel point convolution submodule KPConv performs convolution on the input data, adds and aggregates the results of the convolution, and then outputs feature information through an activation function.

4. The pig body point cloud registration method based on overlapping region prediction according to claim 1, characterized in that: The specific calculation formula of the graph convolution submodule is as follows: wherein, is the feature information after strengthening the domain information of each point; is a subcomponent in the graph convolution submodule, used to enhance the generalization of the pig body point cloud registration network, including a linear layer, normalization, and a LeakyReLU activation function; is the feature information input into the graph convolution submodule; is is the feature information output by the first layer of graph convolution layers; is is the feature information output by the second layer of graph convolution layers; represents the feature information aggregated and spliced from the input graph convolution submodule.

5. The pig body point cloud registration method based on overlapping region prediction according to claim 4, characterized in that: The specific calculation formula of the feature information output by the first layer of graph convolution layers and the feature information output by the second layer of graph convolution layers is as follows: wherein and is the feature information of point i and point j in the feature information of the input graph convolutional layer, and point i and point j have a neighboring relationship, is the feature information of point i in the feature information output by the graph convolutional layer; k is the layer number of the passed graph convolutional layer, is a subcomponent in the graph convolutional submodule, used to enhance the generalization of the pig body point cloud registration network, and contains a linear layer and a LeakyReLU activation function, represents a maximum value function, represents the aggregation and splicing of the feature information of the input graph convolutional layer.

6. The pig body point cloud registration method based on overlapping region prediction according to claim 1, characterized in that, The cross-attention submodule comprises four parallel multi-attention subcomponents and an MLP subcomponent; the four parallel multi-attention subcomponents extract similar parts in the feature information input by the two four parallel multi-attention subcomponents as similar codes, and then output the feature information through the MLP subcomponent full connection network learning and nonlinear mapping.

7. The pig body point cloud registration method based on overlapping region prediction according to claim 1, characterized in that: The decoder decodes the feature information output by the cross-overlapping area attention module, outputs the feature descriptor of each point, and refines the probability of the overlapping area and the topological feature coding of each down-sampled point cloud.

8. The pig body point cloud registration method based on overlapping region prediction according to claim 1, characterized in that: The registration process is: selecting a point whose value of the first dimension of the neighborhood feature information decoded by the decoder is greater than a threshold value, establishing a point pair relationship based on the prediction area and the point feature information of the selected point, estimating a position transformation relationship, and completing registration; wherein the first dimension is the probability of the overlapping area, and the threshold value is 80%-90%.

Images