A point cloud completion method based on anchor point detection

Abstract

The present invention belongs to the technical field of point cloud completion, and relates to a point cloud completion method based on anchor point detection. The method first uses a local deletion operation to preprocess the data set to simulate an incomplete point cloud; then the space where the point cloud is located is divided into multiple anchor points, and a neural network is trained to determine the positive and negative properties of the anchor points to obtain positive sample anchor points with uniform density and regular shape; finally, a dense point cloud is generated by a neural network in the neighborhood of the positive sample anchor point, and diffusion from a single anchor point to a local area is achieved to complete the point cloud completion. To implement this method, we trained a GCNet model adapted thereto, which can determine the positive and negative properties of the anchor points and generate three-dimensional coordinate information of the anchor point diffusion, thereby achieving end-to-end point cloud completion, and solving the problem that the existing method only relies on neural network regression to generate point cloud coordinates, resulting in poor completion effect, and improving the effect of point cloud completion.

Description

Technical Field

[0001] The present invention belongs to the field of point cloud completion, and in particular relates to a point cloud completion method based on anchor point detection. Background Art

[0002] Predicting complete 3D point clouds is a core task in many computer vision tasks.

[0003] Existing methods are usually based on deep learning to extract global features from incomplete point clouds to restore complete point clouds. Although they can predict complete point clouds to a certain extent, they cannot accurately generate the structural details of the object and have the problem of uneven density.

[0004] Applying deep learning technology to point cloud data has become a popular trend in recent years. For point cloud completion tasks, neural networks can be trained to achieve the goal, that is, input an incomplete point cloud structure and generate a complete point cloud structure.

[0005] At present, most network models based on deep learning have poor effects on point cloud completion. The reason is that these models simply classify the task of point cloud completion as a regression problem and train neural networks to predict point cloud coordinates. Therefore, they can only generate point clouds that are generally correct but irregular in shape. Therefore, it is necessary to achieve point cloud completion with regular shape and uniform density. Summary of the invention

[0006] In order to overcome the shortcomings of local feature extraction in the prior art, the present invention proposes a point cloud completion method based on anchor point detection and designs a network structure that matches it to generate regular and uniform point cloud coordinates. Experiments have shown that the invention has excellent effects in both vision and indicators.

[0007] The technical solution adopted by the present invention is:

[0008] A point cloud completion method based on anchor point detection includes the following steps:

[0009] S1. Dataset acquisition and data preprocessing: Obtain the point cloud public dataset ShapeNet, use the local deletion operation to delete the local point cloud in the complete point cloud data, and thus construct an incomplete point cloud;

[0010] S2. Neural network model construction: Based on the idea of ​​point cloud completion from coarse to fine, a sparse point cloud with regular shape and uniform density is first obtained by judging the positive and negative properties of the anchor points, and then a dense point cloud is obtained by diffusing the sparse point cloud. To realize this idea, a GCNet network model is constructed;

[0011] S3, model training and model saving: Use the Adam optimizer to train the network, reduce the loss function, and improve the point cloud completion effect. When the loss function of the model tends to be stable, save the model.

[0012] In step S1, the public data set ShapeNet contains 16 types of point cloud data, and each individual point cloud data file (.pts) of all types is composed of n point cloud coordinates x, y, z in a three-dimensional coordinate system, and the value of n is not the same.

[0013] In the step S1, the specific methods of the local deletion operation include key point definition and selection, and key point neighboring point cloud deletion.

[0014] In step S1, the specific method of defining and selecting the key points is: taking the four points (1,0,0), (0,0,1), (1,0,1), and (-1,0,0) as the key points, and the three numbers in () represent the coordinates of x, y, and z in the three-dimensional coordinate system; to simulate the incomplete point cloud with different missing parts, we use the four key points as the center points in turn, and delete the point cloud data near the center point position; the specific steps of the key point neighboring point cloud deletion operation are: calculating the distance between all points in the point cloud and the center, and deleting the 512 point clouds closest to the center point.

[0015] In step S2, in the sparse point cloud with regular shape and uniform density obtained by judging the positive and negative properties of the anchor points, since the number of negative samples in the space is far greater than the positive samples, too many negative samples have no benefit for network training. Therefore, in order to solve the problem of too many negative samples, we only select three times the number of positive samples in the negative samples for training. In addition, in the selected negative samples, we further divide the negative samples into difficult-to-train negative samples and easy-to-train negative samples by setting a threshold d. The negative samples with a distance less than the threshold d are difficult-to-train negative samples; the negative samples with a distance greater than the threshold d are easy-to-train negative samples, where the ratio of the number of difficult-to-train negative samples to the number of easy-to-train negative samples is 2:1.

[0016] In the step S2, in the dense point cloud obtained by diffusion of the sparse point cloud, since the sparse point cloud can only represent the shape structure of the point cloud as a whole but cannot display its specific details, we generate the point cloud coordinates in the neighborhood of each positive anchor point through a neural network based on the sparse point cloud (positive anchor point samples), and limit the maximum range of the neighborhood through tanh to no more than half of the distance between the two anchor points.

[0017] In step S2, the GCNet includes an encoder and a decoder. The specific goal of the encoder is to extract the coordinate features of the input incomplete point cloud. The specific structure of the encoder is that the input point cloud coordinates first pass through a convolution layer to obtain the feature vector of the point cloud, and then divide the space where the point cloud is located into w×w×w anchor points. The vectors in the grid corresponding to the anchor points are averaged to achieve grid structuring of the features, and finally high-dimensional features are extracted through 3D convolution and residual connection. The specific goal of the decoder is to generate the point cloud coordinates of the missing part through the high-dimensional features extracted by the decoder. The specific structure of the decoder is that the input high-dimensional features first pass through 3D transposed convolution to achieve feature upsampling, which is used to predict the positive and negative anchor points and the diffuse point cloud coordinate information in the neighborhood of the positive anchor points to generate the final complete point cloud.

[0018] In step S3, the loss function includes BCELoss and CD, and the specific formula of BCELoss is:

[0019]

[0020] Where X i is the predicted value, y i is the label, and the specific formula of CD is:

[0021]

[0022] Where CD(S1,S2) represents the average shortest distance in Euclidean space between the predicted point cloud set S1 and the real point cloud set S2; x and y represent a point cloud in the point cloud set, respectively, consisting of three coordinates: x, y, and z.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The present invention abandons the idea of ​​predicting point cloud coordinates by pure regression and trains the neural network GCNet. The neural network model can judge the positive and negative of anchor points and generate three-dimensional coordinate information of anchor point diffusion, realize end-to-end point cloud completion, and solve the problem of poor completion effect caused by existing methods that only rely on neural network regression to generate point cloud coordinates. The effect of point cloud completion is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart of the present invention;

[0026] Figure 2 This is the structural diagram of GCNet of the present invention;

[0027] Figure 3 This is the aircraft point cloud completion effect in Example 1;

[0028] Figure 4 This is the aircraft point cloud completion effect in Example 2;

[0029] Figure 5 This is the aircraft point cloud completion effect in Example 3; DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention is clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] A point cloud completion method based on anchor point detection uses the ShapeNet dataset for point cloud completion training. The dataset contains 16 categories and a total of 15,011 point cloud data, including 12,137 point cloud data in the training set and 2,874 point cloud data in the test set. Each point cloud data is preprocessed through viewpoint operation to obtain an incomplete point cloud for network input. The incomplete point cloud is input into GCNet and trained using BCELoss and CD loss functions. After 100 rounds of training, the weights with the best effect are saved. Figure 1 As shown, the detailed process includes the following steps:

[0032] S1. Dataset acquisition and data preprocessing: Obtain the point cloud public dataset ShapeNet, use the local deletion operation to delete the local point cloud in the complete point cloud data, and thus construct an incomplete point cloud;

[0033] S2. Neural network model construction: Based on the idea of ​​point cloud completion from coarse to fine, a sparse point cloud with regular shape and uniform density is first obtained by judging the positive and negative properties of the anchor points, and then a dense point cloud is obtained by diffusing the sparse point cloud. To realize this idea, a GCNet network model is constructed;

[0034] S3, model training and model saving: Use the Adam optimizer to train the network, reduce the loss function, and improve the point cloud completion effect. When the loss function of the model tends to be stable, save the model.

[0035] In step S1, the public data set ShapeNet contains 16 types of point cloud data, and each individual point cloud data file (.pts) of all types is composed of n point cloud coordinates x, y, z in a three-dimensional coordinate system, and the value of n is not the same.

[0036] In the step S1, the specific methods of the local deletion operation include key point definition and selection, and key point neighboring point cloud deletion.

[0037] In step S1, the specific method of defining and selecting the key points is: taking the four points (1,0,0), (0,0,1), (1,0,1), and (-1,0,0) as the key points, and the three numbers in () represent the coordinates of x, y, and z in the three-dimensional coordinate system; to simulate the incomplete point cloud with different missing parts, we use the four key points as the center points in turn, and delete the point cloud data near the center point position; the specific steps of the key point neighboring point cloud deletion operation are: calculating the distance between all points in the point cloud and the center, and deleting the 512 point clouds closest to the center point.

[0038] In step S2, in the sparse point cloud with regular shape and uniform density obtained by judging the positive and negative properties of the anchor points, since the number of negative samples in the space is far greater than the positive samples, too many negative samples have no benefit for network training. Therefore, in order to solve the problem of too many negative samples, we only select three times the number of positive samples in the negative samples for training. In addition, in the selected negative samples, we further divide the negative samples into difficult-to-train negative samples and easy-to-train negative samples by setting a threshold d. The negative samples with a distance less than the threshold d are difficult-to-train negative samples; the negative samples with a distance greater than the threshold d are easy-to-train negative samples, where the ratio of the number of difficult-to-train negative samples to the number of easy-to-train negative samples is 2:1.

[0039] In the step S2, in the dense point cloud obtained by diffusion of the sparse point cloud, since the sparse point cloud can only represent the shape structure of the point cloud as a whole but cannot display its specific details, we generate the point cloud coordinates in the neighborhood of each positive anchor point through a neural network based on the sparse point cloud (positive anchor point samples), and limit the maximum range of the neighborhood through tanh to no more than half of the distance between the two anchor points.

[0040] In step S2, the GCNet structure is as follows: Figure 2 As shown in the figure, the GCNet includes two parts, an encoder and a decoder. The specific goal of the encoder part is to extract the coordinate features of the input incomplete point cloud; the specific structure of the encoder part is that the input point cloud coordinates first pass through a convolution layer to obtain the feature vector of the point cloud, and then divide the space where the point cloud is located into w×w×w anchor points, and the vectors in the grid corresponding to the anchor points are averaged to achieve grid structuring of the features, and finally extract high-dimensional features through 3D convolution and residual connection; the specific goal of the decoder part is to generate the point cloud coordinates of the missing part through the high-dimensional features extracted by the decoder part; the specific structure of the decoder part is that the input high-dimensional features first pass through 3D transposed convolution to achieve feature upsampling, which is used to predict the positivity of the anchor points and the diffuse point cloud coordinate information in the neighborhood of the positive anchor points, and generate the final complete point cloud.

[0041] In step S3, the loss function includes BCELoss and CD, and the specific formula of BCELoss is:

[0042]

[0043] Where X i is the predicted value, y i is the label, and the specific formula of CD is:

[0044]

[0045] Where CD(S1,S2) represents the average shortest distance in Euclidean space between the predicted point cloud set S1 and the real point cloud set S2; x and y represent a point cloud in the point cloud set, respectively, consisting of three coordinates: x, y, and z.

[0046] The present invention uses Examples 1-3 to intuitively demonstrate the completion effect of GCNet. The GCNet model is trained in the training set.

[0047] Embodiment 1:

[0048] Select an incomplete aircraft point cloud in the test set as input and output the anchor point and the complete aircraft point cloud. Figure 3 shown.

[0049] Embodiment 2:

[0050] Select an incomplete aircraft point cloud in the test set as input and output the anchor point and the complete aircraft point cloud. Figure 4 shown.

[0051] Embodiment 3:

[0052] Select an incomplete aircraft point cloud in the test set as input and output the anchor point and the complete aircraft point cloud. Figure 5 shown.

[0053] The above only expresses the preferred embodiments of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that for ordinary technicians in this field, several modifications, improvements and substitutions can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be based on the attached claims.

Claims

1. A point cloud completion method based on anchor point detection, characterized in that: The following steps are involved: S1. Dataset acquisition and data preprocessing: Obtain the point cloud public dataset ShapeNet, use the local deletion operation to delete the local point cloud in the complete point cloud data, and thus construct an incomplete point cloud; S2. Neural network model construction: Based on the idea of ​​point cloud completion from coarse to fine, a sparse point cloud with regular shape and uniform density is first obtained by judging the positive and negative properties of the anchor points, and then a dense point cloud is obtained by diffusing the sparse point cloud. To realize this idea, a GCNet network model is constructed; S3, model training and model saving: Use the Adam optimizer to train the network, reduce the loss function, and improve the point cloud completion effect. When the loss function of the model tends to be stable, save the model; In the step S2, in the sparse point cloud with regular shape and uniform density obtained by judging the positive and negative properties of the anchor points, only three times the number of positive samples are selected from the negative samples for training; in addition, in the selected negative samples, by setting a threshold d, the negative samples are further divided into difficult-to-train negative samples and easy-to-train negative samples, and the negative samples with a distance less than the threshold d are difficult-to-train negative samples; and the negative samples with a distance greater than the threshold d are easy-to-train negative samples, wherein the ratio of the number of difficult-to-train negative samples to the number of easy-to-train negative samples is 2:1; In the step S2, in the dense point cloud obtained by diffusion of the sparse point cloud, the point cloud coordinates in the neighborhood of each positive anchor point are generated by a neural network on the basis of the sparse point cloud, and the maximum range of the neighborhood is limited by tanh so that it does not exceed half of the distance between two anchor points; In step S2, the GCNet includes an encoder and a decoder. The specific goal of the encoder is to extract the coordinate features of the input incomplete point cloud. The specific structure of the encoder is that the input point cloud coordinates first pass through a convolution layer to obtain the feature vector of the point cloud, and then divide the space where the point cloud is located into w×w×w anchor points. The vectors in the grid corresponding to the anchor points are averaged to achieve grid structuring of the features, and finally high-dimensional features are extracted through 3D convolution and residual connection. The specific goal of the decoder is to generate the point cloud coordinates of the missing part through the high-dimensional features extracted by the decoder. The specific structure of the decoder is that the input high-dimensional features first pass through 3D transposed convolution to achieve feature upsampling, which is used to predict the positive and negative anchor points and the diffuse point cloud coordinate information in the neighborhood of the positive anchor points to generate the final complete point cloud.

2. The point cloud completion method based on anchor point detection according to claim 1, characterized in that: In step S1, the public data set ShapeNet contains 16 types of point cloud data, and each individual point cloud data file of all types is composed of n point cloud coordinates x, y, z in a three-dimensional coordinate system, and the value of n is not the same.

3. The point cloud completion method based on anchor point detection according to claim 1, characterized in that: In the step S1, the specific methods of the local deletion operation include key point definition and selection, and key point neighboring point cloud deletion.

4. The point cloud completion method based on anchor point detection according to claim 3, characterized in that: In step S1, the specific method of defining and selecting the key points is: taking the four points (1,0,0), (0,0,1), (1,0,1), and (-1,0,0) as the key points, and the three numbers in () represent the coordinates of x, y, and z in the three-dimensional coordinate system; to simulate the incomplete point cloud with different missing parts, the four key points are used as the center points in turn, and the point cloud data near the center point position is deleted; the specific steps of the key point adjacent point cloud deletion operation are: calculating the distance between all points in the point cloud and the center, and deleting the 512 point clouds closest to the center point.

5. The point cloud completion method based on anchor point detection according to claim 1, characterized in that: In step S3, the loss function includes BCELoss and CD, and the specific formula of BCELoss is: Where X i is the predicted value, y i is the label, and the specific formula of CD is: Where CD(S1,S2) represents the average shortest distance in Euclidean space between the predicted point cloud set S1 and the real point cloud set S2; x and y represent a point cloud in the point cloud set, respectively, consisting of three coordinates: x, y, and z.

Images