A point cloud feature extraction method based on local point cloud position relationship
Through spatial fill curve cosine function position information cosine combined with local area attention mechanism, the problem of insufficient utilization of local structure information in three-dimensional point cloud data processing is solved, the representativeness and distinction of point cloud feature extraction is improved, and automatic Application effects in the fields of driving and robotics.
Patent Information
- Application Number
- CN202210499455.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-09
- Publication Date
- 2024-11-19
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing three-dimensional point cloud data processing methods cannot fully utilize the local structural information of point cloud data, resulting in a lack of representativeness in feature extraction and the theoretical foundation of deep learning methods in this field is not solid enough.
Using spatial fill curve encoding, cosine function position information encoding and local area attention mechanism, an iterative feature extraction network model is constructed, point cloud features are extracted through Pointnet++ network structure and sampling layer, and combined with attention mechanism to improve the representativeness and distinction of feature extraction Spend.
Effectively process point cloud data, changing it from disorder to order, improving the efficiency of local information extraction, and adaptively adjusting feature focus areas and weights through attention mechanisms, improving the accuracy of point cloud classification and segmentation tasks.
Smart Images

Figure CN114998604B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of three-dimensional point cloud data processing, and in particular to a point cloud feature extraction method based on local point cloud position relationship. Background Art
[0002] Compared with two-dimensional image data, three-dimensional point cloud data contains richer and more complete information, and the objects or scenes it represents are more intuitive and friendly, which is easier for computers to understand. With the development of science and technology, three-dimensional point cloud data is widely used in various fields such as autonomous driving, robots, and high-precision maps, and is increasingly valued and favored by industry and academia. Traditional three-dimensional point cloud data processing methods can no longer meet the requirements of accuracy and complexity in real life and applications. It is increasingly urgent to use deep learning methods to extract more representative features of three-dimensional point cloud data.
[0003] Different from the regularly arranged information in two-dimensional images, three-dimensional point clouds have the characteristics of irregularity and disorder, which makes it difficult to extract the feature information of point clouds. In recent years, great progress has been made in the research on three-dimensional point cloud data processing methods, and a large number of fruitful results have emerged. In particular, with the development of artificial intelligence technology, the methods based on deep learning have greatly improved the accuracy of various tasks of point clouds compared with traditional algorithms. However, at present, both traditional point cloud data processing methods and existing deep learning point cloud processing methods have some defects: 1) The input of point cloud data is the original point cloud data, which is disorganized and does not utilize the relationship between the positions of point clouds; 2) The feature information of the local area of the point cloud cannot be fully utilized, resulting in a lack of representativeness in feature extraction. In addition, since the research on point cloud data processing using deep learning methods has only started in recent years and is still in the exploratory stage, the current theoretical foundation is not solid enough. Summary of the invention
[0004] Aiming at the problem that the local structural information of point cloud data cannot be fully and effectively utilized, space filling curves, position information encoding and attention mechanism of local areas are introduced to obtain more representative and distinguishing point cloud features. The present invention provides a point cloud feature extraction method based on the position relationship of local point clouds, which mainly includes the following steps:
[0005] S1: Obtain original point cloud data from a public 3D point cloud data set, encode and sort the original 3D point cloud data according to the encoding rule of the space filling curve - Z curve;
[0006] S2: The sorted original point cloud data is encoded using the position information encoding technology of the sine and cosine functions, and the encoded data is fused with the original point cloud data to obtain the point cloud data after data enhancement;
[0007] S3: Based on the Pointnet++ network structure, the point cloud data is grouped and sampled, and then the sampling layer and the multi-layer perceptron layer are iteratively used to extract features on each group, thereby constructing an iterative feature extraction network model;
[0008] S4: In combination with the attention mechanism, point cloud features are extracted from the point cloud data in the local neighborhood of the key point in the feature extraction network model to obtain a point cloud feature vector of the local neighborhood;
[0009] S5: Apply the point cloud feature vector extraction method of steps S1-S4 to the point cloud classification and segmentation tasks respectively to obtain specific point cloud classification results, and compare and display the results.
[0010] Furthermore, step S1 is specifically as follows:
[0011] S11: The public 3D point cloud data set includes standard data sets modelnet40, shapenet, and S3DIS as the source of original point cloud data;
[0012] S12: Encode all original point cloud data according to the encoding rule of the space filling curve—Zorder curve to obtain a Zorder value;
[0013] S13: Sort the Zorder values in descending order to obtain the sequence numbers of the point cloud data in the Z curve, that is, the position information.
[0014] Furthermore, step S2 is specifically as follows:
[0015] S21: The point cloud data sorted by the Zorder value is encoded using the sine and cosine function position information encoding technology. The sine and cosine function position information encoding technology used is shown in Formula 1 and Formula 2:
[0016]
[0017]
[0018] In the above formula, d model represents the dimension of the code, pos represents the coding order, PE(pos;2i) and PE(pos;2i+1) represent that the even and odd dimension values of the code are calculated using the sine function and cosine function respectively;
[0019] S22: After encoding each original point cloud data using the position information encoding technology of the Zorder curve sequence, the obtained data is merged with the original point cloud data to form enhanced point cloud data.
[0020] Furthermore, step S3 is specifically as follows:
[0021] S31: Based on the Pointnet++ network structure, the average sampling method is used to sample key points;
[0022] The key point sampling process is as follows: Assume that the point cloud data set obtained after sorting according to the Zorder value is The superscript {1,2,…,N} represents the sequence of point cloud data after Zorder curve encoding and sorting, that is, the logical distance between points. If x is randomly selected n is a key point, then the next key point is x (n+d)%N , d = N / M, N is the total amount of point cloud data, M is the number of key points sampled;
[0023] S32: After sampling the key points, the ball neighborhood method is used to construct the local neighborhood of each key point by grouping, that is, all points in the ball neighborhood formed by taking each key point as the origin and r as the radius are classified into a group of local neighborhood points to obtain a grouping layer; the grouping layer inputs a point set of size M×(3+C), where M is the number of key points, 3 represents the coordinate dimension of the point cloud, and C represents the feature dimension of the key point. The output size of the grouping layer is M×K×(3+C), where K represents the number of neighboring points of each key point;
[0024] S33: In order to extract the local neighborhood features of each key point, each key point is first transformed into a three-dimensional local coordinate system with the key point as the origin, and then a PointNet multi-layer perceptron is used to extract features for each point in the local area, that is, K points in the local area are input into the PointNet multi-layer perceptron to obtain K features of size 1×(3+C′), where C′ is the new feature dimension obtained. Finally, the features of the K points are aggregated using the maximum pooling method to finally obtain a point cloud feature of size 1×(3+C′), which represents the local neighborhood features of the key point.
[0025] S34: Alternate sampling layers and PointNet multi-layer perceptron layers, stacked to form an iterative feature extraction network model.
[0026] Furthermore, step S4 is specifically as follows:
[0027] S41: Input an intermediate feature map F∈R into the feature extraction network model after applying the local area attention mechanism C×H×W , through the one-dimensional channel attention M c ∈R C×1×1 and two-dimensional spatial attention M s ∈R 1×H×W; Take the feature map F = H × W × C as input, first perform a spatial global average pooling and maximum pooling to obtain two 1 × 1 × C channel descriptions, then send them to a two-layer neural network, and then add the two features output by the neural network and obtain the weight coefficient Mc through a Sigmoid activation function. Finally, multiply the weight coefficient with the original feature F to obtain the scaled new feature F':
[0028]
[0029] S42: After channel attention, given a H×W×C feature F', first perform average pooling and maximum pooling in a channel dimension to obtain two H×W×1 channel descriptions, then pass through a 7×7 convolution layer with a Sigmoid activation function to obtain the weight coefficient Ms, and finally, multiply the weight coefficient and feature F' to obtain the scaled new feature F":
[0030]
[0031] Furthermore, in step S5, when applied to the point cloud classification task, a classification network needs to be constructed. The input of the classification network is the feature vector obtained after processing in step S4, with a dimension of N×256. After the input feature is processed by a multi-layer perceptron with a network layer number of (256, 512, 1024), a feature vector with a dimension of 1024×1 is obtained, which is then passed through the fully connected layers of (1024, 512), (512, 256,), and (256, k) in sequence. Each fully connected layer is processed with the relu function, and finally the score of each category is output. The obtained score is the probability that each sample belongs to each category. The classification network uses the cross entropy loss function as the objective function of point cloud classification. The calculation formula of the loss function is shown in Formula 5:
[0032]
[0033] Where M is the total number of categories, y c is a categorical indicator variable, which is 1 if the predicted category is the same as the true category, otherwise it is 0. c The model predicts the probability of belonging to category c.
[0034] Furthermore, in step S5, when applied to the point cloud segmentation task, a segmentation network needs to be constructed. The design of the segmentation network adopts a cascade method to merge the feature vector obtained by the position information encoding method of the Zorder curve sequence with the features extracted from the subsequent feature extraction, and finally outputs the prediction probability of each point;
[0035] The loss function of the point cloud component segmentation task is defined as shown in Formula 6:
[0036]
[0037]
[0038] p ij To convert point x i The probability of predicting semantic category j, y j is the true semantic category of the point; only when j = y i Time ij =1, otherwise y ij = 0, n is the total number of points in the model; w j is the weight coefficient, which indicates the proportion of points with semantic category j in all points in the dataset;
[0039] The loss function of the point cloud indoor scene segmentation task is defined as shown in Formula 8:
[0040]
[0041] Where A is the spatial transformation matrix, I is the unit matrix, n is the total number of points in the model, i is the i-th point cloud, j is the sample semantic category, and y is the j is the true semantic category of the point, y ij is a one-hot vector with only two possible values: 0 and 1. This category is the same as the sample category. i Time ij =1, otherwise y ij =0.
[0042] The beneficial effects brought by the technical solution provided by the present invention are:
[0043] (1) Space filling curves and position information encoding technology are used to process point cloud data, making the point cloud orderly, thereby making the extraction of local information more effective.
[0044] (2) The addition of the hybrid attention mechanism enables the network to adaptively adjust the weights of the areas and features that need attention, making feature extraction more targeted. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:
[0046] Figure 1 It is a flow chart of a point cloud feature extraction method based on the position relationship of local point clouds in an embodiment of the present invention.
[0047] Figure 2 1 is a schematic diagram of the visualization of the point cloud Zorder sequence in an embodiment of the present invention (a) and a flow chart of the position information encoding combined with the Zorder curve sequence (b).
[0048] Figure 3 Schematic diagram of the attention model in an embodiment of the present invention.
[0049] Figure 4 It is a diagram of the overall network structure in an embodiment of the present invention.
[0050] Figure 5 4 is a comparison diagram of the robustness of different numbers of input points in an embodiment of the present invention.
[0051] Figure 6 It is a visualization result diagram of the segmentation result of the ShapeNet dataset in an embodiment of the present invention.
[0052] Figure 7 It is a visualization diagram of the scene segmentation result of the S3DIS data set in an embodiment of the present invention. DETAILED DESCRIPTION
[0053] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.
[0054] The embodiment of the present invention provides a point cloud feature extraction method based on the position relationship of local point clouds, which provides solid technical support for the application of three-dimensional point cloud data in the fields of autonomous driving, robotics, etc., and improves the effect and accuracy of point cloud data in various tasks such as classification and segmentation.
[0055] Please refer to Figure 1 , Figure 1 : is a flow chart of a point cloud feature extraction method based on the position relationship of a local point cloud in an embodiment of the present invention, which specifically includes the following steps:
[0056] S1: Use the public 3D point cloud dataset as the data source, encode and sort the original 3D point cloud data according to the encoding rule of the space filling curve Z curve.
[0057] S2: Extract the sorted data using position information encoding (extracting the potential position information of the sorted point cloud in the Zorder curve sequence) and save the position information of the point cloud in the Zorder sequence for further data enhancement.
[0058] S3: Build a network model and build an iterative feature extraction network model based on the Pointnet++ network structure.
[0059] S4: Apply the local area attention mechanism in the network model to highlight the important features of important areas, that is, extract the point cloud features based on the local point cloud position relationship.
[0060] S5: The point cloud features obtained by the feature extraction network are applied to point cloud classification and segmentation tasks respectively, and the model is trained and the results are compared and displayed.
[0061] Furthermore, step S1 is specifically as follows:
[0062] S11: Because deep learning experiments require a large amount of sample data to train the model, and it is difficult to obtain point cloud data as training samples, we use three internationally published standard data sets, modelnet40, shapenet, and S3DIS, as the source of point cloud data;
[0063] S12: Encode the point cloud data according to the space filling curve Z curve encoding rule, and use an example to explain the Z curve encoding rule. For example, point p(3 10 ,6 10 ) on the 2D 3rd-order Z curve is as follows: take the 3rd, 2nd, and 1st bits of p1(011)2 as the 6th, 4th, and 2nd bits of the Z-order value; then take the 3rd, 2nd, and 1st bits of p2(101)2 as the 5th, 3rd, and 1st bits of the Z-order value. The final result is (011110)2, which is (30). 10 ;
[0064] S13: Figure 2 As shown in (a), all input point cloud data are encoded according to the space filling curve Z curve encoding rule, and the Z-order values obtained are quickly sorted according to the size to obtain the sequence numbers of these point clouds in the Z curve, that is, the position information;
[0065] Furthermore, step S2 is specifically as follows:
[0066] S21: The point cloud sorted in S1 according to the Zorder rule is encoded using the sine and cosine function position information encoding technology. The sine and cosine function position information encoding technology formula 1 and formula 2 are as follows:
[0067]
[0068]
[0069] In the above formula, d model represents the dimension of the code, pos represents the coding order, PE(pos;2i) and PE(pos;2i+1) represent that the even dimension and odd dimension values of the code are calculated using the sine function and cosine function respectively.
[0070] S22: After encoding each point in the point set using the position information encoding technology, the obtained features are merged with the source data to form enhanced point cloud data; the position information encoding process combined with the Zorder curve sequence is as follows: Figure 2 As shown in (b).
[0071] Furthermore, step S3 is specifically as follows:
[0072] S31: In the downsampling process, an average sampling method is used to sample key points. The average sampling method can make full use of the good local feature information maintained by the Zorder curve sequence and can reduce the computational complexity.
[0073] S32: The average sampling process is as follows: Given an input point cloud sample set {x1, x2, …, x N ,}, if we want to extract M representative key points from these N points, first randomly select a point x i As the starting point of the new set, then take a point x at a certain distance d = N / M i Add to the new set until M points are taken out. The distance d here is not the physical Euclidean distance, but the logical distance between the front and the back. Assume that the point cloud sample set {x1,x2,…,x N ,}, the point cloud set obtained after sorting according to the Zorder curve sequence is The superscript {1,2,…,N} represents the sequence of the point cloud after Zorder curve encoding, that is, the logical distance between points. n is a key point, then the next key point is x (n+d)%N , d = N / M.
[0074] S33: After sampling the key points, the local neighborhood of each key point is constructed by grouping. In step S32, the number of key points sampled is M, and the input size of the grouping layer is a point set of M×(3+C), where M is the number of key points, 3 represents the three-dimensional coordinates of the point cloud, and C represents the features of the points. The output size of the grouping layer is M×K×(3+C). K represents the number of adjacent points of each key point. The grouping method used in the downsampling module is the spherical neighborhood method, that is, all points in the spherical neighborhood formed with each key point as the origin and r as the radius are grouped into a group of local neighborhood points.
[0075] S34: In order to extract the local neighborhood features of each key point, each key point is first transformed into a three-dimensional local coordinate system with the key point as the origin. Then a miniature PointNet multi-layer perceptron is used to extract features from the points in the local area. Specifically, the K points in the local area are first input into the MLP to obtain K features of size 1×(3+C′), and then the K point features are aggregated using the maximum pooling method to finally obtain a point cloud feature of size 1×(3+C′), which represents the local neighborhood features of the key point.
[0076] S35: Alternately use sampling modules and multi-layer perceptron layers, stacking them to form an iterative feature extraction network model.
[0077] Furthermore, step S4 is specifically as follows:
[0078] S41: Input an intermediate feature map F∈R C×H×W , the attention model sequentially passes through the 1-dimensional channel attention M c ∈R C ×1×1 and 2D spatial attention M s ∈R 1×H×W Inference. Channel attention module, each channel of the feature indicates a special detector, so it is meaningful for channel attention to focus on what kind of features. In order to summarize spatial features, global average pooling and maximum pooling are used here to utilize different information respectively. Taking the feature F = H × W × C as input, first perform a spatial global average pooling and maximum pooling to obtain two 1 × 1 × C channel descriptions. Then, send them to a two-layer neural network respectively. The number of neurons in the first layer is C / r, the activation function is Relu, and the number of neurons in the second layer is C. This two-layer neural network is shared. Then, add the two features obtained and obtain the weight coefficient Mc through a Sigmoid activation function. Finally, multiply the weight coefficient by the original feature F to obtain the scaled new feature F'.
[0079] S42: After the channel attention module, the spatial attention module is introduced to focus on which features are meaningful. Similar to the channel attention, given a H×W×C feature F', first perform average pooling and maximum pooling of a channel dimension to obtain two H×W×1 channel descriptions, and then splice these two descriptions together according to the channel. Then, through a 7×7 convolution layer with a Sigmoid activation function, the weight coefficient Ms is obtained. Finally, the weight coefficient is multiplied by the feature F' to obtain the scaled new feature F", which emphasizes and highlights the areas that have a positive effect on the final result. The whole process can be expressed by Formula 3 and Formula 4 as follows:
[0080]
[0081]
[0082] The attention mechanism model structure is as follows Figure 3 As shown, the overall structure of the network is as follows Figure 4 shown.
[0083] Furthermore, step S5 is specifically as follows:
[0084] S51: When applied to the point cloud classification task, a classification network is constructed. The input of the classification network is the feature vector obtained after processing in step S4, with a dimension of N×256. The input feature is processed by a multi-layer perceptron with a network layer number of (256,512,1024) to obtain a feature vector of 1024×1 dimension, and then passes through the fully connected layers of (1024,512), (512,256,), and (256,k) in sequence. Each fully connected layer is processed with the relu function, and finally the score of each category is output. The obtained score is the probability that each sample belongs to each category. The classification network uses the cross entropy loss function as the objective function of point cloud classification, and the calculation is shown in Formula 5:
[0085]
[0086] Where M is the total number of categories. In this experiment, M is 40. c It is a categorical indicator variable, which is 1 if the predicted category is the same as the true category, otherwise it is 0. c The model predicts the probability of belonging to category c.
[0087] S52: Training and testing of classification experiments were performed in the ModelNet40 dataset. The overall classification accuracy oA and class average accuracy mA of the model were calculated, and compared with PointNet, PointNet++, Kd-Network and other latest network model structures. The comparison results are shown in Table 1:
[0088] Table 1 Comparison of classification results of ModelNet40 dataset
[0089]
[0090] S53: In order to observe the influence of each designed module on the classification network, an ablation experiment is designed to verify it. SA represents the sampling module, AM represents the attention mechanism module, and the SA and AM modules are added and deleted from the two initial networks PointNet and PointNet++ to observe the influence on the accuracy of the experimental results. The results are shown in Table 2:
[0091] Table 2 Classification ablation experiment results of ModelNet40 dataset
[0092]
[0093]
[0094] S54: In order to test the robustness of the model, the input point cloud is sampled at different densities. The experiment uses the ModelNet40 classification dataset. For each model category in the dataset, four different sets of sampled point clouds are set to be input into the network, 1024, 512, 256, and 128. The overall classification accuracy oA is calculated for different numbers of input points and compared with the basic network PointNet and PointNet++ models. Figure 5 As shown;
[0095] S55: When applied to the segmentation task, a segmentation network is constructed. The design of the segmentation network uses a cascade method to fuse the intermediate features obtained in the front with the intermediate features in the back, and finally inputs the predicted probability of each point. The loss function of the point cloud component segmentation task is defined as shown in Formula 6
[0096]
[0097]
[0098] The loss function of the point cloud indoor scene segmentation task is defined as shown in Formula 8:
[0099]
[0100] Where A is the spatial transformation matrix, I is the unit matrix, n is the total number of points in the model, i is the i-th point cloud, j is the sample semantic category, and y is the j is the true semantic category of the point, y ij is a one-hot vector with only two possible values: 0 and 1. This category is the same as the sample category. i Time ij =1, otherwise y ij =0.
[0101] Compared with the component segmentation experiment, the point cloud indoor scene segmentation loss function adds this part, namely the regularization term This is because the number of features of the input point cloud is 9. In addition to the three-dimensional coordinates (x, y, z), the features of each point also include RGB color and normalized coordinate information, which is consistent with the input of PointNet (PointNet has confirmed that adding color and normalized coordinate information can improve the accuracy of point cloud segmentation). For a point cloud of size N×9, it is difficult for the spatial transformation network to optimize a spatial transformation matrix of size 9×9, so a regularization term is added to the network loss to make the spatial transformation matrix tend to be orthogonal, because orthogonal matrices usually do not lose input information. Adding regularization terms to the loss function can improve the speed and accuracy of network optimization to a certain extent.
[0102] S56: 3D object component segmentation experiments and indoor scene segmentation experiments were conducted on the ShapeNet dataset and the S3DIS dataset, respectively. The predicted Mean Intersection over Union (MIoU) values of each category and the mIoU values of the category instances were calculated, and then compared with multiple models and the results were visualized, as shown in Table 3:
[0103] Table 3 Segmentation results on ShapeNet dataset
[0104]
[0105] Table 4 shows the component segmentation ablation experiment performed on the ShapeNet dataset.
[0106] Table 4 Segmentation ablation experiment
[0107]
[0108] Table 5 shows the comparison of scene segmentation results on the S3DIS dataset.
[0109] Table 5.9 Experimental comparison of scene segmentation on the S3DIS dataset
[0110]
[0111] The segmentation results on the ShapeNet dataset are visualized as follows Figure 6 As shown, the S3DIS dataset scene segmentation results are visualized as follows Figure 7 shown.
[0112] S57: In order to observe the influence of each designed module on the segmentation network, an ablation experiment is designed to verify it. SA represents the sampling module, AM represents the attention mechanism module, and the SA and AM modules are added and deleted from the two initial networks PointNet and PointNet++ to observe the influence on the accuracy of the experimental results.
[0113] The beneficial effects of the present invention are:
[0114] (1) Space filling curves and position information encoding technology are used to process point cloud data, making the point cloud orderly, thereby making the extraction of local information more effective.
[0115] (2) The addition of the hybrid attention mechanism enables the network to adaptively adjust the weights of the areas and features that need attention, making feature extraction more targeted.
[0116] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A point cloud feature extraction method based on the position relationship of local point clouds, Features: The following steps are involved: S1: Obtain original point cloud data from a public 3D point cloud data set, encode and sort the original 3D point cloud data according to the encoding rule of the space filling curve - Z curve; The specific steps of step S1 are as follows: S11: The public 3D point cloud data set includes standard data sets modelnet40, shapenet, and S3DIS as the source of original point cloud data; S12: Encode all original point cloud data according to the encoding rule of the space filling curve—Zorder curve to obtain a Zorder value; S13: sorting the Zorder values in descending order to obtain the sequence numbers of the point cloud data in the Z curve, that is, the position information; S2: The sorted original point cloud data is encoded using the position information encoding technology of the sine and cosine functions, and the encoded data is fused with the original point cloud data to obtain the point cloud data after data enhancement; Step S2 is specifically as follows: S21: The point cloud data sorted by the Zorder value is encoded using the sine and cosine function position information encoding technology. The sine and cosine function position information encoding technology used is shown in Formula 1 and Formula 2: In the above formula, d model represents the dimension of the code, pos represents the coding order, PE(pos;2i) and PE(pos;2i+1) represent the even and odd dimension values of the code calculated using the sine function and cosine function respectively; S22: After encoding each original point cloud data using the position information encoding technology of the Zorder curve sequence, the obtained data is merged with the original point cloud data to form enhanced point cloud data; S3: Based on the Pointnet++ network structure, the point cloud data is grouped and sampled, and then the sampling layer and the multi-layer perceptron layer are iteratively used to extract features on each group, thereby constructing an iterative feature extraction network model; Step S3 is specifically as follows: S31: Based on the Pointnet++ network structure, the average sampling method is used to sample key points; The key point sampling process is as follows: Assume that the point cloud data set obtained after sorting according to the Zorder value is The superscript {1,2,…,N} represents the sequence of point cloud data after Zorder curve encoding and sorting, that is, the logical distance between points. If x is randomly selected n is a key point, then the next key point is x (n+d)%N , d = N / M, N is the total amount of point cloud data, M is the number of key points sampled; S32: After sampling the key points, the ball neighborhood method is used to construct the local neighborhood of each key point by grouping, that is, all points in the ball neighborhood formed by taking each key point as the origin and r as the radius are classified into a group of local neighborhood points to obtain a grouping layer; the grouping layer inputs a point set of size M×(3+C), where M is the number of key points, 3 represents the coordinate dimension of the point cloud, and C represents the feature dimension of the key point. The output size of the grouping layer is M×K×(3+C), where K represents the number of neighboring points of each key point; S33: In order to extract the local neighborhood features of each key point, each key point is first transformed into a three-dimensional local coordinate system with the key point as the origin, and then a PointNet multi-layer perceptron is used to extract features for each point in the local area, that is, K points in the local area are input into the PointNet multi-layer perceptron to obtain K features of size 1×(3+C′), where C′ is the new feature dimension obtained. Finally, the features of the K points are aggregated using the maximum pooling method to finally obtain a point cloud feature of size 1×(3+C′), which represents the local neighborhood features of the key point. S34: Alternately use sampling layers and PointNet multi-layer perceptron layers, stacking to form an iterative feature extraction network model; S4: In combination with the attention mechanism, point cloud features are extracted from the point cloud data in the local neighborhood of the key point in the feature extraction network model to obtain a point cloud feature vector of the local neighborhood; S5: Apply the point cloud feature vector extraction method of steps S1-S4 to the point cloud classification and segmentation tasks respectively to obtain specific point cloud classification results, and compare and display the results.
2. A point cloud feature extraction method based on local point cloud position relationship as claimed in claim 1, Features: Step S4 is specifically as follows: S41: Input an intermediate feature map F∈R into the feature extraction network model after applying the local area attention mechanism C×H×W , through the one-dimensional channel attention M c ∈R C×1×1 and two-dimensional spatial attention M s ∈R 1×H×W ; With the feature map F = H × W × C as input, we first perform a spatial global average pooling and a maximum pooling to obtain two 1 × 1 × C channel descriptions, and then send them to a two-layer neural network. After adding the two features output by the neural network, we obtain the weight coefficient Mc through a Sigmoid activation function. Finally, we multiply the weight coefficient with the original feature F to obtain the scaled new feature F': S42: After channel attention, given a H×W×C feature F', first perform average pooling and maximum pooling in a channel dimension to obtain two H×W×1 channel descriptions, then pass through a 7×7 convolution layer with a Sigmoid activation function to obtain the weight coefficient Ms, and finally, multiply the weight coefficient and feature F' to obtain the scaled new feature F":
3. A point cloud feature extraction method based on local point cloud position relationship as claimed in claim 1, Features: In step S5, when applied to the point cloud classification task, a classification network needs to be constructed. The input of the classification network is the feature vector obtained after processing in step S4, with a dimension of N×256. After the input feature is processed by a multi-layer perceptron with a network layer number of (256, 512, 1024), a feature vector with a dimension of 1024×1 is obtained. Then, it passes through the fully connected layers of (1024, 512), (512, 256,), and (256, k) in sequence. Each fully connected layer is processed with the relu function, and finally the score of each category is output. The obtained score is the probability that each sample belongs to each category. The classification network uses the cross entropy loss function as the objective function of point cloud classification. The calculation formula of the loss function is shown in Formula 5: Where M is the total number of categories, y c is a categorical indicator variable, which is 1 if the predicted category is the same as the true category, otherwise it is 0. c The model predicts the probability of belonging to category c.
4. A point cloud feature extraction method based on local point cloud position relationship as claimed in claim 1, Features: In step S5, when applied to the point cloud segmentation task, a segmentation network needs to be constructed. The design of the segmentation network adopts a cascade method to merge the feature vector obtained by the position information encoding method of the Zorder curve sequence with the features extracted from the subsequent feature extraction, and finally outputs the prediction probability of each point; The loss function of the point cloud component segmentation task is defined as shown in Formula 6: p ij To convert point x i The probability of predicting semantic category j, y j is the true semantic category of the point; only when j = y i Time ij =1, otherwise y ij = 0, n is the total number of points in the model; w j is the weight coefficient, which indicates the proportion of points with semantic category j in all points in the dataset; The loss function of the point cloud indoor scene segmentation task is defined as shown in Formula 8: Where A is the spatial transformation matrix, I is the unit matrix; n is the total number of points in the model, i is the i-th point cloud, represents the regularization term, j is the sample semantic category, y j is the true semantic category of the point, only when j = y i When the one-hot vector y ij =1, otherwise the one-hot vector y ij =0.
