A point cloud semantic segmentation method based on random sampling

Abstract

The present invention discloses a point cloud semantic segmentation method based on random sampling, which performs semantic segmentation on a point cloud composed of any number of three-dimensional coordinate points to distinguish point clouds in different parts. The method structure described in the present invention consists of five parts: a point cloud preprocessing part, a feature pre-extraction part, a downsampling part, an upsampling part, and a classification prediction part. It involves the application and improvement of the following technical contents, including the RandLA‑Net algorithm, the T‑Net algorithm, and the kdTree algorithm. The present invention performs rapid semantic segmentation on large-scale point clouds based on random sampling, has high accuracy and fast computing speed, and has significant advantages over similar algorithms.

Description

Technical Field

[0001] The present invention belongs to the field of point cloud processing, and in particular relates to a point cloud semantic segmentation method based on random sampling. Background Art

[0002] In the field of semantic segmentation of point clouds, recent research has focused on how to solve the sparsity problem of LiDAR point clouds at long distances. This sparsity greatly reduces the semantic segmentation accuracy of point clouds. In recent years, state-of-the-art point cloud networks such as KpConv, PointCNN, PointTransformer, and Swin3D have performed well in large scenarios such as autonomous driving. However, in small-scale scenarios, accurate segmentation of objects with more complex shapes has rarely been studied, such as segmenting different objects held by a person's hand.

[0003] The random sampling-based RandLA-Net achieves fast and accurate semantic segmentation results on large-scale points captured by LiDAR. However, this method ignores the rotation diversity of small objects in small-scale scenes. Therefore, it is necessary to design a point cloud semantic segmentation method based on random sampling for accurate segmentation of complex objects in small-scale scenes. Summary of the invention

[0004] To solve the above problems, the present invention discloses a point cloud semantic segmentation method based on random sampling. On the one hand, based on the local feature extraction process of RandLA-Net, the rotation diversity of the object is generalized by adding the transformation matrix predicted by T-Net; on the other hand, based on the kdTree algorithm, the point cloud is sampled several times quickly by the center neighbor, and a point cloud semantic segmentation method based on random sampling is designed. Experiments show that this method has significant advantages over similar methods in the accurate segmentation of complex objects in small-scale scenes.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] (1) For a set of input point clouds to be segmented (N, 3), the input point clouds are randomly sampled into sampling point clouds (N d ,3),The size of the sampling point cloud is a fixed value.

[0007] The input point cloud has a size of (N, 3), which means it is composed of N three-dimensional coordinates, where N can be any value.

[0008] The sampling point cloud randomly samples the input point cloud into a sampling point cloud by a random sampling method. The size of the sampling point cloud is (N d , 3), where N d It is a preset fixed value. For N<N d In such cases, sampling can be repeated.

[0009] (2) For the sampling point cloud in step (1), use the kdTree algorithm to perform center neighbor sampling and obtain the center neighbor point index D 1 , D 2 , ..., D n , and the upsampling index U 1 , U 2 , ..., U n .

[0010] The central neighbor point index is significant in that for the n times of downsampling in the downsampling part of the present invention, the corresponding central neighbor point index needs to be input. The size of each central neighbor point index is as follows:

[0011] D 1 , D 2 , ..., D n =(I 1 , K 1 ), (I 2 , K 2 )...(I n , K n ) Formula 1

[0012] I in formula 1 n , K n Represents the number of downsamplings in the downsampling part. n K of the center points n The neighboring points in the sampling point cloud (N d , 3). The result of querying the center neighbor point index when the 1st dimension is 0 represents I n The center point itself is in the sampling point cloud (N d ,3) indices. Note that for D i+1 The index is based on D i The index is the result of the query in the sample point cloud, so D i+1 All point clouds indexed should be D i The index contains all the point clouds.

[0013] The upsampling index is significant in that for each of the n upsamplings in the upsampling part of the present invention, the corresponding upsampling index needs to be input. The size of each upsampling index is as follows:

[0014] U 1 , U 2 , ..., U n =(I n-1 , 1), (I n-2 , 1), ...(I 1 , 1), (N d,1) Formula 2

[0015] The upsampling index is used to query the index of the nearest point of a low-size point cloud in a high-size point cloud, so that the high-dimensional low-resolution tensor after downsampling can be spliced ​​to the low-dimensional high-resolution tensor, while the high-dimensional features are preserved while restoring the point cloud size. It should be noted that each upsampling index is based on the index of each center neighbor point, so the upsampling index should be obtained after the center neighbor point index.

[0016] (3) For the sampled point cloud described in step (1), a set of pre-features (N d , d), pass the pre-features through T-Net to obtain a set of transformation matrices (d, d), perform matrix multiplication on the pre-features and the transformation matrix to obtain the transformed features (N d , d).

[0017] (4) For the transformed features (N) in step (3) d , d), using the center neighbor point index in step (2), perform n downsampling in sequence, and get the result (I n , 4 n d). This step aims to reduce the point cloud size and increase the feature dimension by aggregating features from the neighborhood of each center point to maximize the perception of the local geometric features of the point cloud. This module consists of two core components: the local space encoding module and the attention pooling module. The attention pooling module is directly connected to the local space encoding module. A downsampling includes the following sub-steps:

[0018] (4.1) The input point cloud (N d , d) The size feature and the corresponding three-dimensional coordinates are combined into (N d , 3+d), where 3 represents the 3D coordinate.

[0019] (4.2) Substitute (N) in (4.1) d , 3+d) point cloud features are obtained through a Shared MLP (N d Based on the corresponding center neighbor point index (I, K), the query results are obtained: the center neighbor point coordinates of size (I, K, 3) and the center neighbor point features of size (I, K, 2d).

[0020] (4.3) The coordinates of the center neighbor points in (4.2) are encoded according to formula 3 to obtain local encoding features of size (I, K, 10), and then passed through a Shared MLP to obtain local encoding features of size (I, K, 2d). The local encoding features are concatenated with the center neighbor point features of (I, K, 2d) to obtain local features of (I, K, 4d).

[0021] (4.4) The local feature of (I, K, 4d) in (4.3) is passed through a Shared MLP to obtain the feature weight (I, K, 1), and then multiplied by the weight to obtain the weighted feature of (I, K, 4d). The weighted feature is summed and pooled in the middle dimension to obtain the local feature of (I, 4d).

[0022] (4.5) The local features in (4.4) are passed through a Shared MLP to obtain the output features (I, 4d), which are then concatenated with their corresponding three-dimensional coordinates and used as the input for the next downsampling process.

[0023]

[0024] In Formula 3, C R , C c , C n Respectively represent the relative coordinates of a neighboring point and its center point, the coordinates of its center point, and the coordinates of the neighboring point, and r represents the distance from a neighboring point to the center point. Therefore, the spatial encoding will occupy 3+3+3+1=10 dimensions, so that it outputs a local encoding feature of size (I,K,10) in (4.3).

[0025] (5) The downsampled output features (I n , 4 n d), based on the upsampling index in step (2), upsampling is performed n times in sequence to obtain the result (N d , d). The upsampling process combines the smaller-sized high-dimensional point cloud features with the larger-sized low-dimensional point cloud features, with the aim of providing a basis for semantic segmentation for each point. An upsampling process includes the following sub-steps:

[0026] (5.1) The output point cloud (I n-i , d) and the i-1th downsampled output point cloud (I n-i+1 , 4d), and the corresponding upsampling index (I n-i+1 , 1) as input.

[0027] (5.2) Substitute (I) in (5.1) n-i+1 , 4d) point cloud features, through its (I n-i+1 , 1) upsampling index to query, and get (I n-i , 4d) and compared with the output point cloud of the i-th downsampling (I n-i , d) are spliced ​​to obtain (I n-i , point cloud features of 5d).

[0028] (5.3) Replace (I in (5.2) n-i, 5d) point cloud features after a Shared MLP, get (I n-i , d)’s output point cloud features are used as the input for the next upsampling.

[0029] (6) For step (5) (N d , d) after the upsampling result, after passing through multiple layers of fully connected layers, through the Softmax activation function, we get (N d , k), where k represents the number of categories to be classified. At this point, the semantic segmentation task is completed. That is, for each point in the point cloud, its category is predicted.

[0030] The method structure of the present invention is composed of five parts: point cloud preprocessing part, feature pre-extraction part, downsampling part, upsampling part, and classification prediction part. Steps (1) and (2) correspond to the point cloud preprocessing part of the present invention; step (3) corresponds to the feature pre-extraction part; step (4) corresponds to the downsampling part; step (5) corresponds to the upsampling part; and step (6) corresponds to the classification prediction part.

[0031] The beneficial effects of the present invention are:

[0032] (1) Through the steps of point cloud preprocessing, feature pre-extraction, downsampling, upsampling, classification prediction, etc., the point cloud composed of any number of three-dimensional coordinate points is quickly semantically segmented. It has high accuracy and fast computing speed, which has significant advantages over similar algorithms.

[0033] (2) The present invention improves the RandLA-Net algorithm, T-Net algorithm, and kdTree algorithm. Based on the local feature extraction process of RandLA-Net, the rotation diversity of objects is generalized by adding the transformation matrix predicted by T-Net. In addition, based on the kdTree algorithm, the present invention performs several fast center neighbor samplings on the point cloud and designs a point cloud semantic segmentation method based on random sampling. Experiments show that this method has significant advantages over similar methods in the accurate segmentation of complex objects in small-scale scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0035] Figure 1 This is a flow chart of the method for semantic segmentation of point clouds based on random sampling.

[0036] Figure 2 It is a downsampling flowchart based on the local spatial encoding module and the attention pooling module.

[0037] Figure 3 This is a flowchart of upsampling.

[0038] Figure 4 This is a point cloud semantic segmentation effect diagram, where darker points are classified as hands and lighter points are classified as objects.

[0039] Figure 5 is the set of objects used for training in generalization experiments.

[0040] Figure 6 is the set of objects used for testing in the generalization experiment. DETAILED DESCRIPTION

[0041] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0042] In comparison with steps (1) to (6) in the Summary of the Invention, detailed description is given with reference to the following conditions:

[0043] (a) The sampling point cloud (N) in step (1) of the invention d ,3), the size of its point cloud is 1200, that is, N d =1200

[0044] (b) The center neighbor point index D in step (2) of the invention 1 , D 2 , ..., D ｎ and upsampling index U 1 , U 2 , ..., U n , where n = 4, I 1 , I 2 , I 3 , I 4 =384,192,96,24, and K 1 , K 2 , K 3 , K 4 =20,15,10,5

[0045] (c) In the step of invention, d = 8 representing the feature dimension

[0046] (d) Summary of the Invention In step (6), the number of categories required for classification is 2, that is, k = 2. In a specific implementation, taking the point cloud of a hand holding an object as an example, the part of the input point cloud belonging to the object and the part belonging to the hand are classified.

[0047] Referring to the conditions in (a)-(d), a point cloud semantic segmentation method based on random sampling has the following process: Figure 1 As shown, the specific implementation is as follows:

[0048] (1) For a set of input point clouds (N, 3) to be segmented, the input point clouds are randomly sampled into sampling point clouds (1200, 3), and the size of the sampling point clouds is a fixed value.

[0049] The input point cloud has a size of (N, 3), which means it is composed of N three-dimensional coordinates, where N can be any value.

[0050] The sampling point cloud randomly samples the input point cloud into a sampling point cloud by a random sampling method. The size of the sampling point cloud is (N d , 3), where N d It is a pre-set fixed value. For the case where N<1200, sampling can be repeated.

[0051] (2) For the sampling point cloud in step (1), use the kdTree algorithm to perform center neighbor sampling and obtain the center neighbor point index D 1 , D 2 , D 3 , D 4 , and the upsampling index U 1 , U 2 , U 3 , U 4 .

[0052] The central neighbor point index is significant in that for the n times of downsampling in the downsampling part of the present invention, the corresponding central neighbor point index needs to be input. The size of each central neighbor point index is as follows:

[0053] D 1 , D 2 , D 3 , D 4 =(I 1 , K 1 ), (I 2 , K 2 ), (I 3 , K 3 ), (I 4 , K 4 ) Formula 4

[0054] I in formula 4 i , K i Represents the number of downsamplings in the downsampling part. i K of the center points i The index of the neighboring point in the sampling point cloud (1200, 3), where I 1 , I 2 , I 3 , I 4 =384,192,96,24, and K 1 , K2 , K 3 , K 4 =20, 15, 10, 5. The result of querying the center neighbor point index when the 1st dimension is 0 represents I i The index of the center point itself in the sampling point cloud (1200, 3). Note that for D i+1 The index is based on D i The index is the result of the query in the sample point cloud, so D i+1 All point clouds indexed should be D i The index contains all the point clouds.

[0055] The upsampling index is significant in that for each of the n upsamplings in the upsampling part of the present invention, the corresponding upsampling index needs to be input. The size of each upsampling index is as follows:

[0056] U 1 , U 2 , U 3 , U 4 =(I 3 , 1), (I 2 , 1), (I 1 , 1), (N d ,1) Formula 5

[0057] The upsampling index is used to query the index of the nearest point of a low-size point cloud in a high-size point cloud, so that the high-dimensional low-resolution tensor after downsampling can be spliced ​​to the low-dimensional high-resolution tensor, while the high-dimensional features are preserved while restoring the point cloud size. It should be noted that each upsampling index is based on the index of each center neighbor point, so the upsampling index should be obtained after the center neighbor point index.

[0058] (3) For the sampled point cloud described in step (1), a set of pre-features (1200, 8) is extracted through Shared MLP, and the pre-features are passed through T-Net to obtain a set of transformation matrices (8, 8). The pre-features are matrix multiplied with the transformation matrix to obtain the transformed features (1200, 8).

[0059] (4) For the transformed feature (1200,8) in step (3), use the center neighbor point index in step (2) to perform four downsamplings in sequence to obtain the result (I 4 , 256×8). This step aims to reduce the point cloud size and increase the feature dimension by aggregating features from the neighborhood of each center point to maximize the perception of the local geometric features of the point cloud. This module consists of two core components: the local space encoding module and the attention pooling module. The attention pooling module is directly connected to the local space encoding module. The downsampling step is as follows Figure 2As shown. A downsampling process consists of the following steps:

[0060] (4.1) The input point cloud (N d , d) The size feature and the corresponding three-dimensional coordinates are combined into (N d , 3+d), where 3 represents the 3D coordinate.

[0061] (4.2) Substitute (N) in (4.1) d , 3+d) point cloud features are obtained through a Shared MLP (N d Based on the corresponding center neighbor point index (I, K), the query results are obtained: the center neighbor point coordinates of size (I, K, 3) and the center neighbor point features of size (I, K, 2d).

[0062] (4.3) The coordinates of the center neighbor points in (4.2) are encoded according to formula 3 to obtain local encoding features of size (I, K, 10), and then passed through a Shared MLP to obtain local encoding features of size (I, K, 2d). The local encoding features are concatenated with the center neighbor point features of (I, K, 2d) to obtain local features of (I, K, 4d).

[0063] (4.4) The local feature of (I, K, 4d) in (4.3) is passed through a Shared MLP to obtain the feature weight (I, K, 1), and then multiplied by the weight to obtain the weighted feature of (I, K, 4d). The weighted feature is summed and pooled in the middle dimension to obtain the local feature of (I, 4d).

[0064] (4.5) The local features in (4.4) are passed through a Shared MLP to obtain the output features (I, 4d), which are then concatenated with their corresponding three-dimensional coordinates and used as the input for the next downsampling process.

[0065]

[0066] In Formula 3, C R , C c , C n Respectively represent the relative coordinates of a neighboring point and its center point, the coordinates of its center point, and the coordinates of the neighboring point, and r represents the distance from a neighboring point to the center point. Therefore, the spatial encoding will occupy 3+3+3+1=10 dimensions, so that it outputs a local encoding feature of size (I,K,10) in (4.3).

[0067] (5) The downsampled output features (I 4, 256×8), and based on the upsampling index in step (2), upsampling is performed four times in sequence to obtain the result (1200,8). The upsampling process combines the high-dimensional point cloud features of a smaller size with the low-dimensional point cloud features of a larger size, in order to provide a basis for semantic segmentation for each point. The upsampling process is as follows Figure 3 As shown. One upsampling includes the following steps:

[0068] (5.1) The output point cloud (I 4-i , d) and the i-1th downsampled output point cloud (I 4-i+1 , 4d), and the corresponding upsampling index (I 4-i+1 , 1) as input.

[0069] (5.2) Replace (I) in (5.1) 4-i+1 , 4d) point cloud features, through its (I 4-i+1 , 1) upsampling index to query, and get (I n-i , 4d) and compared with the output point cloud of the i-th downsampling (I 4-i , d) are spliced ​​to obtain (I 4-i , point cloud features of 5d).

[0070] (5.3) Replace (I in (5.2) 4-i , 5d) point cloud features after a Shared MLP, get (I 4-i , d)’s output point cloud features are used as the input for the next upsampling.

[0071] (6) For the upsampling result of (1200,8) in step (5), after passing through multiple layers of fully connected layers, the Softmax activation function is used to obtain the semantic segmentation result of (1200,2), where 2 represents the number of categories required for classification. At this point, the semantic segmentation task is completed. That is, for each point in the point cloud, its category is predicted.

[0072] In a specific embodiment, the effect of the method of the present invention is verified by experiments. The experiments include generalization ability experiments and performance comparison experiments. The experimental system is set up as follows:

[0073] 1. In terms of hardware, it is based on Kinect2 depth camera and RTX 2080Ti GPU.

[0074] 2. In terms of software, the above functional code is written based on python 3.9 and pytorch 1.12.1.

[0075] 3. In the training phase of the neural network, the neural network of the present invention is trained by thousands of point clouds containing different objects grasped by human hands. The training process is based on the adaptive gradient Adam optimizer.

[0076] Its specific implementation effects are as follows: Figure 4 As shown, the darker points are classified as hands, while the lighter points are classified as objects.

[0077] In the generalization ability experiment, based on Figure 5 The method of the present invention is trained based on the object set of Figure 6 The method of the present invention is tested on a set of objects. The experiment shows that the method of the present invention has excellent generalization ability, excellent classification effect on unknown objects, and is suitable for application under complex conditions. The experimental results are shown in Table 1.

[0078] Table 1

[0079] Test of generalization ability to unknown objects

[0080]

[0081] In the performance comparison experiment, similar methods were tested based on the same application scenario. The experiment shows that the method described in the present invention has the advantages of high accuracy and low computational complexity in this application scenario. The experimental results are shown in Table 2.

[0082] Table 2

[0083] Performance comparison experiment

[0084]

[0085] Document 1 in Table 2 is:

[0086] Qi CR, Yi L, Su H, et al., "Pointnet++: Deep hierarchical featurelearning on point sets in ametric space," Advances in neural informationprocessing systems, 2017, Vol.30.

[0087] Document 2 is:

[0088] Hu Q, Yang B, Xie L, et al., "Randla-net: Efficient semantic segmentation of large-scale point clouds," In Proceedings of the IEEE / CVF conference on computer vision and pattern recognition., 2020, pp. 11108-11117.

[0089] Reference 3 is:

[0090] Qiu S, Anwar S, Barnes N., "Semantic segmentation for real point cloud scenes via bilateral augmentation and adaptive fusion," In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition., 2021, pp. 1757-1767.

[0091] Reference 4 is:

[0092] Wu X, Lao Y, Jiang L, et al., "Point transformer v2: Grouped vector attention and partition-based pooling," In Advances in Neural Information Processing Systems, 2022, Vol. 35, pp. 33330-33342.

[0093] Reference 5 is:

[0094] Yang Y Q, Guo Y X, Xiong J Y, et al., "Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene Understanding," arXiv preprint arXiv:2304.06906, 2023.

[0095] The above-mentioned embodiments only express certain implementation methods of the present application, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of this application shall be subject to the attached claims.

Claims

1. A point cloud semantic segmentation method based on random sampling, characterized in that: The method comprises the following steps: (1) For a set of input point clouds to be segmented (N, 3), the input point clouds are randomly sampled into sampling point clouds (N d ,3),The size of the sampling point cloud is a fixed value; (2) For the sampling point cloud in step (1), the kdTree algorithm is used to perform center neighbor sampling and obtain the center neighbor point indexes D1, D2, ..., D n , and upsampling indices U1, U2, ..., U n ; (3) For the sampled point cloud described in step (1), a set of pre-features (N d , d), pass the pre-features through T-Net to obtain a set of transformation matrices (d, d), perform matrix multiplication on the pre-features and the transformation matrix to obtain the transformed features (N d , d); (4) For the transformed features (N) in step (3) d , d), using the center neighbor point index in step (2), perform n downsampling in sequence, and get the result (I n , 4 n d); (5) The downsampled output features (I n , 4 n d), based on the upsampling index in step (2), upsampling is performed n times in sequence, and the high-dimensional point cloud features of the smaller size are merged with the low-dimensional point cloud features of the larger size to obtain the result (N d , d); (6) For step (5) (N d , d) after the upsampling result, after passing through multiple layers of fully connected layers, through the Softmax activation function, we get (N d , k), where k represents the number of categories required for classification; at this point, the semantic segmentation task is completed; that is, for each point in the point cloud, its category is predicted.

2. The point cloud semantic segmentation method based on random sampling according to claim 1, characterized in that: In step (1), The input point cloud has a size of (N, 3), which means it is composed of N three-dimensional coordinates, where N is an arbitrary value; The sampling point cloud randomly samples the input point cloud into a sampling point cloud by a random sampling method. The size of the sampling point cloud is (N d , 3), where N d It is a preset fixed value. For N<N d In this case, repeated sampling is performed.

3. The point cloud semantic segmentation method based on random sampling according to claim 1, characterized in that: In step (2), The size of the index for each center neighbor point is as follows: D1,D2,...,D n =(I1, K1),(I2, K2)...(I n , K n ) Formula 1 I in formula 1 n , K n Represents the number of downsamplings in the downsampling part. n K of the center points n The neighboring points in the sampling point cloud (N d , 3) in the index; the result of querying the center neighbor point index when the 1st dimension is 0, representing I n The center point itself is in the sampling point cloud (N d ,3) in the index; for D i+1 The index is based on D i The index is the result of the query in the sample point cloud, so D i+1 All point clouds indexed by D i The index contains all point clouds; The dimensions for each upsampling index are as follows: U1,U2,...,U n = (I n-1 , 1),(I n-2 ,1)...(I1,1),(N d ,1) Formula 2 The upsampling index is designed to query the index of the nearest point of a low-size point cloud in a high-size point cloud, so that the high-dimensional low-resolution tensor after downsampling can be spliced ​​to the low-dimensional high-resolution tensor, while restoring the point cloud size and retaining the high-dimensional features; each upsampling index is based on the index of each central neighbor point, so the upsampling index is obtained after the central neighbor point index.

4. The point cloud semantic segmentation method based on random sampling according to claim 1, characterized in that: The purpose of step (4) is to reduce the point cloud size and increase the feature dimension by aggregating features from the neighborhood of each center point to maximize the perception of the local geometric features of the point cloud. Downsampling includes two parts: local space encoding and attention pooling; attention pooling is performed after local space encoding; a downsampling in step (4) includes the following sub-steps: (4.1) The input point cloud (N d , d) The size feature and the corresponding three-dimensional coordinates are combined into (N d , 3+d), where 3 represents the 3D coordinate; (4.2) Substitute (N) in (4.1) d , 3+d) point cloud features are obtained through a Shared MLP (N d , 2d) point cloud features; Based on the corresponding center neighbor point index (I, K), the query results are obtained: the center neighbor point coordinates of size (I, K, 3) and the center neighbor point features of size (I, K, 2d); (4.3) The coordinates of the center neighbor points in (4.2) are encoded according to formula 3. In Formula 3, C R , C c , C n Respectively represent the relative coordinates of a neighboring point and its center point, the coordinates of its center point, and the coordinates of the neighboring point, and r represents the distance from a neighboring point to the center point; therefore, the spatial encoding will occupy 3+3+3+1=10 dimensions, and obtain the local encoding features of size (I,K,10), and pass it through a Shared MLP to obtain the local encoding features of size (I,K,2d); the local encoding features are concatenated with the central neighbor point features of (I,K,2d) to obtain the local features of (I,K,4d); (4.4) The local features of (I, K, 4d) in (4.3) are passed through a Shared MLP to obtain the feature weight (I, K, 1), and then multiplied by the weight to obtain the weighted features of (I, K, 4d); the weighted features are summed and pooled in the middle dimension to obtain the local features of (I, 4d); (4.5) The local features in (4.4) are passed through a Shared MLP to obtain the output features (I, 4d), which are then concatenated with their corresponding three-dimensional coordinates and used as the input for the next downsampling process.

5. The point cloud semantic segmentation method based on random sampling according to claim 1, characterized in that: The purpose of step (5) is to provide a basis for semantic segmentation for each point; in step (5), one upsampling includes the following sub-steps: (5.1) The output point cloud (I n-i , d) and the i-1th downsampled output point cloud (I n-i+1 , 4d), and the corresponding upsampling index (I n-i+1 , 1) as input; (5.2) Replace (I) in (5.1) n-i+1 , 4d) point cloud features, through its (I n-i+1 , 1) upsampling index to query, and get (I n-i , 4d) and compared with the output point cloud of the i-th downsampling (I n-i , d) are spliced ​​to obtain (I n-i , 5d) point cloud features; (5.3) Replace (I in (5.2) n-i , 5d) point cloud features after a Shared MLP, get (I n-i , d)’s output point cloud features are used as the input for the next upsampling.

Images