Three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism

Abstract

A three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism adopts a network structure in the style of U-Net, performs feature extraction and propagation on the point cloud through multi-level feature encoding and decoding, and first centralizes and standardizes local neighborhood features through a geometric affine module at each level to enhance the model's ability to capture local area diversity; secondly, a local neighborhood graph is constructed, an asymmetric edge feature learning function is designed to extract local topological information; further deep edge features are extracted in combination with a residual network; finally, a grouping vector attention mechanism is used to selectively fuse local features and global context information under the condition of low parameter quantity, and the point cloud segmentation is completed through cross-level feature interpolation up-sampling to the original scale. The present application can significantly improve the point cloud segmentation precision and is suitable for the perception of complex scenes by devices such as mobile robots equipped with three-dimensional sensors.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of cloud data processing and environment perception, and particularly relates to a three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism. BACKGROUND

[0002] Point cloud is a common three-dimensional data representation form composed of a large number of irregular points in space, which can be used to perceive and represent the geometry and structure of the real world, and is a bridge connecting the physical world and the digital world. With the rapid development of sensor technology (such as lidar, depth camera and 3D scanning device, etc.), point cloud data has been widely applied in many fields. In autonomous driving, point cloud can be used to accurately detect and identify target objects (such as vehicles, pedestrians, etc.) on the road, providing key support for safe driving; in robot navigation, point cloud data can efficiently capture environmental obstacles, terrain features and object information, providing key reference for robot path planning and obstacle avoidance tasks; in three-dimensional reconstruction, point cloud is widely used to convert physical scenes into three-dimensional digital models, helping cultural heritage protection, architectural design and virtual reality applications. However, due to the characteristics of point cloud data such as noise, disorder, uneven density, sparsity, etc., how to efficiently and accurately classify and segment on this basis is a complex and challenging task. The solution to these problems is of great significance to promote the application of point cloud technology in complex scenes.

[0003] Currently, the development of point cloud segmentation technology mainly relies on deep learning methods, which can be divided into the following four types: projection-based methods, discretization-based methods, hybrid-based methods, and point-based methods. Projection-based methods project three-dimensional point clouds onto two-dimensional planes, converting three-dimensional problems into two-dimensional image processing tasks, and using existing convolutional neural networks for processing. However, due to the compression of three-dimensional point clouds into two-dimensional space, some spatial information may be lost when dealing with complex geometric structures. Discretization-based methods map point cloud data to a three-dimensional voxel grid, forming a structure similar to a three-dimensional image, and then use a three-dimensional convolutional neural network (3D CNN) for processing. However, in high-resolution point clouds, the discretization process may result in excessive computational load and storage requirements, making it difficult to balance resolution and computational efficiency, and the discretization process may also cause loss of local geometric information. Hybrid-based methods combine the characteristics of projection and discretization, allowing the use of both two-dimensional and three-dimensional feature information to improve model perception and generalization ability, but the network structure is complex and requires high computational resources. In 2017, Qi et al. from Stanford University first proposed the point-based method PointNet, which directly operates on irregular point clouds and learns their features point by point. PointNet++ was proposed the same year, which further improved point cloud processing performance through multi-scale recursive local feature extraction. In the following years, Wang et al. proposed a dynamic graph convolutional network EdgeConv to generate edge features describing points and their neighbors, preserving local neighborhood relationships. Thomas et al. proposed KPConv, which introduced a variable convolution kernel to improve the flexibility and expressiveness of point cloud processing. Recently, Zhao et al. proposed Point Transformer, which uses attention mechanisms to capture contextual information. Ma et al. proposed PointMLP, which abandoned complex local feature extraction modules and used lightweight residual networks to significantly improve the inference speed of point cloud segmentation without sacrificing accuracy.

[0004] In summary, although current point cloud processing methods have made some progress in classification and segmentation tasks, there are still challenges in efficiently and accurately extracting local geometric features and global features from point clouds in complex environments, as well as overcoming sparsity and other issues. Traditional methods are often limited by information loss or excessive computational resource consumption when dealing with complex geometric structures. While point-based deep learning methods perform well in handling sparse and irregular point clouds, there is still room for improvement in the fusion of local features and global contextual information. SUMMARY

[0005] In order to overcome the deficiency that the existing point cloud segmentation technology is difficult to capture complex local geometric structure and global context information, thereby causing the precision of segmentation to be limited, the application provides a three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism, which provides a new solution for high-precision point cloud segmentation. The method takes a U-Net style feature encoding and decoding network as a framework, combines the farthest point sampling and cross-level feature interpolation method, and innovatively uses a graph neural network to extract edge features of a local neighborhood, thereby enhancing the expression ability of the model to local topological information. By introducing a residual network with affine transformation, deep edge features are further extracted, and the perception ability of the model to complex geometric structures is significantly improved. Meanwhile, a grouping vector attention mechanism is used to realize selective fusion of local features and global context information with low parameter quantity, thereby effectively improving the precision and efficiency of point cloud segmentation.

[0006] The technical scheme adopted by the application to solve the technical problems is:

[0007] A three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism, comprising the following steps:

[0008] Step one, preprocessing the original point cloud data, including feature enhancement, class weight balance, sampling, batch construction and feature standardization, considering a point cloud data set with N points, represented as X={x i |i=1,…,N} wherein each point x i =(p i ,f i ) includes spatial coordinates and a feature vector The size of D depends on the sensor type; the laser radar scanning obtains three-dimensional point cloud data in a Cartesian coordinate system, that is, each point has only three basic dimensions p i =(x i ,y i ,z i ), respectively representing the position coordinates of the point, and the feature vector f i used in the data set is an RGB color feature;

[0009] Step two, local neighborhood graph construction, including downsampling, grouping, affine transformation and graph structure modeling; first, representative points are uniformly selected from the original point cloud by farthest point sampling (FPS), reducing data complexity while preserving geometric features; then, a local neighborhood is defined for each sampling point, and its nearest neighborhood point set is obtained using the k-nearest neighbor algorithm; then, a geometric affine module is introduced to decenter and standardize the neighborhood features to overcome the sparse and irregular geometry of the local area of the point cloud; finally, by constructing a local neighborhood graph, designing an asymmetric edge feature learning function, combining global features and local differences, explicitly modeling the local topology and global relationship of the point cloud, and fully extracting edge features;

[0010] Step three, local neighborhood graph feature extraction based on residual network and context feature fusion based on grouped vector attention, based on the local neighborhood graph, first, edge features are extracted by a multi-layer residual network with shared weights, and after the edge features are aggregated by max-pooling operation, the local neighborhood graph features are formed by concatenating the center point features; then, using the Transformer structure with residual connection, the attention weight of the feature channel is dynamically allocated through the grouped vector attention (GAV) mechanism, which flexibly captures key edge features, and the relative position encoding is introduced to learn the absolute position relationship of the neighborhood point set;

[0011] Step four, hierarchical upsampling feature decoding, including inverse distance weighted average interpolation and feature propagation, completing segmentation output and backpropagation; the downsampling, grouping, affine transformation, local feature extraction and context feature fusion modules together form a feature abstraction layer, and after four layers of recursive feature abstraction, the low-resolution global point cloud feature vector is obtained, which contains rich context semantic information; Since point cloud segmentation is a dense classification task, it is necessary to classify the features of each point, so a structure is used that couples the feature encoder and the decoder, and the feature decoder includes four feature propagation modules, which cross-layer concatenate the sparse point set features containing high-dimensional semantic features to the previous layer point set by interpolation.

[0012] Further, the process of step one is:

[0013] Step 1.1: Use the point cloud processing library (PCL) to calculate the normal vector of the point cloud, introduce an additional feature dimension to enhance the representation ability of the original point cloud data, and use the normal vector information to provide more detailed local geometric features;

[0014] Step 1.2: Count the number of points in each class in the original point cloud data to obtain the frequency distribution vector And calculate the weight distribution vector according to it, as shown in the following formula:

[0015]

[0016] where C is the total number of semantic categories, i.e., the number of different categories in the dataset, h i is the number of points in the point cloud that belong to the i-th category;

[0017] The weight of each category is scaled by the maximum weight ratio and cubic root smoothing, as follows:

[0018]

[0019] where w j is the scaled weight of the j-th category, and w is the weight distribution vector;

[0020] Step 1.3: Since the distribution of different point cloud points is quite different, a part of points needs to be selected according to the point distribution during the training process, so the sampling probability p i is assigned to the i-th point cloud:

[0021]

[0022] where N i is the number of points in the i-th point cloud, and R is the total number of point clouds. According to the sampling probability, the sampling times M i of the i-th point cloud can be obtained as:

[0023]

[0024] where r s is the sampling rate, and N s is the number of points in a single sampling, both of which are set by hyperparameters;

[0025] Step 1.4: A point is randomly selected as the center point x c of the current point cloud (x c , y c , z c ), a cubic sampling block with a length and width of block_size is constructed, and sampling is completed in this area:

[0026]

[0027] where b is the size of a cubic sampling block block_size, (x i , y i ) is the horizontal and vertical coordinates of the i-th point x i , and when the number of points in the block is insufficient, random sampling with replacement is used to fill up to the number of points in a single sampling N s ;

[0028] In actual training, point cloud data is divided into several batches and input into the neural network one by one to increase optimization efficiency; a batch consists of batch_size sampling blocks, and each sampling block contains a fixed number of points num_point, which is numerically equal to N. s batch_size and num_point are hyperparameters set according to the GPU memory capacity to maximize GPU memory utilization while ensuring training stability.

[0029] Step 1.5: Perform coordinate normalization and decentering operations on the sampled point cloud, and also normalize the RGB values ​​to finally construct a point cloud with a feature vector dimension of 12. Each point x i eigenvector f i as follows:

[0030]

[0031] Where (x) c ,y c ,z c (n) represents decentralized coordinates. x ,n y ,n z (n) represents the normalized coordinates. R ,n G ,n B () represents the normalized RGB value. It is the normal vector.

[0032] The process of step two is as follows:

[0033] Step 2.1: Downsampling is used to select some representative points from a relatively dense point set as sampling points, representing the point cloud with fewer data points while ensuring that the topology of the point cloud remains unchanged; the process of farthest point sampling FPS is as follows:

[0034] 2.1.1) Given a point cloud A set of sample points S = {s} i |i=1,…,M}, initialize the candidate farthest point set S. t = {s1}, where s1 is a random selection and is removed from the original point set S;

[0035] 2.1.2) To select the next point as the farthest point p, the following constraints must be satisfied:

[0036]

[0037] This constraint means: with S tUsing the point closest to the median S as the reference, calculate the Euclidean distance between the midpoints of the two sets, and select the median S of S. t The largest point is taken as the farthest point, and this point is added to S. t Simultaneously, delete this point in S, where d(p,S) is the Euclidean distance between the farthest point p and the point set S, and d(q,s) is the distance between p and S. i ) is S t Midpoint q and midpoint s of S i The Euclidean distance between them is expressed as follows:

[0038]

[0039] Where, p t =(x n ,y n ,z n Let S be a set t The coordinate vector of the midpoint q, p = (x m ,y m ,z m ) is the midpoint s of set S i The coordinate vector, ||·||2 represents the 2-norm;

[0040] 2.1.3) Repeat steps 2.1.1) to 2.1.2) above to update the farthest point set S. t until S t The number of sampling points reaches N;

[0041] Step 2.2: To extract local features, it is necessary to define a neighborhood range for each sampling point in the point cloud. Given a sample set S = {s} within the point cloud X. i The subset S obtained by downsampling |i=1,…,M} is t ={s i For each sampling point in |i=1,…,N}, the k-nearest neighbor algorithm is used to search for the k nearest points to the centroid to form a grouped point set.

[0042] Step 2.3: Introduce a geometric affine module to perform decentralization and standardization processing on local features, assuming... To obtain the local neighborhood features containing k nearest neighbors, we perform an affine transformation on the features of the nearest neighbors using the following formula:

[0043]

[0044] in, and For learnable parameters, ⊙ represents element-wise multiplication, and ε = 10. -5σ is a parameter for numerical stability in the normalization operation. It is a scalar that describes the characteristic deviation between all local nearest neighbors and channels, as shown below:

[0045]

[0046] Step 2.4: For a local neighborhood of a point cloud, its essence is still an isolated and unordered set of points, lacking explicit local topological structure and global relationships. Therefore, a graph structure is constructed to model the local neighborhood. Consider a local region of a point cloud. Its corresponding directed graph representation is as follows Where V = {1, 2, ..., k} represents considering the k local nearest neighbors as vertices. If the edges are ε, then this local neighborhood graph can be represented as χ(i)={j:(i,j)∈E}∪{i}. This graph contains self-loops, meaning that each node also points to itself. The edge feature is defined as ε. ij =f Θ (x i ,x j ),in For a set of learnable parameters Θ, a nonlinear function f Θ Defined as follows:

[0047] ε ij =f Θ (x i ,x j )=f Θ (x i ,x j -x i (11)

[0048] This asymmetric edge feature learning function incorporates the center point x i The difference between captured global and relative features x j -x i Captured local neighborhood information.

[0049] In step three, the core operation of the feature extraction and fusion module is abstractly represented as follows:

[0050]

[0051] Where, Φ pre (·) is the local feature extraction function, A(·) is the aggregation function, and Φ pre (·) represents the context feature fusion function, and the process is as follows:

[0052] Step 3.1: For the local neighborhood graph structure, use the function Φ pre (·) Extract edge features. Φ pre(·) is an abstract function f in step 2.4 Θ (·) is a specific implementation of the mapping function, which is composed of a stack of three layers of shared-weight residual blocks ResP Block, and the residual block is two homogeneous residual MLP modules denoted as MLP(x) + x, each MLP module is composed of a fully connected layer, a batch normalization layer and an activation function, as shown in the following formula:

[0053] MLP(x i ,x j -x i )=ReLU(BN(θ m ·(x j -x i )+φ m ·x i )) (13)

[0054] where Θ=(θ1,…,θ M ,φ1,…,φ M ) is the learnable weight parameter in the fully connected layer (FC), BN(·) is the batch normalization operation, and ReLU=max(0,x) is a common nonlinear activation function in deep learning;

[0055] Step 3.2: Since point cloud data has no order, the network processing point cloud data needs to have permutation invariance, that is, it is not sensitive to the arrangement order of input points, therefore, in the aggregation process of edge features, the aggregation function A(·) adopts the max pooling operation, and the expression is:

[0056]

[0057] The aggregated edge feature ε i ′ j is concatenated with the center point feature x i to obtain the local neighborhood graph feature G;

[0058] Step 3.3: The neighborhood graph feature G is processed by the Φ pos (·) function to fuse local and global context information to obtain the context feature Φ pos (·) is a Transformer structure with residual connection, which adds a Point Transformer layer after the first MLP, which uses vector self-attention mechanism to flexibly assign attention weights to each channel of edge feature, so that the center point can dynamically capture the key edge features in the neighborhood graph;

[0059] Using the group vector attention operator GAV, first use linear projection or MLP to project the center point feature x i and the edge feature ε′ ijprojected to ψ(ε′ ij ) and α(ε′ ij ) are respectively query, key and value, the value vector is evenly divided into g groups, 1≤g≤D, the query is subtracted from the key to produce a vector that can modulate individual feature channels, and an additional relative position encoding δ(p i -p j ) is added to learn the absolute position relationship of the local point set again, and the attention weight encoding formula is as follows:

[0060] w ij =γ(δ mul (p i -p j )⊙(q-k)+δ bias (p i -p j )) (15)

[0061] wherein is a learnable weight encoding function, and ⊙ represents element-wise multiplication, p i and p j are the coordinates of the center point x i and the nearest neighbor point x j . is a multiplication and offset position encoding function MLP, q is a query vector, and k is a key vector;

[0062] The γ(·) function is composed of a grouping linear layer ζ(·), a batch normalization layer, an activation layer and a fully connected layer, allowing information exchange between different groups, wherein the grouping linear layer projects different groups of input vectors r independently using different parameters, as shown in the following formula:

[0063] γ(r)=ReLU(BN(ζ(r)) (16)

[0064]

[0065] wherein D g =D / g is the number of feature channels in each group, is a learnable parameter, and the input vector BN(·) is a batch normalization layer, and ReLU(·) is an activation function;

[0066] Step 3.4: Grouped vector attention aggregation, the output of the attention weight encoding is a grouped attention vector with g channels instead of D channels, and the channels of v within the same attention group share the same scalar attention weight from the grouped attention vector, and the grouped vector aggregation formula is as follows:

[0067]

[0068] wherein, is the output feature vector, l is the current group number, m is the learnable parameter is the channel number.

[0069] The process of step four is:

[0070] Step 4.1: In each layer feature propagation module, the global feature is spliced to the last layer point set by using the inverse distance weighted average formula IDW. The inverse distance weight method depends on the power value of the inverse distance. The closer the known point to the unknown point, the greater the influence of the known point on the interpolation result. The process is as follows:

[0071] 4.1.1) Let the known point set be X={x i |i=1,…,N l}, wherein N l is the number of points in the lth layer, each point x i =(p i ,f i ) includes spatial coordinates p i =(x i ,y i ,z i ) and feature vector f i , the target is to propagate the f i feature to the layer where l-1 is located, N l <N l-1 ;

[0072] 4.1.2) Calculate the Euclidean distance d(p,p i ) between the N l-1 layer point set x and each point x i in the N l point set;

[0073] 4.1.3) For each N l layer point x i , assign a weight ω to it:

[0074]

[0075] wherein q is the power parameter, which controls the influence of distance on weight, and is set to 2, and ε=10 -5 is the parameter to avoid denominator 0;

[0076] 4.1.4) Select the k nearest x i points to x, and weight the normalized weight of x i with the feature f i to obtain the feature f (j) (x) of the jth point x:

[0077]

[0078] Where k is the number of nearest neighbors, which is set to 3;

[0079] Step 4.2: Update the feature vector of each point using a unit feature propagation network. This network consists of fully connected layers with shared weights, batch normalization layers, and ReLU layers. Repeat the interpolation and propagation process until the features are propagated to the point set at the original resolution. Finally, reduce the network complexity with a Dropout layer with a dropout rate of 0.5, and then transform the feature dimension to C using an MLP. The segmentation score matrix output after softmax is as follows:

[0080]

[0081] Here, Ψ is a matrix of dimension N×C, where the elements y ij This represents the predicted probability that the i-th point belongs to the j-th semantic category;

[0082] Step 4.3: Backpropagation of the point cloud segmentation network uses the cross-entropy loss function, the calculation formula of which is as follows:

[0083]

[0084] Where N is the number of point cloud points, C is the number of semantic categories, and y ij Let y be the true label of the i-th point. ij Let be the predicted probability that the i-th point belongs to the j-th semantic category.

[0085] The technical concept of this invention is as follows: Existing point cloud segmentation techniques face challenges in handling complex geometric structures and capturing contextual information. On the one hand, point cloud data is disordered, sparse, and unevenly dense, making it difficult for existing methods to achieve a balance between local features and global information modeling. On the other hand, traditional methods suffer from information loss when processing local neighborhood relationships, and deep neural networks are prone to overfitting due to excessive parameter counts. This invention combines a geometric affine module to normalize local neighborhoods, constructs a neighborhood graph using a graph neural network to capture local topological information, then uses a residual network to extract deep edge features, and effectively fuses local and global contextual features through grouped vector attention.

[0086] The beneficial effects of this invention are mainly reflected in the following aspects: while significantly improving the segmentation accuracy, the method of this invention reduces the computational efficiency of the model, enhances the generalization ability of the model, and is suitable for efficient point cloud semantic segmentation in complex scenarios. Attached Figure Description

[0087] Figure 1is the principle diagram of a three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism.

[0088] Figure 2 is the schematic diagram of sampling block and batch construction.

[0089] Figure 3 is the schematic diagram of constructing neighborhood graph and extracting edge features.

[0090] Figure 4 is the schematic diagram of feature extraction and fusion module.

[0091] Figure 5 is the schematic diagram of cross-layer feature interpolation.

[0092] Figure 6 is the schematic diagram of feature propagation to the original resolution to complete segmentation prediction.

[0093] Figure 7 is the schematic diagram of mIoU.

[0094] Figure 8 is the visualization result of point cloud scene semantic segmentation. DETAILED DESCRIPTION

[0095] The application will be further described below with reference to the accompanying drawings.

[0096] Reference Figures 1-6 A three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism, comprising the following steps:

[0097] Step one, pre-processing the original point cloud data, including feature enhancement, class weight balance, sampling, batch construction and feature standardization, considering a point cloud data set with N points, represented as X = {x i |i = 1, …, N}, where each point x i = (p i , f i ) includes spatial coordinates and feature vector The size of D depends on the type of sensor. In general, the laser radar scan obtains three-dimensional point cloud data in the Cartesian coordinate system, that is, each point has only three basic dimensions p i = (x i , y i , z i ), respectively representing the position coordinates of the point, and the feature vector f i of the data set used is the RGB color feature, and the process is as follows:

[0098] Step 1.1: Use the point cloud processing library PCL to calculate the normal vector of the point cloud, introduce additional feature dimensions to enhance the expression ability of the original point cloud data, and use the normal vector information to provide more fine local geometric features;

[0099] Step 1.2: Count the number of points of each category in the original point cloud data to obtain the frequency distribution vector And calculate the weight distribution vector according to it, as shown in the following formula:

[0100]

[0101] Where C is the total number of semantic categories, that is, the number of different categories in the data set (such as walls, floors, tables, etc.), h i is the number of points in the point cloud belonging to the i-th category;

[0102] Because the number of points of some categories in the point cloud is too small, compared with the number of points of the categories with more points, it is more difficult to learn their feature information, so it is necessary to calculate the balance weight for each category, and the weight of each category is scaled by the maximum weight ratio and the cubic root smoothing, as follows:

[0103]

[0104] Where w j is the scaled weight of the j-th category, and w is the weight distribution vector.

[0105] Step 1.3: Because the distribution of different point cloud points is greatly different, the number of points needs to be selected according to the number of points during the training process, so the sampling probability p i is assigned to the i-th point cloud:

[0106]

[0107] Where N i is the number of points of the i-th point cloud, R is the total number of point clouds, and the sampling number M i of the i-th point cloud can be obtained according to the sampling probability:

[0108]

[0109] Where r s is the sampling rate, and N s is the number of points in a single sampling, which are set by hyperparameters.

[0110] Step 1.4: Randomly select a point as the center point x c of the current point cloud = (x c , y c , z c ), and as Figure 2The cubic sampling block with the length and width of block_size is constructed in the shown way, and the sampling is completed in the region:

[0111]

[0112] where b is the size of a cubic sampling block block_size, (x i ,y i ) is the horizontal and vertical coordinates of the i-th point x i . When the number of points in the block is insufficient, it is filled up to the number of single sampling points N s by random sampling with replacement;

[0113] In the actual training process, the point cloud data is divided into several batches batch, which is input into the neural network one by one to increase the optimization efficiency. A batch is composed of batch_size sampling blocks, and each sampling block contains a fixed number of points num_point, which is equal to N s in value. batch_size and num_point are hyperparameters that are reasonably set according to the memory capacity, so as to maximize the utilization of memory while ensuring the stability of training;

[0114] Step 1.5: Coordinate normalization and decentralization operations are performed on the sampled point cloud, and RGB values are also normalized, finally constructing a point cloud with a feature vector dimension of 12 The feature vector f i of each point x i is as follows:

[0115]

[0116] where (x c ,y c ,z c ) is the decentralized coordinate, (n x ,n y ,n z ) is the normalized coordinate, (n R ,n G ,n B ) is the normalized RGB value, is the normal vector;

[0117] Step two involves constructing a local neighborhood graph, including downsampling, grouping, affine transformation, and graph structure modeling. First, representative points are uniformly selected from the original point cloud using farthest-point sampling (FPS) to reduce data complexity while preserving geometric features. Then, a local neighborhood is defined for each sampled point, and the k-nearest neighbor algorithm is used to obtain its nearest neighbor set. Next, a geometric affine module is introduced to decentralize and standardize the neighborhood features to overcome the sparsity and irregularity of the geometric structure in local areas of the point cloud. Finally, by constructing a local neighborhood graph and designing an asymmetric edge feature learning function, combined with global features and local differences, the local topology and global relationships of the point cloud are explicitly modeled to fully extract edge features. The process is as follows:

[0118] Step 2.1: Downsampling is used to select some representative points from a relatively dense point set as sampling points. While ensuring the topological structure of the point cloud remains unchanged, it uses fewer data points to represent the point cloud. To ensure that the geometric features of the point cloud are preserved even at low resolution, the sampling process covers the point set space as uniformly as possible. Compared with random sampling, farthest-point sampling (FPS) has advantages in coverage. Its algorithm flow is as follows:

[0119] 2.1.1) Given a point cloud A set of sample points S = {s} i |i=1,…,M}, initialize the candidate farthest point set S. t = {s1}, where s1 is a random selection and is removed from the original point set S;

[0120] 2.1.2) To select the next point as the farthest point p, the following constraints must be satisfied:

[0121]

[0122] This constraint means: with S t Using the point closest to the median S as the reference, calculate the Euclidean distance between the midpoints of the two sets, and select the median S of S. t The largest point is taken as the farthest point, and this point is added to S. t Simultaneously, delete this point in S, where d(p,S) is the Euclidean distance between the farthest point p and the point set S, and d(q,s) is the distance between p and S. i ) is S t Midpoint q and midpoint s of S i The Euclidean distance between them is expressed as follows:

[0123]

[0124] Where, p t =(x n ,y n ,z n Let S be a sett The coordinate vector of the midpoint q, p = (x m ,y m ,z m ) is the midpoint s of set S i The coordinate vector, ||·||2 represents the 2-norm;

[0125] 2.1.3) Repeat steps 2.1.1) to 2.1.2) above to update the farthest point set S. t until S t The number of sampling points reaches N;

[0126] Step 2.2: To extract local features, it is necessary to define a neighborhood range for each sampling point in the point cloud. Given a sample set S = {s} within the point cloud X. i The subset S obtained by downsampling |i=1,…,M} is t ={s i For each sampling point in the range |i=1,…,N}, the k-nearest neighbor (kNN) algorithm is used to search for the k nearest points to the centroid to form a grouped point set.

[0127] Step 2.3: The geometric structures of different local regions may vary significantly. Directly using a residual network with shared weights is insufficient to fully capture this diversity of local regions, thus affecting the model's expressive power. Therefore, a geometric affine module is introduced to decentralize and standardize local features. Let... To obtain the local neighborhood features containing k nearest neighbors, we perform an affine transformation on the features of the nearest neighbors using the following formula:

[0128]

[0129] in, and For learnable parameters, ⊙ represents element-wise multiplication (Hadamard product), ε = 10 -5 σ is a parameter for numerical stability in the normalization operation. It is a scalar that describes the characteristic deviation between all local nearest neighbors and channels, as shown below:

[0130]

[0131] Step 2.4: For a local neighborhood of a point cloud, its essence is still an isolated and unordered set of points, lacking explicit local topological structure and global relationships. Therefore, a graph structure is constructed to model the local neighborhood. Consider a local region of a point cloud. Its corresponding directed graph representation is as follows Where V = {1, 2, ..., k} represents considering k local nearest neighbors as vertices. For edges, this local neighborhood graph can be represented as χ(i) = {j: (i, j) ∈ E} ∪ {i}, which contains self-loops, i.e., each node also points to itself. Define edge features as ε ij = f Θ (x i ,x j ), where is a nonlinear function of a set of learnable parameters Θ. Here f Θ is defined as follows:

[0132] ε ij = f Θ (x i ,x j ) = f Θ (x i ,x j -x i ) (11)

[0133] This asymmetric edge feature learning function combines the global features and relative feature differences x i -x j captured by the center point x i , as shown in Figure 3 .

[0134] Step three, local neighborhood graph feature extraction based on residual network and context feature fusion based on group attention vector, on the basis of local neighborhood graph, first extract edge features through multi-layer residual network with shared weights, and after the edge features are aggregated by max-pooling operation, the local neighborhood graph features are formed by splicing with the center point features; then, by using the Transformer structure with residual connection, the attention weight of the feature channel is dynamically allocated through the group vector attention (GAV) mechanism, and the key edge features are flexibly captured, and the relative position coding is introduced to learn the absolute position relationship of the neighborhood point set, the feature extraction and fusion module is as shown in Figure 4 , and its core operation is abstractly represented as:

[0135]

[0136] where Φ pre (·) is a local feature extraction function, A(·) is an aggregation function, and Φ pre (·) is a context feature fusion function.

[0137] The process of step three is as follows:

[0138] Step 3.1: For the local neighborhood graph structure, use the function Φ pre (·) to extract edge features, and Φ pre (·) is the abstract function fΘ The specific implementation of (·) is composed of a stack of three layers of shared-weight residual blocks ResP Block, and the residual block is two homogeneous residual MLP modules denoted as MLP(x) + x, each of which is composed of a fully connected layer, a batch normalization layer and an activation function, as shown in the following formula:

[0139] MLP(x i ,x j -x i )=ReLU(BN(θ m ·(x j -x i )+φ m ·x i )) (13)

[0140] wherein Θ=(θ1,…,θ M ,φ1,…,φ M ) are the learnable weight parameters in the fully connected layer (FC), BN(·) is the batch normalization operation, and ReLU=max(0,x) is a common nonlinear activation function in deep learning;

[0141] Step 3.2: Since the point cloud data has no order, the network processing the point cloud data needs to have permutation invariance, i.e., it is not sensitive to the arrangement order of the input points. Therefore, in the aggregation process of edge features, the aggregation function A(·) adopts a max pooling operation, and the expression is:

[0142]

[0143] The aggregated edge features ε i ′ j and the center point features x i are spliced to obtain the local neighborhood graph features G;

[0144] Step 3.3: The neighborhood graph features G are processed by the Φ pos (·) function to fuse the local and global context information to obtain the context features Φ pos (·) is a Transformer structure with residual connection, and a Point Transformer layer is added after the first MLP. The vector self-attention mechanism (Vector Self Attention) can flexibly assign attention weights to each channel of edge features, allowing the center point to dynamically capture the key edge features in the neighborhood graph. Since the residual network in step 3.1 causes the model depth and the number of channels to increase sharply, the number of weight encoding parameters also increases sharply, which may cause serious overfitting and limit the model generalization ability.

[0145] Thus the present application uses a group vector attention operator GAV which computes the relation as Figure 4 As shown, first the midpoint feature x i and edge feature ε i ′ j are projected to ψ(ε′ ij ) and α(ε′ ij ) respectively as Query, Key and Value, the value vector is averaged into g groups, 1≤g≤D, the query is subtracted from the key to produce a vector that can modulate individual feature channels, and additional relative position encoding δ(p i -p j ) is added to learn the absolute position relation of local point set again, the attention weight encoding formula is as follows:

[0146] w ij =γ(δ mul (p i -p j )⊙(q-k)+δ bias (p i -p j )) (15)

[0147] wherein is a learnable weight encoding function, ⊙ denotes element-wise multiplication (Hadamard product), p i and p j are the coordinates of the center point x i and the neighboring point x j . is a multiplication and offset position encoding function (MLP), q is the query vector, and k is the key vector.

[0148] Specifically, the γ(·) function is composed of a group linear layer ζ(·), a batch normalization layer, an activation layer and a fully connected layer, allowing information exchange between different groups, wherein the group linear layer projects different groups of input vector r independently using different parameters, as shown in the following formula:

[0149] γ(r)=ReLU(BN(ζ(r)) (16)

[0150]

[0151] wherein D g =D / g is the number of feature channels per group, is a learnable parameter, the input vector BN(·) is a batch normalization layer, and ReLU(·) is an activation function.

[0152] Step 3.4: Grouped vector attention aggregation, the output of attention weight encoding is the grouped attention vector of g channels instead of D channels, and the channels of v within the same attention group share the same scalar attention weight from the grouped attention vector. The grouped vector aggregation formula is as follows:

[0153]

[0154] wherein, is the output feature vector, l is the current group number, m is a learnable parameter channel number of the feature vector;

[0155] Step four, hierarchical upsampling feature decoding, including inverse distance weighted average interpolation and feature propagation, completing segmentation output and back propagation, the above down-sampling, grouping, affine transformation, local feature extraction and context feature fusion module jointly form a feature abstraction layer, after four layers of recursive feature abstraction, the low-resolution global point cloud feature vector is obtained, which contains rich context semantic information; Since point cloud segmentation belongs to a dense classification task, the feature information of the points needs to be classified point by point, therefore, a structure coupling the feature encoder and the decoder is adopted. The feature decoder includes four feature propagation modules, which splices the sparse point set features containing high-dimensional semantic features to the previous layer point set across levels by interpolation method; the process is:

[0156] Step 4.1: In each feature propagation module, the global feature is spliced to the previous layer point set by using the inverse distance weighted average formula (IDW), the inverse distance weight method mainly depends on the power value of the inverse distance, the closer the known point to the unknown point, the greater the influence on the interpolation result, as shown in Figure 5 , and the algorithm flow is as follows:

[0157] 4.1.1) Let the known point set be X = {x i |i = 1, …, N l}, wherein N l is the number of points at the l level, each point x i = (p i , f i ) includes spatial coordinates p i = (x i , y i , z i ) and feature vector f i , the target is to propagate the f i feature to the l-1 level, N l < N l-1 ;

[0158] 4.1.2) Calculate the distance between the N l-1 level point set x and the Nl Euclidean distance d(p, p i ) i )

[0159] 4.1.3) For each point x l in the N i th level, assign it a weight ω

[0160]

[0161] where q is the power parameter, controlling the influence of distance on weight, set to 2, and ε = 10 -5 to avoid parameters with denominator of 0;

[0162] 4.1.4) Select the k nearest points of x i , and get the feature f i of the jth point x by weighting the normalized weight of x i and the feature f (j) :

[0163]

[0164] where k is the number of nearest points, set to 3;

[0165] Step 4.2: Update the feature vector of each point using the unit feature propagation network, as shown in the following figure. The network consists of fully connected layers, batch normalization layers and ReLU layers with shared weights, repeating the interpolation and propagation process until the features are propagated to the original resolution point set. Finally, reduce the network complexity through a Dropout layer with a dropout rate of 0.5, and then transform the feature dimension to C (the number of semantic categories) through MLP. The segmentation score matrix output by softmax is as follows: Figure 6

[0166]

[0167] where Ψ is a matrix with dimensions N x C, and the element y ij represents the predicted probability of the ith point belonging to the jth semantic category.

[0168] Step 4.3: The backpropagation of the point cloud segmentation network uses the cross-entropy loss function, whose calculation formula is as follows:

[0169]

[0170] where N is the number of point cloud points, C is the number of semantic categories, y ij is the true label of the ith point, and y ij is the predicted probability of the ith point belonging to the jth semantic category. ​

[0171] The experimental comparison scheme of the present embodiment is as follows:

[0172] 1) Dataset:

[0173] In the point cloud scene semantic segmentation test experiment of the method of the present application, the dataset used is S3DIS (Stanford Large-Scale 3D Indoor Spaces), which is a large-scale indoor point cloud dataset created by Stanford University. The dataset covers 271 room scenes in 6 different buildings, and the categories of the room scenes include 11 types, including office, conference room, corridor, auditorium, open space, lobby, lounge, food storage room, copy room, storage room, and bathroom. The semantic labels include 13 types, including ceiling, floor, wall, beam, column, window, door, table, chair, sofa, bookshelf, board, and miscellaneous elements. The S3DIS dataset is composed of a series of.txt files, each.txt file has several lines of data, each line of data represents a point, and each point has 6 features: xyz coordinate features and RGB color features.

[0174] 2) Evaluation index:

[0175] In the point cloud segmentation task, to evaluate the performance of the model, the mean intersection over union (mIoU) and overall accuracy (OA) are usually used as evaluation indexes. The following is a description of the two:

[0176] The mean intersection over union (mIoU) is an index to measure the degree of overlap between the predicted segmentation result and the true semantic label, defined as follows:

[0177]

[0178] where p ij represents the probability of classifying i class as j class, and C is the total number of classes. The above formula is transformed as:

[0179]

[0180] where TP represents the number of points with positive labels and positive predictions (true positives), FP represents the number of points with negative labels and positive predictions (false positives), and FN represents the number of points with positive labels and negative predictions (false negatives). The schematic diagram is shown in Figure 7 ;

[0181] The overall accuracy (OA) evaluates the classification accuracy of the model for all points, that is, the ratio of the number of points correctly predicted to the total number of points on the test set. The formula is:

[0182]

[0183] Wherein, TP is the number of points correctly classified as class i, and N is the total number of points.

[0184] 3) Experiment 1: Based on single region validation method and 6-fold cross-validation method, the method of the present application is compared with the existing method. The S3DIS dataset is divided into Area_1~Area_6 according to different buildings, and the single region validation method takes Area_5 as the test set alone, and the remaining regions as the training set. For the 6-fold cross-validation method, one of the regions is selected as the test set each time, and the remaining regions are used as the training set. The average result of six times is taken as the final performance index. Table 1 is the training parameter table of the method of the present application;

[0185]

[0186] Table 1

[0187] The training parameters used by the method of the present application are shown in Table 1. Before the data is input into the neural network, the point cloud is rotated, translated, scaled, and random noise is added, etc. to improve the robustness of the model. After the training is completed, the model is evaluated using the test set, and the sliding window method is used to test in blocks. After the test is completed, the segmentation prediction results are visualized using the software MeshLab. The visualization results of part of the rooms in the Area_5 region are shown in Figure 8 The mIoU and OA are calculated, and the experimental results of the single region validation are shown in Table 2, and the experimental results of the 6-fold cross-validation are shown in Table 3.

[0188] Table 2 is the single region validation result (Area_5) based on the S3DIS dataset;

[0189]

[0190] Table 2

[0191] Table 3 is the 6-fold cross-validation result based on the S3DIS dataset;

[0192]

[0193] Table 3

[0194] From the above experimental results, it can be seen that based on the performance indicators under the two evaluation modes, the method of the present application is better than some existing models. In the Area_5 area verification, the mIoU and OA of the method of the present application reached 65.4% and 88.1%. And in the 6-fold cross-validation, the mIoU and OA of the method of the present application reached 68.1% and 88.5%, which improved by several percentage points compared with the better-performing GACNet and PointWeb models. Figure 6 The semantic segmentation results in different scenes such as conference rooms, corridors, and lounges are shown, and it can be seen that the segmentation results of the present method are generally close to the true labels, and the segmentation effect of the boundary details and small objects is more excellent.

[0195] 4) Experiment two: This experiment is an ablation experiment, different model variants are designed, and the independent contribution and role of each module are analyzed by gradually adding innovative modules. Still using the S3DIS dataset, the evaluation is carried out in Area_5 single area. The comparative baseline method of this experiment is the PointNet++ basic framework, which only contains the basic point cloud segmentation structure. The experimental results are shown in Table 4, where GNN represents graph neural network, ResNet represents residual network, and GVA represents grouping vector attention mechanism.

[0196] Table 4 is a model ablation experiment (based on Area_5 single area verification);

[0197]

[0198] Table 4

[0199] From the above ablation experiment, it can be seen that by adding the neighborhood graph structure to the baseline method (PointNet++) to capture the local point cloud geometric relationship, the mIoU is improved by 2.5%. By introducing affine transformation and deep residual network (ResNet) structure can capture deeper edge features, which makes the mIoU improve by 5.7%. Finally, the grouping vector attention (GVA) is added to realize the selective aggregation of edge features, which makes the mIoU improve by 3.7%. The addition of each module can significantly improve the model performance, which proves that the performance of the complete model (PointNet+++GNN+ResNet+GVA) is significantly higher than that of adding a single module, and the modules have significant synergistic effect, which can verify the advancement and innovation of the method of the present application.

[0200] The content described in the embodiments of the present specification is only a list of implementation forms of the inventive concept, and is only for illustrative purposes. The protection scope of the present application should not be regarded as being limited to the specific forms described in the embodiments, and the protection scope of the present application also extends to equivalent technical means that can be thought of by those skilled in the art according to the inventive concept.

Claims

1. A three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism, characterized in that, The method comprises the following steps: Step one, pre-process the original point cloud data, including feature enhancement, class weight balance, sampling, batch construction and feature standardization, considering a point cloud dataset with one point is represented as where each point includes spatial coordinates and a feature vector , The size of depends on the sensor type; the laser radar scan obtains three-dimensional point cloud data in the Cartesian coordinate system, that is, each point has only three basic dimensions , respectively representing the position coordinates of the point, and the feature vector contained in the used dataset is the RGB color feature; Step two involves constructing a local neighborhood graph, including downsampling, grouping, affine transformation, and graph structure modeling. First, representative points are uniformly selected from the original point cloud using farthest point sampling (FPS) to reduce data complexity while preserving geometric features. Then, a local neighborhood is defined for each sampled point, utilizing... k The nearest neighbor algorithm obtains the set of its nearest neighboring points; then, a geometric affine module is introduced to decentralize and standardize the neighborhood features to overcome the sparse and irregular geometric structure of the local region of the point cloud; finally, by constructing a local neighborhood graph, an asymmetric edge feature learning function is designed, and by combining global features and local differences, the local topology and global relationship of the point cloud are explicitly modeled to fully extract edge features. Step three, local neighborhood graph feature extraction based on residual network and context feature fusion based on grouping vector attention, based on the local neighborhood graph, first extract edge features through a multi-layer residual network with shared weights, and after aggregating the edge features through max-pooling operation, splice the center point features to form the local neighborhood graph features; then, by using the Transformer structure with residual connection, the attention weight of the feature channel is dynamically allocated through the grouping vector attention (GAV) mechanism, which can flexibly capture the key edge features, and the relative position encoding is introduced to learn the absolute position relationship of the neighborhood point set; Step four, hierarchical upsampling feature decoding, including inverse distance weighted average interpolation and feature propagation, completing segmentation output and back propagation; the down-sampling, grouping, affine transformation, local feature extraction and context feature fusion modules jointly form a feature abstraction layer, and after four layers of recursive feature abstraction, a low-resolution global point cloud feature vector is obtained, which contains rich context semantic information; since point cloud segmentation belongs to a dense classification task, the feature information of the points needs to be classified point by point, therefore, a structure coupling the feature encoder and the decoder is adopted, and the feature decoder comprises four feature propagation modules, which cross-layer splice the sparse point set features containing high-dimensional semantic features to the previous layer point set through interpolation.

2. The three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism according to claim 1, wherein, The process of step one is: Step 1.1: using the point cloud processing library (PCL) to calculate the normal vector of the point cloud, introducing an additional feature dimension to enhance the expression ability of the original point cloud data, and using the normal vector information to provide more fine local geometric features; Step 1.2: Count the number of points of each category in the original point cloud data, and get the frequency distribution vector and calculate the weight distribution vector according to it, as shown in the following formula: (1)； wherein, is the total number of semantic classes, i.e. the number of different classes in the dataset, is the number of points in the point cloud that belong to the th class. The weight of each class is scaled by the maximum weight ratio and the cubic root smoothing, and the formula is as follows: (2)； wherein, is the scaled weight for the th category, is the weight distribution vector; Step 1.3: Because of the large difference between the point cloud point number distribution, a part of points needs to be selected according to the point number distribution and input into the network during the training process, so the sampling probability is assigned to the first point cloud :​ (3)； wherein, is the number of points of the th point cloud, is the total number of point clouds, and the sampling number of the th point cloud is obtained according to the sampling probability is: (4)； wherein, is a sampling rate, is a number of single sampling points, both set by hyperparameters; Step 1.4: Randomly select a point as the center point of the current point cloud A cuboid sampling block with block_size in length and width is constructed and sampling is completed in this region: (5)； wherein, block_size is the size of a cubic sampling block, is the horizontal and vertical coordinates of the point, and when the number of points in the block is insufficient, the number of points is supplemented to the single sampling number by random sampling with replacement ; In the actual training process, the point cloud data is divided into several batches and input to the neural network one by one to increase the optimization efficiency; a batch is composed of batch_size sampling blocks, and each sampling block contains a fixed number of points num_point, which is equal to , batch_size and num_point are hyperparameters set according to the memory capacity to maximize the utilization of memory while ensuring the stability of training; Step 1.5: Coordinate normalization and centering operation is performed on the sampled point cloud, and the RGB values are also normalized, and finally the feature vector dimension of the point cloud is constructed as 12 The feature vector of each point is as follows: ​ (6)； wherein is a decentralized coordinate, is a normalized coordinate, is a normalized RGB value, is a normal vector.

3. The three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism according to claim 1 or 2, characterized in that, The process of step two is: Step 2.1: down-sampling is used to select some representative points as sampling points from the relatively dense point set, and under the premise of ensuring the topological structure of the point cloud unchanged, the point cloud is represented by fewer data points; Step 2.2: To extract local features, a neighborhood range needs to be defined for each sample point in the point cloud, given a sample set within the point cloud , whose subset is obtained by downsampling , using a nearest neighbor algorithm to search the nearest points to the "center" of each sample point, the group point set is formed by the sample points Step 2.3: Introduce the geometric affine module to decentralize and standardize the local features, set For the local neighborhood feature containing neighbor points, the affine transformation of the local neighborhood neighbor point feature is performed using the following formula: (9)； wherein, and are learnable parameters, denotes an element-wise multiplication, is a parameter for numerical stability in the normalization operation, is a scalar describing the feature bias between all local region neighbors and channels, as follows: (10)； Step 2.4: For a point cloud local neighborhood, its essence is still a set of isolated unordered points between points, lacking explicit local topology and global relationship, so the local neighborhood is modeled by constructing a graph structure, considering a point cloud local region , its corresponding directed graph is represented as , where represents taking local neighbor points as vertices, as edges, then this local neighborhood graph can be represented as , which contains self-loops, i.e. each node also points to itself, and the edge feature is defined as , where is a set of learnable parameters , and is a nonlinear function, and is defined as follows: (11)； This asymmetric edge feature learning function combines the center point captured global features and relative feature differences captured local neighborhood information.

4. The three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism according to claim 3, wherein, In step 2.1, the process of the farthest point sampling (FPS) is as follows: 2.1.1) Given a point cloud a set of sample points within the point cloud , initialize a set of candidate farthest points where is randomly chosen and remove it from the original set of points ; 2.1.2) Selecting the next point as the farthest point , subject to the following constraints: (7)； The constraint means: with Mid-range Using the nearest point as the reference, calculate the Euclidean distance between the midpoints of the two sets, and select... Mid-range The largest point is taken as the farthest point, and this point is added to the list. At the same time Delete this point, where For the farthest point with point set Euclidean distance, for midpoint and midpoint The Euclidean distance between them is expressed as follows: (8)； wherein is the set midpoint coordinate vector of the midpoint, is the set midpoint coordinate vector of the midpoint, denotes the 2-norm; 2.1.3) Repeat steps 2.1.1) - 2.1.2) above, updating the farthest point set until the number of points reaches the downsampled point number .

5. The three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism according to claim 3, wherein, In step three, the core operation of the feature extraction and fusion module is abstractly represented as: (12)； wherein, is a local feature extraction function, is an aggregation function, is a context feature fusion function, the process is as follows: Step 3.1: For the local neighborhood graph structure, use the function Extract edge features, For the abstract function in step 2.4 In its specific implementation, the abstract function consists of a stack of three layers of ResP Blocks with shared weights. Each ResP Block is two homogeneous ResP MLP modules, represented as follows: Each MLP module consists of a fully connected layer, a batch normalization layer, and an activation function, as shown in the following equation: (13)； wherein, are learnable weight parameters in a fully connected layer FC, is a batch normalization operation, is a commonly used non-linear activation function in deep learning; Step 3.2: Since the point cloud data has no order, the network that processes the point cloud data needs to have permutation invariance, that is, it is not sensitive to the arrangement order of the input points. Therefore, in the aggregation process of edge features, the aggregation function The maximum pooling operation max pooling is adopted, and the expression is as follows: (14)； after the aggregation of the edge features with the center point feature stitching, to obtain a local neighborhood graph feature ; Step 3.3: neighborhood graph features By Contextual features are obtained by fusing local and global context information through function processing , is a Transformer structure with residual connection, which adds a Point Transformer layer after the first MLP. It uses vector self-attention mechanism to flexibly assign attention weights to edge features of each channel, allowing the center point to dynamically capture the key edge features in the neighborhood graph. Using a grouped vector attention operator GAV, first project the mid-point features and edge features to , and respectively as query Query, key Key and value Value, average the channels of the value vector into groups, differencing the query from the key produces a vector that can modulate individual feature channels, and add extra relative position encodings again learn the absolute positional relationships of the local point set, the attention weight encoding formula is as follows: (15)； wherein, is a learnable weight encoding function, denotes element-wise multiplication, and is a center point and a near neighbor point with coordinates, is a multiplication and offset position encoding function MLP, is a query vector, is a key vector; a grouping linear layer , a batch normalization layer, an activation layer, and a fully connected layer, allowing information exchange between different groups, wherein the grouping linear layer projects the input vector such that different groups use different parameters independently, as shown in the following equation: (16)； (17)； wherein is the number of feature channels for each group, is a learnable parameter, the input vector , is a batch normalization layer, is an activation function; Step 3.4: Grouped vector attention aggregation, the output of attention weight encoding is channels instead of grouped attention vectors for channels, the same scalar attention weight from the grouped attention vector is shared by channels within the same attention group, the grouped vector aggregation formula is as follows: (18) wherein, is an output feature vector, is a current packet number, is a learnable parameter is a channel number.

6. The three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism according to claim 1 or 2, characterized in that, The process of step four is: Step 4.1: in each feature propagation module, the inverse distance weighted average formula (IDW) is used to cross-layer splice the global features to the previous layer point set, and the inverse distance weight method depends on the power value of the inverse distance, and the closer the known point to the unknown point, the greater the influence on the interpolation result; Step 4.2: Update the feature vector of each point by using the unit feature propagation network, which consists of fully connected layers, batch normalization layers and ReLU layers sharing weights, repeat the process of interpolation and propagation until the features are propagated to the point set of the original resolution, and finally reduce the network complexity by a Dropout layer with a dropout rate of 0.5, and then transform the feature dimension by MLP to The segmentation score matrix output by softmax is as follows: (21)； wherein, is a matrix of dimension wherein the element represents the predicted probability that the th point belongs to the th semantic class; Step 4.3: the back propagation of the point cloud segmentation network uses the cross-entropy loss function, and the calculation formula is as follows: (22)； wherein, is the number of points in the point cloud, is the number of semantic classes, is the true label of the th point, is the predicted probability that the th point belongs to the th semantic class.

7. The three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism according to claim 6, wherein, The process of step 4.1 is as follows: 4.1.1) Let a set of known points be where is the number of points in the th level, each point comprises spatial coordinates and a feature vector , the goal is to propagate the feature to the level where ;​ 4.1.2) Calculations hierarchical point set with Euclidean distance of each point in the point set ;​ 4.1.3) For each Hierarchical points Assign weights to them for: (19)； wherein is a power parameter that controls the influence of the distance on the weight and is set to 2, is a parameter to avoid division by zero; 4.1.4) Select distance Click the nearest indivual Point, will Normalized weights and features Weighted to obtain the first Points Features : (20)； wherein For the number of near neighbor points, set to 3.

Images