Three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism

Abstract

A three-dimensional point cloud semantic segmentation method based on graph convolution and a grouping vector attention mechanism adopts a network structure of a U-Net style, feature extraction and propagation are performed on point cloud through multi-level feature coding and decoding, each level firstly centralizes and standardizes local neighborhood features through a geometric affine module, and then the local neighborhood features are subjected to feature extraction and propagation; the capturing capability of the model on the diversity of the local region is enhanced; secondly, constructing a local neighborhood graph, and designing an asymmetric edge feature learning function to extract local topological information; deep edge features are further extracted in combination with a residual network; and finally, selectively fusing local features and global context information under the condition of low parameter quantity by using a grouping vector attention mechanism, and performing cross-level feature interpolation up-sampling to an original scale to complete point cloud segmentation. The method can significantly improve the point cloud segmentation precision, and is suitable for the perception of a mobile robot and other devices carrying a three-dimensional sensor on a complex scene.

Description

Technical Field

[0001] The present invention belongs to the field of cloud data processing and environmental perception, and specifically relates to a three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism. Background Art

[0002] Point cloud is a common three-dimensional data representation, which consists of a large number of irregular points in space. It can be used to perceive and represent the geometric shapes and structures of the real world, and is a bridge between the physical world and the digital world. With the rapid development of sensor technology (such as lidar, depth camera and 3D scanning equipment), point cloud data has been widely used in many fields. In autonomous driving, point cloud can be used to accurately detect and identify target objects (such as vehicles, pedestrians, etc.) on the road, providing key support for safe driving; in robot navigation, point cloud data can efficiently capture obstacles, terrain features and object information in the environment, providing key reference for the robot's path planning and obstacle avoidance tasks; in the field of three-dimensional reconstruction, point cloud is widely used to convert physical scenes into three-dimensional digital models, helping cultural heritage protection, architectural design and virtual reality applications. However, due to the characteristics of point cloud data such as noise, disorder, uneven density and sparsity, how to perform efficient and accurate classification and segmentation on this basis has become a complex and challenging task. The solution to these problems is of great significance to promote the application of point cloud technology in complex scenes.

[0003] At present, the development of point cloud segmentation technology mainly relies on deep learning methods, which are mainly divided into the following four types: projection-based methods, discretization-based methods, hybrid-based methods and point-based methods. The projection-based method converts the three-dimensional problem into a two-dimensional image processing task by projecting the three-dimensional point cloud onto a two-dimensional plane, and uses the existing convolutional neural network for processing. However, since the three-dimensional point cloud is compressed into a two-dimensional space, it may cause the loss of some spatial information when processing complex geometric structures. The discretization-based method maps the point cloud data into a three-dimensional voxel grid to form a structure similar to a three-dimensional image, and then uses a three-dimensional convolutional neural network (3D CNN) for processing. However, in high-resolution point clouds, the discretization process may lead to excessive computational workload and storage requirements, making it difficult to balance resolution and computational efficiency, and the discretization process will also lead to the loss of local geometric information. The hybrid-based method combines the characteristics of projection and discretization, and can simultaneously utilize two-dimensional and three-dimensional feature information to improve the perception and generalization capabilities of the model, but the network structure is relatively complex and requires high computing resources. In 2017, Qi et al. from Stanford University first proposed the point-based method PointNet, which operates directly on irregular point clouds and learns their features point by point. PointNet++ proposed in the same year further improved the performance of point cloud processing by recursively extracting local features at multiple scales. In the following years, Wang et al. proposed a dynamic graph convolutional network EdgeConv to generate edge features describing points and adjacent points, preserving local neighborhood relationships. Thomas et al. proposed KPConv, introducing variable convolution kernels to improve the flexibility and expressiveness of point cloud processing. Recently, Zhao et al. proposed Point Transformer, which uses the attention mechanism to capture contextual information. Ma et al. proposed PointMLP, which abandons the complex local feature extraction module and uses a lightweight residual network to significantly improve the inference speed of point cloud segmentation without losing accuracy.

[0004] In summary, although the current point cloud processing methods have made some progress in classification and segmentation tasks, they still face the challenges of how to efficiently and accurately extract local geometric features and global features of point clouds in complex environments, as well as overcome sparsity and other issues. Traditional methods are often limited by information loss or excessive consumption of computing resources when processing complex geometric structures; and although point-based deep learning methods perform well in processing sparse and irregular point clouds, there is still room for further improvement in the ability to integrate local features with global contextual information. Summary of the invention

[0005] In order to overcome the shortcomings of existing point cloud segmentation technologies that are difficult to capture complex local geometric structures and global contextual information, resulting in limited improvement in segmentation accuracy, the present invention provides a 3D point cloud semantic segmentation method based on graph convolution and grouped vector attention mechanism, which provides a new solution for high-precision point cloud segmentation. This method uses the U-Net style feature encoding and decoding network as the framework, combined with the farthest point sampling and cross-level feature interpolation methods, and innovatively uses graph neural networks to extract edge features of local neighborhoods, enhancing the model's ability to express local topological information; by introducing a residual network with affine transformation, deep edge features are further extracted, significantly improving the model's perception of complex geometric structures; at the same time, the grouped vector attention mechanism is used to achieve selective fusion of local features and global contextual information with a low number of parameters, effectively improving the accuracy and efficiency of point cloud segmentation.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A 3D point cloud semantic segmentation method based on graph convolution and group vector attention mechanism includes the following steps:

[0008] Step 1: Preprocess the original point cloud data, including feature enhancement, category weight balancing, sampling, batch construction and feature standardization. Consider a point cloud dataset with N points, represented by X = {x i |i=1,…,N}, where each point x i =(p i ,f i ) include spatial coordinates and the eigenvector The size of D depends on the type of sensor; the laser radar scan obtains three-dimensional point cloud data in the Cartesian coordinate system, that is, each point has only three basic dimensions p i =(x i ,y i ,z i ), respectively representing the position coordinates of the points, and the feature vector f contained in the dataset used i is the RGB color feature;

[0009] Step 2: Local neighborhood graph construction, including downsampling, grouping, affine transformation and graph structure modeling; first, representative points are uniformly selected from the original point cloud through the farthest point sampling FPS to reduce data complexity while retaining geometric features; then, a local neighborhood is defined for each sampling point, and the k-nearest neighbor algorithm is used to obtain its nearest neighborhood point set; then, a geometric affine module is introduced to decentralize and standardize the neighborhood features to overcome the sparse and irregular geometric structure of the local area of ​​the point cloud; finally, by constructing a local neighborhood graph and designing an asymmetric edge feature learning function, the local topological structure and global relationship of the point cloud are explicitly modeled, and the edge features are fully extracted;

[0010] Step 3: The local neighborhood graph feature extraction based on the residual network is combined with the context feature fusion based on the grouped vector attention. On the basis of the local neighborhood graph, the edge features are first extracted through a multi-layer residual network with shared weights, and after the edge features are aggregated by the maximum pooling operation, they are concatenated with the center point features to form the local neighborhood graph features. Subsequently, the Transformer structure with residual connections is used to dynamically allocate the attention weights of the feature channels through the grouped vector attention GAV mechanism to flexibly capture the key edge features. At the same time, relative position encoding is introduced to learn the absolute position relationship of the neighborhood point set.

[0011] Step 4: hierarchical upsampling feature decoding, including inverse distance weighted average interpolation and feature propagation, to complete segmentation output and back propagation; downsampling, grouping, affine transformation, local feature extraction and context feature fusion modules together constitute a feature abstraction layer. After four layers of recursive feature abstraction, a low-resolution global point cloud feature vector is obtained, which contains rich contextual semantic information; since point cloud segmentation is an intensive classification task, it is necessary to classify the feature information of the points point by point. Therefore, a structure that couples the feature encoder and decoder is adopted. The feature decoder includes four layers of feature propagation modules. Through interpolation, the sparse point set features containing high-dimensional semantic features are spliced ​​across levels to the point set of the previous layer.

[0012] Further, the process of step one is:

[0013] Step 1.1: Use the point cloud processing library PCL to calculate the normal vector of the point cloud, introduce additional feature dimensions to enhance the expression ability of the original point cloud data, and use the normal vector information to provide more refined local geometric features;

[0014] Step 1.2: Count the number of points in each category in the original point cloud data to obtain the frequency distribution vector And calculate the weight distribution vector based on it, as shown in the following formula:

[0015]

[0016] Among them, C is the total number of semantic categories, that is, the number of different categories in the dataset, and h i is the number of points belonging to the i-th category in the point cloud;

[0017] The weight of each category is scaled by the maximum weight ratio and cube root smoothing, as follows:

[0018]

[0019] Among them, w j is the scaled weight of the j-th category, and w is the weight distribution vector;

[0020] Step 1.3: Since the point distribution of different point clouds varies greatly, it is necessary to select a portion of points according to the point distribution to input into the network during training. Therefore, the sampling probability p is assigned to the i-th point cloud. i :

[0021]

[0022] Among them, N i is the number of points in the i-th point cloud, and R is the total number of point clouds. According to the sampling probability, the sampling times M of the i-th point cloud can be obtained. i for:

[0023]

[0024] Among them, r s is the sampling rate, N s is the number of single sampling points, all set by hyperparameters;

[0025] Step 1.4: Randomly select a point as the center point x of the current point cloud c =(x c ,y c ,z c ), construct a cubic sampling block with a length and width of block_size, and complete the sampling in this area:

[0026]

[0027] Where b is the size of a cubic sampling block block_size, (x i ,y i ) is the i-th point x i When the number of points in a block is insufficient, random sampling with replacement is used to fill the number of single sampling points N. s ;

[0028] In the actual training process, the point cloud data is divided into several batches and input into the neural network one by one to increase the optimization efficiency; a batch consists of batch_size sampling blocks, each sampling block contains a fixed number of points num_point, which is numerically equal to N s , batch_size and num_point are hyperparameters set according to the video memory capacity to maximize the utilization of video memory while ensuring training stability;

[0029] Step 1.5: Normalize and decenter the sampled point cloud, normalize the RGB values, and finally construct a point cloud with a feature vector dimension of 12. Each point x i The eigenvector f i as follows:

[0030]

[0031] Where (x c ,y c ,z c ) is the decentralized coordinate, (n x ,n y ,n z ) is the normalized coordinate, (n R ,n G ,n B ) is the normalized RGB value, is the normal vector.

[0032] The process of step 2 is:

[0033] Step 2.1: Downsampling is used to select some representative points from a relatively dense point set as sampling points. Under the premise of ensuring that the topological structure of the point cloud remains unchanged, fewer data points are used to represent the point cloud; the process of sampling FPS at the farthest point is as follows:

[0034] 2.1.1) Given a point cloud A sample point set S = {s i |i=1,…,M}, initialize the candidate farthest point set S t ={s1}, where s1 is randomly selected and removed from the original point set S;

[0035] 2.1.2) Select the next point as the farthest point p, which must satisfy the following constraints:

[0036]

[0037] The constraint means: tThe point closest to S in the middle is used as the benchmark, the Euclidean distance between the points in the two sets is calculated, and the point with the closest distance S in S is selected. t The largest point is taken as the farthest point and added to S t , and delete this point in S at the same time, where d(p,S) is the Euclidean distance between the farthest point p and the point set S, and d(q,s i ) is S t The midpoint q and the midpoint s of S i The Euclidean distance between them is expressed as follows:

[0038]

[0039] Among them, p t =(x n ,y n ,z n ) is the set S t The coordinate vector of the midpoint q, p = (x m ,y m ,z m ) is the midpoint s of the set S i The coordinate vector of , ||·||2 represents the 2-norm;

[0040] 2.1.3) Repeat the above steps 2.1.1) to 2.1.2) to update the farthest point set S t , until S t The number of points reaches the number of downsampling points N;

[0041] Step 2.2: In order to extract local features, it is necessary to define a neighborhood range for each sampling point in the point cloud. Given a sample set S = {s i |i=1,…,M}, the subset obtained by downsampling is S t ={s i |i=1,…,N}, taking each sampling point as the "center of mass", the k nearest neighbor algorithm is used to search for the k point sets closest to the center of mass to form a grouped point set

[0042] Step 2.3: Introduce the geometric affine module to decentralize and standardize local features. To obtain the local neighborhood features containing k neighboring points, use the following formula to perform affine transformation on the neighboring point features of the local neighborhood:

[0043]

[0044] in, and is a learnable parameter, ⊙ represents element-wise multiplication, ε=10 -5It is a numerical stability parameter in the normalization operation. σ is a scalar that describes the characteristic deviation between all local region neighboring points and channels, as shown below:

[0045]

[0046] Step 2.4: For a local neighborhood of a point cloud, its essence is still an isolated and unordered set of points between points, lacking an explicit local topological structure and global relationship. Therefore, the local neighborhood is modeled by constructing a graph structure. Consider a local area of ​​a point cloud The corresponding directed graph is represented as Where V = {1, 2, ..., k} means that k local neighboring points are regarded as vertices, As an edge, this local neighborhood graph can be expressed as χ(i) = {j:(i,j)∈E}∪{i}. The graph contains self-loops, that is, each node also points to itself, and the edge feature is defined as ε ij =f Θ (x i ,x j ),in is a nonlinear function with a set of learnable parameters Θ, and f Θ Defined as:

[0047] ε ij =f Θ (x i ,x j ) = f Θ (x i ,x j -x i ) (11)

[0048] This asymmetric edge feature learning function combines the center point x i The captured global features and relative feature differences x j -x i Captured local neighborhood information.

[0049] In step 3, the core operation of the feature extraction and fusion module is abstractly represented as follows:

[0050]

[0051] Among them, Φ pre (·) is the local feature extraction function, A(·) is the aggregation function, Φ pre (·) is the context feature fusion function, and the process is as follows:

[0052] Step 3.1: Use the function Φ for the local neighborhood graph structure pre (·) Extract edge features. Φ pre(·) is the abstract function f in step 2.4 Θ The specific implementation of (·) is that the mapping function consists of three layers of weight-sharing residual blocks ResP Block stacked together. The residual block is two homogeneous residual MLP modules, expressed as MLP(x)+x. Each MLP module consists of a fully connected layer, a batch normalization layer, and an activation function, as shown in the following formula:

[0053] MLP(x i ,x j -x i )=ReLU(BN(θ m ·(x j -x i )+φ m ·x i )) (13)

[0054] Where θ=(θ1,…,θ M ,φ1,…,φ M ) is the learnable weight parameter in the fully connected layer (FC), BN(·) is the batch normalization operation, and ReLU=max(0,x) is a common nonlinear activation function in deep learning;

[0055] Step 3.2: Since point cloud data is disordered, the network that processes point cloud data needs to be permutation invariant, that is, it is insensitive to the order of input points. Therefore, in the aggregation process of edge features, the aggregation function A(·) uses the maximum pooling operation, which is expressed as:

[0056]

[0057] The edge feature ε after aggregation i ' j With the center point feature x i Splicing to obtain the local neighborhood graph feature G;

[0058] Step 3.3: Transform the neighborhood graph feature G through Φ pos (·) Function processing, integrating local and global context information to obtain context features Φ pos (·) is a Transformer structure with residual connections. A Point Transformer layer is added after the first MLP. The vector self-attention mechanism is used to flexibly assign attention weights to the edge features of each channel, allowing the center point to dynamically capture the more critical edge features in the neighborhood graph.

[0059] Using the grouped vector attention operator GAV, we first use linear projection or MLP to transform the midpoint feature x i and edge feature ε′ ijProject to ψ(ε′ ij ) and α(ε′ ij ) as the query, key, and value respectively, and the value vector The channels are evenly divided into g groups, 1≤g≤D, and the query and key are subtracted to generate a vector that can modulate a single feature channel, and an additional relative position encoding δ(p i -p j ) Again learn the absolute position relationship of the local point set, and the attention weight encoding formula is as follows:

[0060] w ij =γ(δ mul (p i -p j )⊙(qk)+δ bias (p i -p j )) (15)

[0061] in, is a learnable weight encoding function, ⊙ represents element-wise multiplication, p i and p j is the center point x i and neighboring points x j The coordinates of . is the multiplication and shift position encoding function MLP, q is the query vector, k is the key vector;

[0062] The γ(·) function consists of a grouped linear layer ζ(·), a batch normalization layer, an activation layer, and a fully connected layer, allowing information exchange between different groups. Different groups of input vectors r are independently projected using different parameters, as shown in the following formula:

[0063] γ(r)=ReLU(BN(ζ(r)) (16)

[0064]

[0065] Where D g =D / g is the number of feature channels for each group, is a learnable parameter, and the input vector BN(·) is the batch normalization layer, ReLU(·) is the activation function;

[0066] Step 3.4: Group vector attention aggregation. The output of attention weight encoding is the group attention vector of the g channel instead of the D channel. The channels of v in the same attention group share the same scalar attention weight from the group attention vector. The group vector aggregation formula is as follows:

[0067]

[0068] in, is the output feature vector, l is the current group number, and m is the learnable parameter The channel number of the

[0069] The process of step 4 is as follows:

[0070] Step 4.1: In each layer of feature propagation module, the inverse distance weighted average formula IDW is used to splice the global features across layers to the point set of the previous layer. The inverse distance weight method depends on the power value of the inverse distance. The closer the known point is to the unknown point, the greater the influence on the interpolation result. The process is as follows:

[0071] 4.1.1) Let the known point set be X = {x i |i=1,…,N l}, where N l is the number of points at level l, each point x i =(p i ,f i ) include the spatial coordinates p i =(x i ,y i ,z i ) and the eigenvector f i , the goal is to convert f i The feature is propagated to level l-1, N l <N l-1 ;

[0072] 4.1.2) Calculate N l-1 Level point set x and N l For each point x in the point set i The Euclidean distance d(p,p i );

[0073] 4.1.3) For each N l Level point x i , and assign the weight ω to it as:

[0074]

[0075] Among them, q is the power parameter, which controls the influence of distance on weight and is set to 2, ε=10 -5 To avoid parameters with denominator equal to 0;

[0076] 4.1.4) Select the k x points closest to point x i Point, x i The normalized weights and features f i Weighted to get the feature f of the jth point x (j) (x):

[0077]

[0078] Among them, k is the number of neighboring points, which is set to 3;

[0079] Step 4.2: Update the feature vector of each point using the unit feature transfer network, which consists of a fully connected layer with shared weights, a batch normalization layer, and a ReLU layer. Repeat the interpolation and transfer process until the feature is propagated to the point set of the original resolution. Finally, reduce the network complexity through a Dropout layer with a discard rate of 0.5, and then transform the feature dimension to C through MLP. The segmentation score matrix output by softmax is as follows:

[0080]

[0081] Here, Ψ is a matrix of dimension N×C, where the element y ij Represents the predicted probability that the i-th point belongs to the j-th semantic category;

[0082] Step 4.3: The back propagation of the point cloud segmentation network uses the cross entropy loss function, which is calculated as follows:

[0083]

[0084] Among them, N is the number of point cloud points, C is the number of semantic categories, and y ij is the true label of the i-th point, y ij is the predicted probability that the i-th point belongs to the j-th semantic category.

[0085] The technical concept of the present invention is: the existing point cloud segmentation technology faces difficulties in processing complex geometric structures and capturing contextual information. On the one hand, point cloud data is disordered, sparse and unevenly dense, and existing methods are difficult to achieve a balance in modeling local features and global information; on the other hand, traditional methods have information loss problems when dealing with local neighborhood relationships, and deep neural networks are prone to overfitting due to excessive parameters. The present invention combines a geometric affine module to normalize the local neighborhood, constructs a neighborhood graph through a graph neural network to capture local topological information, and then uses a residual network to extract deep edge features, and effectively fuses local and global context features through grouped vector attention.

[0086] The beneficial effects of the present invention are mainly manifested in that while significantly improving the segmentation accuracy, the method of the present invention reduces the model calculation efficiency, enhances the generalization ability of the model, and is suitable for efficient point cloud semantic segmentation in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1This is a principle block diagram of the 3D point cloud semantic segmentation method based on graph convolution and group vector attention mechanism.

[0088] Figure 2 It is a schematic diagram of sampling block and batch construction.

[0089] Figure 3 It is a schematic diagram of constructing a neighborhood graph to extract edge features.

[0090] Figure 4 It is a schematic diagram of the feature extraction and fusion module.

[0091] Figure 5 It is a schematic diagram of cross-layer feature interpolation.

[0092] Figure 6 This is a schematic diagram of feature propagation to the original resolution to complete segmentation prediction.

[0093] Figure 7 This is a schematic diagram of the various parts of mIoU.

[0094] Figure 8 It is the visualization result of semantic segmentation of point cloud scene. DETAILED DESCRIPTION

[0095] The present invention will be further described below in conjunction with the accompanying drawings.

[0096] Reference Figure 1 to Figure 6 , a 3D point cloud semantic segmentation method based on graph convolution and group vector attention mechanism, comprising the following steps:

[0097] Step 1: Preprocess the original point cloud data, including feature enhancement, category weight balancing, sampling, batch construction and feature standardization. Consider a point cloud dataset with N points, represented by X = {x i |i=1,…,N}, where each point x i =(p i ,f i ) include spatial coordinates and the eigenvector The size of D depends on the type of sensor. Usually, the laser radar scans the 3D point cloud data in the Cartesian coordinate system, that is, each point has only three basic dimensions p i =(x i ,y i ,z i ), respectively representing the position coordinates of the points, and the feature vector f contained in the dataset used i For RGB color features, the process is:

[0098] Step 1.1: Use the point cloud processing library PCL to calculate the normal vector of the point cloud, introduce additional feature dimensions to enhance the expression ability of the original point cloud data, and use the normal vector information to provide more refined local geometric features;

[0099] Step 1.2: Count the number of points in each category in the original point cloud data to obtain the frequency distribution vector And calculate the weight distribution vector based on it, as shown in the following formula:

[0100]

[0101] Where C is the total number of semantic categories, that is, the number of different categories in the dataset (such as wall, floor, table, etc.), h i is the number of points belonging to the i-th category in the point cloud;

[0102] Since some categories in the point cloud have too few points, their feature information is more difficult to learn than categories with more points. Therefore, it is necessary to calculate a balanced weight for each category and scale the weight of each category by the maximum weight ratio and cube root smoothing. The formula is as follows:

[0103]

[0104] Among them, w j is the scaled weight of the j-th category, and w is the weight distribution vector;

[0105] Step 1.3: Since the point distribution of different point clouds varies greatly, it is necessary to select a portion of points according to the point distribution to input into the network during training. Therefore, the sampling probability p is assigned to the i-th point cloud. i :

[0106]

[0107] Among them, N i is the number of points in the i-th point cloud, R is the total number of point clouds, and the sampling times M of the i-th point cloud can be obtained according to the sampling probability i for:

[0108]

[0109] Among them, r s is the sampling rate, N s is the number of single sampling points, all set by hyperparameters;

[0110] Step 1.4: Randomly select a point as the center point x of the current point cloud c =(x c ,y c ,z c ), such as Figure 2The method shown here constructs a cubic sampling block with a length and width of block_size and completes sampling within the area:

[0111]

[0112] Where b is the size of a cubic sampling block block_size, (x i ,y i ) is the i-th point x i When the number of points in a block is insufficient, random sampling with replacement is used to fill it up to the single sampling point number N. s ;

[0113] In the actual training process, the point cloud data is divided into several batches and input into the neural network one by one to increase the optimization efficiency. A batch consists of batch_size sampling blocks, each sampling block contains a fixed number of points num_point, which is numerically equal to N s , batch_size and num_point are hyperparameters that are reasonably set according to the video memory capacity to maximize the utilization of video memory while ensuring training stability;

[0114] Step 1.5: Normalize and decentralize the sampled point cloud, and normalize the RGB values, and finally construct a point cloud with a feature vector dimension of 12. Each point x i The eigenvector f i as follows:

[0115]

[0116] Where (x c ,y c ,z c ) is the decentralized coordinate, (n x ,n y ,n z ) is the normalized coordinate, (n R ,n G ,n B ) is the normalized RGB value, is the normal vector;

[0117] Step 2: Local neighborhood graph construction, including downsampling, grouping, affine transformation and graph structure modeling. First, representative points are uniformly selected from the original point cloud through the farthest point sampling FPS to reduce data complexity while retaining geometric features. Then, a local neighborhood is defined for each sampling point, and the k-nearest neighbor algorithm is used to obtain its nearest neighborhood point set; then, a geometric affine module is introduced to decentralize and standardize the neighborhood features to overcome the sparse and irregular geometric structure of the local area of ​​the point cloud; finally, by constructing a local neighborhood graph, designing an asymmetric edge feature learning function, combining global features and local differences, explicitly modeling the local topological structure and global relationship of the point cloud, and fully extracting edge features. The process is:

[0118] Step 2.1: Downsampling is used to select some representative points from a relatively dense point set as sampling points. Under the premise of ensuring that the topological structure of the point cloud remains unchanged, fewer data points are used to represent the point cloud. In order to ensure that the geometric features of the point cloud can still be retained at low resolution, the sampling process covers the point set space as evenly as possible; compared with random sampling, the farthest point sampling FPS has an advantage in coverage. The algorithm flow is as follows:

[0119] 2.1.1) Given a point cloud A sample point set S = {s i |i=1,…,M}, initialize the candidate farthest point set S t ={s1}, where s1 is randomly selected and removed from the original point set S;

[0120] 2.1.2) Select the next point as the farthest point p, which must satisfy the following constraints:

[0121]

[0122] The constraint means: t The point closest to S in the middle is used as the benchmark, the Euclidean distance between the points in the two sets is calculated, and the point with the closest distance S in S is selected. t The largest point is taken as the farthest point and added to S t , and delete this point in S at the same time, where d(p,S) is the Euclidean distance between the farthest point p and the point set S, and d(q,s i ) is S t The midpoint q and the midpoint s of S i The Euclidean distance between them is expressed as follows:

[0123]

[0124] Among them, p t =(x n ,y n ,z n ) is the set St The coordinate vector of the midpoint q, p = (x m ,y m ,z m ) is the midpoint s of the set S i The coordinate vector of , ||·||2 represents the 2-norm;

[0125] 2.1.3) Repeat 2.1.1) to 2.1.2) above to update the farthest point set S t , until S t The number of points reaches the number of downsampling points N;

[0126] Step 2.2: In order to extract local features, it is necessary to define a neighborhood range for each sampling point in the point cloud. Given a sample set S = {s i |i=1,…,M}, the subset obtained by downsampling is S t ={s i |i=1,…,N}, taking each sampling point as the "center of mass", the k nearest neighbor (kNN) algorithm is used to search for the k points closest to the center of mass to form a grouping point set

[0127] Step 2.3: The geometric structures of different local regions may be significantly different. It is difficult to fully capture the diversity of such local regions by directly using the residual network with shared weights, which in turn affects the expressiveness of the model. Therefore, a geometric affine module is introduced to decentralize and standardize the local features. To obtain the local neighborhood features containing k neighboring points, use the following formula to perform affine transformation on the neighboring point features of the local neighborhood:

[0128]

[0129] in, and is a learnable parameter, ⊙ represents element-wise multiplication (Hadamard product), ε=10 -5 It is a numerical stability parameter in the normalization operation. σ is a scalar that describes the characteristic deviation between all local region neighboring points and channels, as shown below:

[0130]

[0131] Step 2.4: For a local neighborhood of a point cloud, its essence is still an isolated and unordered set of points between points, lacking explicit local topological structure and global relationship. Therefore, the local neighborhood is modeled by constructing a graph structure; Consider a local area of ​​a point cloud The corresponding directed graph is represented as Where V = {1, 2, ..., k} means that k local neighboring points are regarded as vertices, As an edge, this local neighborhood graph can be expressed as χ(i) = {j:(i,j)∈E}∪{i}. This graph contains self-loops, that is, each node also points to itself. Define the edge feature as ε ij =f Θ (x i ,x j ),in is a nonlinear function of a set of learnable parameters Θ. Θ Defined as:

[0132] ε ij =f Θ (x i ,x j )=f Θ (x i ,x j -x i ) (11)

[0133] This asymmetric edge feature learning function combines the center point x i The captured global features and relative feature differences x j -x i The local neighborhood information captured, such as Figure 3 shown.

[0134] Step 3: Extract local neighborhood graph features based on residual network and fuse context features based on grouped vector attention. On the basis of the local neighborhood graph, edge features are first extracted through a multi-layer residual network with shared weights, and after the edge features are aggregated by the maximum pooling operation, they are concatenated with the center point features to form local neighborhood graph features. Subsequently, the Transformer structure with residual connections is used to dynamically allocate the attention weights of feature channels through the grouped vector attention (GAV) mechanism to flexibly capture key edge features. At the same time, relative position encoding is introduced to learn the absolute position relationship of the neighborhood point set. This feature extraction and fusion module is as follows: Figure 4 As shown, its core operation is abstractly represented as:

[0135]

[0136] Among them, Φ pre (·) is the local feature extraction function, A(·) is the aggregation function, Φ pre (·) is the context feature fusion function;

[0137] The process of step three is:

[0138] Step 3.1: Use the function Φ for the local neighborhood graph structure pre (·) Extract edge features, Φ pre (·) is the abstract function f in step 2.4Θ The specific implementation of (·) is that the mapping function consists of three layers of weight-sharing residual blocks ResP Block stacked together. The residual block is two homogeneous residual MLP modules, expressed as MLP(x)+x. Each MLP module consists of a fully connected layer, a batch normalization layer, and an activation function, as shown in the following formula:

[0139] MLP(x i ,x j -x i )=ReLU(BN(θ m ·(x j -x i )+φ m ·x i )) (13)

[0140] Where θ=(θ1,…,θ M ,φ1,…,φ M ) is the learnable weight parameter in the fully connected layer (FC), BN(·) is the batch normalization operation, and ReLU=max(0,x) is a common nonlinear activation function in deep learning;

[0141] Step 3.2: Since point cloud data is disordered, the network that processes point cloud data needs to be permutation invariant, that is, it is insensitive to the order of input points. Therefore, in the aggregation process of edge features, the aggregation function A(·) uses the maximum pooling operation, which is expressed as:

[0142]

[0143] The edge feature ε after aggregation i ' j With the center point feature x i Splicing to obtain the local neighborhood graph feature G;

[0144] Step 3.3: Transform the neighborhood graph feature G through Φ pos (·) Function processing, integrating local and global context information to obtain context features Φ pos (·) is a Transformer structure with residual connections. A Point Transformer layer is added after the first MLP. The Vector Self Attention mechanism can flexibly assign attention weights to the edge features of each channel, allowing the center point to dynamically capture the more critical edge features in the neighborhood graph. Since the residual network in step 3.1 causes a sharp increase in the model depth and the number of channels, the number of weight encoding parameters also increases sharply, which may lead to serious overfitting and limit the generalization ability of the model.

[0145] Therefore, the present invention uses the grouping vector attention operator GAV, whose calculation relationship is as follows: Figure 4 As shown, firstly, linear projection or MLP is used to transform the midpoint feature x i and edge feature ε i ' j Project to ψ(ε′ ij ) and α(ε′ ij ) as query, key, and value respectively, and the value vector The channels are evenly divided into g groups, 1≤g≤D, and the query and key are subtracted to generate a vector that can modulate a single feature channel, and an additional relative position encoding δ(p i -p j ) Again learn the absolute position relationship of the local point set, and the attention weight encoding formula is as follows:

[0146] w ij =γ(δ mul (p i -p j )⊙(qk)+δ bias (p i -p j )) (15)

[0147] in, is a learnable weight encoding function, ⊙ represents element-wise multiplication (Hadamard product), p i and p j is the center point x i and neighboring points x j The coordinates of . is the multiplication and shift positional encoding function (MLP), q is the query vector, and k is the key vector;

[0148] Specifically, the γ(·) function consists of a grouped linear layer ζ(·), a batch normalization layer, an activation layer, and a fully connected layer, allowing information exchange between different groups. Different groups of input vectors r are independently projected using different parameters, as shown in the following formula:

[0149] γ(r)=ReLU(BN(ζ(r)) (16)

[0150]

[0151] Where D g =D / g is the number of feature channels for each group, is a learnable parameter, and the input vector BN(·) is the batch normalization layer, ReLU(·) is the activation function;

[0152] Step 3.4: Group vector attention aggregation, the output of the attention weight encoding is the group attention vector of the g channel instead of the D channel, and the channels of v in the same attention group share the same scalar attention weight from the group attention vector. The group vector aggregation formula is as follows:

[0153]

[0154] in, is the output feature vector, l is the current group number, and m is the learnable parameter The channel number of

[0155] Step 4: hierarchical upsampling feature decoding, including inverse distance weighted average interpolation and feature propagation, to complete segmentation output and back propagation. The above-mentioned downsampling, grouping, affine transformation, local feature extraction and context feature fusion modules together constitute a feature abstraction layer. After four layers of recursive feature abstraction, a low-resolution global point cloud feature vector is obtained, which contains rich contextual semantic information. Since point cloud segmentation is an intensive classification task, it is necessary to classify the feature information of the points point by point, so a structure that couples the feature encoder and decoder is adopted. The feature decoder includes four layers of feature propagation modules, which interpolate the sparse point set features containing high-dimensional semantic features across levels to the point set on the previous level. The process is:

[0156] Step 4.1: In each layer of feature propagation module, the inverse distance weighted average formula (IDW) is used to splice the global features across layers to the point set of the previous layer. The inverse distance weight method mainly depends on the power value of the inverse distance. The closer the known point is to the unknown point, the greater the impact on the interpolation result. For example, Figure 5 As shown, the algorithm flow is as follows:

[0157] 4.1.1) Let the known point set be X = {x i |i=1,…,N l}, where N l is the number of points at level l, each point x i =(p i ,f i ) include the spatial coordinates p i =(x i ,y i ,z i ) and the eigenvector f i , the goal is to convert f i The feature is propagated to level l-1, N l <N l-1 ;

[0158] 4.1.2) Calculate N l-1 Level point set x and Nl For each point x in the point set i The Euclidean distance d(p,p i )

[0159] 4.1.3) For each N l Level point x i , and assign the weight ω to it as:

[0160]

[0161] Among them, q is the power parameter, which controls the influence of distance on weight and is set to 2, ε=10 -5 To avoid parameters with denominator equal to 0;

[0162] 4.1.4) Select the k x points closest to point x i Point, x i The normalized weights and features f i Weighted to get the feature f of the jth point x (j) (x):

[0163]

[0164] Among them, k is the number of neighboring points, which is set to 3;

[0165] Step 4.2: Use the unit feature transfer network to update the feature vector of each point, such as Figure 6 As shown in the figure, the network consists of a fully connected layer, a batch normalization layer, and a ReLU layer with shared weights. The interpolation and transfer process is repeated until the features are propagated to the point set of the original resolution. Finally, the Dropout layer with a discard rate of 0.5 is used to reduce the network complexity. The feature dimension is then transformed into C (the number of semantic categories) through MLP. The segmentation score matrix output by softmax is as follows:

[0166]

[0167] Here, Ψ is a matrix of dimension N×C, where the element y ij Represents the predicted probability that the i-th point belongs to the j-th semantic category;

[0168] Step 4.3: The back propagation of the point cloud segmentation network uses the cross entropy loss function, which is calculated as follows:

[0169]

[0170] Among them, N is the number of point cloud points, C is the number of semantic categories, and y ij is the true label of the i-th point, y ij is the predicted probability that the i-th point belongs to the j-th semantic category.

[0171] The scheme of the experimental comparison of this embodiment is as follows:

[0172] 1) Dataset:

[0173] In the point cloud scene semantic segmentation test experiment of the method of the present invention, the dataset used is S3DIS (Stanford Large-Scale 3D Indoor Spaces), which is a large-scale indoor point cloud dataset created by Stanford University. The dataset covers 271 room scenes in 6 different buildings, of which there are 11 categories of room scenes, including offices, conference rooms, corridors, auditoriums, open spaces, lobbies, lounges, food storage rooms, copy rooms, storage rooms and toilets; there are 13 semantic labels, including ceilings, floors, walls, beams, columns, windows, doors, tables, chairs, sofas, bookcases, boards, and miscellaneous elements; the S3DIS dataset is composed of a series of .txt files, each of which has several lines of data, each line of data represents a point, and each point has 6-dimensional features: xyz coordinate features and RGB color features;

[0174] 2) Evaluation indicators:

[0175] In point cloud segmentation tasks, to evaluate the performance of the model, the mean intersection over union (mIoU) and overall accuracy (OA) are usually used as evaluation indicators. The following is a description of the two:

[0176] Mean Intersection over Union (mIoU) is an indicator to measure the degree of overlap between the predicted segmentation result and the true semantic label, and is defined as follows:

[0177]

[0178] Among them, p ij It represents the probability of predicting category i as category j, C is the total number of categories, and the above formula is transformed into:

[0179]

[0180] Among them, TP represents the number of points whose labels are positive examples and the model predicts them to be positive examples (true positive examples), FP represents the number of points whose labels are negative examples and the model predicts them to be positive examples (false positive examples), and FN represents the number of points whose labels are positive examples and the model predicts them to be negative examples (false negative examples). The schematic diagram is as follows Figure 7 As shown;

[0181] Overall Accuracy (OA) evaluates the classification accuracy of the model for all points, that is, the ratio of the number of correctly predicted points to the total number of points in all test sets. The formula is:

[0182]

[0183] Among them, TP is the number of points correctly classified as category i, and N is the total number of points.

[0184] 3) Experiment 1: This experiment is based on the single area validation method and the 6-fold cross validation method to compare the method of the present invention with the existing methods. The S3DIS data set is divided into six areas, Area_1 to Area_6, according to different buildings. The single area validation method uses Area_5 as a test set and the remaining areas as training sets. For the 6-fold cross validation method, one of the areas is selected as the test set each time, and the remaining areas are used as training sets. This is repeated six times and the average result is taken as the final performance indicator. Table 1 is the training parameter table of the method of the present invention;

[0185]

[0186] Table 1

[0187] The training parameters used by the method of the present invention are shown in Table 1. Before the data is input into the neural network, the point cloud is rotated, translated, scaled, and random noise is added to improve the robustness of the model. After the training is completed, the model is evaluated using the test set, and the sliding window method is used to test the segmentation prediction results. After the test is completed, the software MeshLab is used to visualize the segmentation prediction results. The visualization results of some rooms in the Area_5 area are shown in Figure 1. Figure 8 The experimental results of calculating mIoU and OA using single region validation are shown in Table 2, and the experimental results using 6-fold cross validation are shown in Table 3.

[0188] Table 2 shows the verification results of a single area (Area_5) based on the S3DIS dataset;

[0189]

[0190] Table 2

[0191] Table 3 shows the 6-fold cross validation results based on the S3DIS dataset;

[0192]

[0193] Table 3

[0194] From the above experimental results, it can be seen that based on the performance indicators in the two evaluation modes, the method of the present invention is superior to some existing models. In the Area_5 regional validation, the mIoU and OA of the method of the present invention reached 65.4% and 88.1%. In the 6-fold cross-validation, the mIoU and OA of the method of the present invention reached 68.1% and 88.5%, which is several percentage points higher than the models with better performance such as GACNet and PointWeb. Figure 6 The semantic segmentation results in different scenes such as conference rooms, corridors, and lounges are shown. It can be seen that the segmentation results of this method are generally close to the true labels, especially the segmentation effect of boundary details and small objects is better.

[0195] 4) Experiment 2: This experiment is an ablation experiment. Different model variants are designed. By gradually adding innovative modules, the independent contribution and role of each module are analyzed. The S3DIS dataset is still used and the evaluation is performed in the single area of ​​Area_5. The comparison baseline method of this experiment is the PointNet++ basic framework, which only contains the basic point cloud segmentation structure. The experimental results are shown in Table 4, where GNN represents graph neural network, ResNet represents residual network, and GVA represents group vector attention mechanism.

[0196] Table 4 shows the model ablation experiment (based on Area_5 single area verification);

[0197]

[0198] Table 4

[0199] From the above ablation experiments, it can be seen that adding a neighborhood graph structure to the baseline method (PointNet++) to capture the geometric relationship of the local point cloud improves the mIoU by 2.5%. By introducing affine transformation and deep residual network (ResNet) structure, deeper edge features can be captured, which improves the mIoU by 5.7%. Finally, the group vector attention (GVA) is added to achieve selective aggregation of edge features, which improves the mIoU by 3.7%. The addition of each module can significantly improve the performance of the model, proving that the performance of the complete model (PointNet+++GNN+ResNet+GVA) is significantly higher than the effect of adding a single module, and the synergy between modules is significant, which can verify the advancement and innovation of the method of the present invention.

[0200] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A 3D point cloud semantic segmentation method based on graph convolution and group vector attention mechanism, characterized in that: The method comprises the following steps: Step 1: Preprocess the original point cloud data, including feature enhancement, category weight balancing, sampling, batch construction and feature standardization. Consider a point cloud dataset with N points, represented by X = {x i |i=1,…,N}, where each point x i =(p i ,f i ) include spatial coordinates and the eigenvector The size of D depends on the type of sensor; the laser radar scan obtains three-dimensional point cloud data in the Cartesian coordinate system, that is, each point has only three basic dimensions p i =(x i ,y i ,z i ), respectively representing the position coordinates of the points, and the feature vector f contained in the dataset used i is the RGB color feature; Step 2: Local neighborhood graph construction, including downsampling, grouping, affine transformation and graph structure modeling; first, representative points are uniformly selected from the original point cloud through the farthest point sampling FPS to reduce data complexity while retaining geometric features; then, a local neighborhood is defined for each sampling point, and the k-nearest neighbor algorithm is used to obtain its nearest neighborhood point set; then, a geometric affine module is introduced to decentralize and standardize the neighborhood features to overcome the sparse and irregular geometric structure of the local area of ​​the point cloud; finally, by constructing a local neighborhood graph and designing an asymmetric edge feature learning function, the local topological structure and global relationship of the point cloud are explicitly modeled, and the edge features are fully extracted; Step 3: The local neighborhood graph feature extraction based on the residual network is combined with the context feature fusion based on the grouped vector attention. On the basis of the local neighborhood graph, the edge features are first extracted through a multi-layer residual network with shared weights, and after the edge features are aggregated by the maximum pooling operation, they are concatenated with the center point features to form the local neighborhood graph features. Subsequently, the Transformer structure with residual connections is used to dynamically allocate the attention weights of the feature channels through the grouped vector attention GAV mechanism to flexibly capture the key edge features. At the same time, relative position encoding is introduced to learn the absolute position relationship of the neighborhood point set. Step 4: hierarchical upsampling feature decoding, including inverse distance weighted average interpolation and feature propagation, to complete segmentation output and back propagation; downsampling, grouping, affine transformation, local feature extraction and context feature fusion modules together constitute a feature abstraction layer. After four layers of recursive feature abstraction, a low-resolution global point cloud feature vector is obtained, which contains rich contextual semantic information; since point cloud segmentation is an intensive classification task, it is necessary to classify the feature information of the points point by point. Therefore, a structure that couples the feature encoder and decoder is adopted. The feature decoder includes four layers of feature propagation modules. Through interpolation, the sparse point set features containing high-dimensional semantic features are spliced ​​across levels to the point set of the previous layer.

2. The three-dimensional point cloud semantic segmentation method based on graph convolution and group vector attention mechanism as claimed in claim 1, characterized in that: The process of step one is: Step 1.1: Use the point cloud processing library PCL to calculate the normal vector of the point cloud, introduce additional feature dimensions to enhance the expression ability of the original point cloud data, and use the normal vector information to provide more refined local geometric features; Step 1.2: Count the number of points in each category in the original point cloud data to obtain the frequency distribution vector And calculate the weight distribution vector based on it, as shown in the following formula: Among them, C is the total number of semantic categories, that is, the number of different categories in the dataset, and h i is the number of points belonging to the i-th category in the point cloud; The weight of each category is scaled by the maximum weight ratio and cube root smoothing, as follows: Among them, w j is the scaled weight of the j-th category, and w is the weight distribution vector; Step 1.3: Since the point distribution of different point clouds varies greatly, it is necessary to select a portion of points according to the point distribution to input into the network during training. Therefore, the sampling probability p is assigned to the i-th point cloud. i : Among them, N i is the number of points in the i-th point cloud, R is the total number of point clouds, and the sampling times M of the i-th point cloud are obtained according to the sampling probability. i for: Among them, r s is the sampling rate, N s is the number of single sampling points, all set by hyperparameters; Step 1.4: Randomly select a point as the center point x of the current point cloud c =(x c ,y c ,z c ), construct a cubic sampling block with a length and width of block_size, and complete the sampling in this area: Where b is the size of a cubic sampling block block_size, (x i ,y i ) is the i-th point x i When the number of points in a block is insufficient, random sampling with replacement is used to fill the number of single sampling points N. s ; In the actual training process, the point cloud data is divided into several batches and input into the neural network one by one to increase the optimization efficiency; a batch consists of batch_size sampling blocks, each sampling block contains a fixed number of points num_point, which is numerically equal to N s , batch_size and num_point are hyperparameters set according to the video memory capacity to maximize the utilization of video memory while ensuring training stability; Step 1.5: Normalize and decentralize the sampled point cloud, and normalize the RGB values, and finally construct a point cloud with a feature vector dimension of 12. Each point x i The eigenvector f i as follows: Where (x c ,y c ,z c ) is the decentralized coordinate, (n x ,n y ,n z ) is the normalized coordinate, (n R ,n G ,n B ) is the normalized RGB value, is the normal vector.

3. The three-dimensional point cloud semantic segmentation method based on graph convolution and group vector attention mechanism according to claim 1 or 2, characterized in that: The process of step 2 is: Step 2.1: Downsampling is used to select some representative points from a relatively dense point set as sampling points, and use fewer data points to represent the point cloud while ensuring that the topological structure of the point cloud remains unchanged; Step 2.2: In order to extract local features, it is necessary to define a neighborhood range for each sampling point in the point cloud. Given a sample set S = {s i |i=1,…,M}, the subset obtained by downsampling is S t ={s i |i=1,…,N}, taking each sampling point as the "center of mass", the k nearest neighbor algorithm is used to search for the k point sets closest to the center of mass to form a grouped point set Step 2.3: Introduce the geometric affine module to decentralize and standardize local features. To obtain the local neighborhood features containing k neighboring points, use the following formula to perform affine transformation on the neighboring point features of the local neighborhood: in, and is a learnable parameter, ⊙ represents element-wise multiplication, ε=10 -5 It is a numerical stability parameter in the normalization operation. σ is a scalar that describes the characteristic deviation between all local region neighboring points and channels, as shown below: Step 2.4: For a local neighborhood of a point cloud, its essence is still an isolated and unordered set of points between points, lacking an explicit local topological structure and global relationship. Therefore, the local neighborhood is modeled by constructing a graph structure. Consider a local area of ​​a point cloud The corresponding directed graph is represented as Where V = {1, 2, ..., k} means that k local neighboring points are regarded as vertices, As an edge, this local neighborhood graph can be expressed as χ(i) = {j:(i,j)∈E}∪{i}. The graph contains self-loops, that is, each node also points to itself, and the edge feature is defined as ε ij =f Θ (x i ,x j ), where f Θ : is a nonlinear function with a set of learnable parameters Θ, and f Θ Defined as: ε ij =f Θ (x i ,x j )=f Θ (x i ,x j -x i ) (11) This asymmetric edge feature learning function combines the center point x i The captured global features and relative feature differences x j -x i Captured local neighborhood information.

4. The three-dimensional point cloud semantic segmentation method based on graph convolution and grouping vector attention mechanism as claimed in claim 3, characterized in that: In step 2.1, the process of sampling FPS at the farthest point is as follows: 2.1.1) Given a point cloud A sample point set S = {s i |i=1,…,M}, initialize the candidate farthest point set S t ={s1}, where s1 is randomly selected and removed from the original point set S; 2.1.2) Select the next point as the farthest point p, which must satisfy the following constraints: The constraint means: t The point closest to S in the middle is used as the benchmark, the Euclidean distance between the points in the two sets is calculated, and the point with the closest distance S in S is selected. t The largest point is taken as the farthest point and added to S t , and delete this point in S at the same time, where d(p,S) is the Euclidean distance between the farthest point p and the point set S, and d(q,s i ) is S t The midpoint q and the midpoint s of S i The Euclidean distance between them is expressed as follows: Among them, p t =(x n ,y n ,z n ) is the set S t The coordinate vector of the midpoint q, p = (x m ,y m ,z m ) is the midpoint s of the set S i The coordinate vector of , ||·||2 represents the 2-norm; 2.1.3) Repeat the above steps 2.1.1) to 2.1.2) to update the farthest point set S t , until S t The number of points reaches the downsampling point number N.

5. The three-dimensional point cloud semantic segmentation method based on graph convolution and group vector attention mechanism according to claim 1 or 2, characterized in that: In step 3, the core operation of the feature extraction and fusion module is abstractly represented as follows: Among them, Φ pre (·) is the local feature extraction function, A(·) is the aggregation function, Φ pre (·) is the context feature fusion function, and the process is as follows: Step 3.1: Use the function Φ for the local neighborhood graph structure pre (·) Extract edge features, Φ pre (·) is the abstract function f in step 2.4 Θ The specific implementation of (·) is that the mapping function consists of three layers of weight-sharing residual blocks ResP Block stacked together. The residual block is two homogeneous residual MLP modules, expressed as MLP(x)+x. Each MLP module consists of a fully connected layer, a batch normalization layer, and an activation function, as shown in the following formula: MLP(x i ,x j -x i )=ReLU(BN(θ m ·(x j -x i )+φ m ·x i )) (13) Where θ=(θ1,…,θ M ,φ1,…,φ M ) is the learnable weight parameter in the fully connected layer FC, BN(·) is the batch normalization operation, and ReLU=max(0,x) is a common nonlinear activation function in deep learning; Step 3.2: Since point cloud data is disordered, the network that processes point cloud data needs to be permutation invariant, that is, it is insensitive to the order of input points. Therefore, in the aggregation process of edge features, the aggregation function A(·) uses the maximum pooling operation, which is expressed as: The edge feature ε after aggregation i ' j With the center point feature x i Splicing to obtain the local neighborhood graph feature G; Step 3.3: Transform the neighborhood graph feature G through Φ pos (·) Function processing, integrating local and global context information to obtain context features Φ pos (·) is a Transformer structure with residual connections. A Point Transformer layer is added after the first MLP. The vector self-attention mechanism is used to flexibly assign attention weights to the edge features of each channel, allowing the center point to dynamically capture the more critical edge features in the neighborhood graph. Using the grouped vector attention operator GAV, we first use linear projection or MLP to transform the midpoint feature x i and edge feature ε′ ij Project to ψ(ε′ ij ) and α(ε′ ij ) as the query, key, and value respectively, and the value vector The channels are evenly divided into g groups, 1≤g≤D, and the query and key are subtracted to generate a vector that can modulate a single feature channel, and an additional relative position encoding δ(p i -p j ) Again learn the absolute position relationship of the local point set, and the attention weight encoding formula is as follows: w ij =γ(δ mul (p i -p j )⊙(qk)+δ bias (p i -p j )) (15) Among them, γ(·): is a learnable weight encoding function, ⊙ represents element-wise multiplication, p i and p j is the center point x i and neighboring points x j The coordinates of mul (·),δ bias (·): is the multiplication and shift position encoding function MLP, q is the query vector, k is the key vector; The γ(·) function consists of a grouped linear layer ζ(·), a batch normalization layer, an activation layer, and a fully connected layer, allowing information exchange between different groups, where the grouped linear layer ζ(·): Different groups of input vectors r are independently projected using different parameters, as shown in the following formula: γ(r)=ReLU(BN(ζ(r)) (16) Where D g =D / g is the number of feature channels for each group, is a learnable parameter, and the input vector BN(·) is the batch normalization layer, ReLU(·) is the activation function; Step 3.4: Group vector attention aggregation. The output of attention weight encoding is the group attention vector of the g channel instead of the D channel. The channels of v in the same attention group share the same scalar attention weight from the group attention vector. The group vector aggregation formula is as follows: in, is the output feature vector, l is the current group number, and m is the learnable parameter The channel number of the 6. The three-dimensional point cloud semantic segmentation method based on graph convolution and grouped vector attention mechanism according to claim 1 or 2, characterized in that: The process of step 4 is as follows: Step 4.1: In each layer of feature propagation module, the inverse distance weighted average formula IDW is used to splice the global features across layers to the point set of the previous layer. The inverse distance weight method depends on the power value of the inverse distance. The closer the known point is to the unknown point, the greater the influence on the interpolation result; Step 4.2: Update the feature vector of each point using the unit feature transfer network, which consists of a fully connected layer with shared weights, a batch normalization layer, and a ReLU layer. Repeat the interpolation and transfer process until the feature is propagated to the point set of the original resolution. Finally, reduce the network complexity through a Dropout layer with a discard rate of 0.5, and then transform the feature dimension to C through MLP. The segmentation score matrix output by softmax is as follows: Here, Ψ is a matrix of dimension N×C, where the element y ij Represents the predicted probability that the i-th point belongs to the j-th semantic category; Step 4.3: The back propagation of the point cloud segmentation network uses the cross entropy loss function, which is calculated as follows: Among them, N is the number of point cloud points, C is the number of semantic categories, and y ij is the true label of the i-th point, y ij is the predicted probability that the i-th point belongs to the j-th semantic category.

7. The three-dimensional point cloud semantic segmentation method based on graph convolution and group vector attention mechanism as claimed in claim 6, characterized in that: The process of step 4.1 is as follows: 4.1.1) Let the known point set be X = {x i |i=1,…,N l }, where N l is the number of points at level l, each point x i =(p i ,f i ) include the spatial coordinates p i =(x i ,y i ,z i ) and the eigenvector f i , the goal is to convert f i The feature is propagated to level l-1, N l <N l-1 ; 4.1.2) Calculate N l-1 Level point set x and N l For each point x in the point set i The Euclidean distance d(p,p i ); 4.1.3) For each N l Level point x i , and assign the weight ω to it as: in, q is a power parameter that controls the effect of distance on weight and is set to 2, ε = 10 -5 To avoid parameters with denominator equal to 0; 4.1.4) Select the k x points closest to point x i Point, x i The normalized weights and features f i Weighted to get the feature f of the jth point x (j) (x): in, k is the number of neighboring points and is set to 3.

Images