Three-dimensional point cloud semantic segmentation method and device

Abstract

The invention discloses a three-dimensional point cloud semantic segmentation method and device. The method comprises the steps of obtaining three-dimensional point cloud data to be segmented; and inputting the semantic tag into a pre-trained segmentation model to obtain the semantic tag. The segmentation model adopts a local geometric enhancement module, a dynamic attention module and a feedforward network which are connected in sequence. The core of the method is that anisotropic graph message passing is carried out through a graph neural network, and local geometric features are explicitly enhanced; and through a grouping vector attention mechanism, a position modulation item based on a relative position between points and a density modulation item based on a local density are introduced in a combined manner, and an attention weight is dynamically adjusted, so that adaptive feature aggregation of a point cloud heterostructure and complex geometry is realized. And the segmentation precision and contour sharpness in sparse and dense mixed regions and high-curvature boundaries are effectively improved.

Description

Technical Field

[0001] This application relates to the field of point cloud analysis technology, and more specifically, to a three-dimensional point cloud semantic segmentation method and apparatus. Background Technology

[0002] 3D point cloud semantic segmentation is a core task in fields such as autonomous driving, robot navigation, and 3D reconstruction, aiming to assign a precise semantic label to each point in the point cloud. Due to the inherent disorder, sparsity, and strong non-uniformity of spatial sampling in point cloud data, efficient and accurate scene-level semantic segmentation has always been a significant challenge. Existing deep learning methods typically combine local operators with global context modules to process point clouds. Early works such as PointNet ensured permutation invariance through symmetric functions, but lacked explicit modeling of local geometry, resulting in insensitivity to details such as object boundaries and curvature. Its improved version, PointNet++, introduced local perception capabilities through hierarchical grouping and sampling, but the fidelity of its geometric features heavily depends on the sampling strategy and neighborhood selection, leading to significant performance degradation in distant sparse regions or complex geometric structures.

[0003] To enhance local geometric modeling, graph convolution and kernel-based convolution methods have been proposed. These methods improve the representation of anisotropic structures and boundaries to some extent by explicitly encoding the relationship between points and their neighborhoods. However, these methods often decouple modeling from the global context, for example, by simply connecting a global attention or pooling module after a local convolution branch. This loosely coupled design can lead to scene-level semantic inconsistencies when long-range dependencies are needed to eliminate local ambiguity. The Transformer architecture has been introduced into the point cloud processing field due to its powerful long-range dependency modeling capabilities. For example, the Point Transformer and its improved version PTv2 improve the efficiency and locality of the model by introducing relative position encoding and grouped vector attention mechanisms. However, these methods have several inherent limitations: First, key geometric information is often only indirectly injected through static position encoding, failing to be fully integrated with features before entering attention computation. This causes the attention mechanism to compute on features with weak geometric cues, leading to boundary blurring and class leakage. Second, its attention weight generation mechanism is inherently static, making it difficult to adaptively respond to the density and structural heterogeneity prevalent in point clouds. Specifically, fixed neighborhood and aggregation weights can cause over-smoothing of features in dense point cloud regions, while insufficient aggregation occurs in sparse regions, making it difficult to accurately identify small targets or high-curvature structures in the distance.

[0004] Furthermore, the computational and memory complexity of standard self-attention mechanisms is proportional to the square of the number of points, which severely hinders their application on large-scale scene-level point clouds and their deployment on resource-constrained edge devices. Although existing work has attempted to optimize efficiency from the operator or data side, these measures are often applied in isolation and fail to systematically address the issue of maintaining boundary fidelity while controlling data scale, resulting in a trade-off between accuracy and efficiency.

[0005] In summary, existing technologies suffer from the following core problems: First, there is insufficient explicit local geometric prior embedding before attention computation; second, the attention weight allocation mechanism lacks dynamic adaptability to density and geometric structure; and third, there is a lack of a systematic efficiency optimization scheme that coordinates operator-side linearization and data-side scale control. Therefore, there is an urgent need in this field for a unified framework that can collaboratively address local geometric enhancement, dynamic weight modulation, and computational scalability to achieve high-precision and high-efficiency large-scale point cloud semantic segmentation. Summary of the Invention

[0006] The purpose of this application is to overcome the shortcomings of existing technologies and provide a three-dimensional point cloud semantic segmentation method and apparatus. By enhancing local geometric features through graph neural networks and introducing a dynamic attention mechanism that modulates position and density, adaptive feature aggregation for heterogeneous point cloud structures and complex geometries is achieved, thereby significantly improving the segmentation accuracy and contour clarity in sparse, dense mixed regions and high curvature boundaries.

[0007] The objective of this application is achieved through the following technical solution: Firstly, this application proposes a three-dimensional point cloud semantic segmentation method, the method comprising: Obtain the 3D point cloud data to be segmented, which includes the 3D coordinates and initial features of multiple points; The 3D point cloud data is input into a pre-trained segmentation model to obtain the semantic label of each point; The segmentation model consists of a local geometry enhancement module, a dynamic attention module, and a feedforward network connected in sequence. The local geometry enhancement module uses a graph neural network to perform anisotropic graph message passing on the point cloud data to aggregate the geometric information of the adjacency graph and enhance the local geometric features of the point cloud data. The dynamic attention module performs grouped vector attention calculation on the enhanced local geometric features and introduces a position modulation term based on the relative position between points and a density modulation term based on the local density of points during the calculation process to dynamically adjust the attention weights. Output the semantic segmentation results of the point cloud data.

[0008] In one possible embodiment, the step of the local geometry enhancement module performing anisotropic graph message passing on point cloud data through a graph neural network includes: The local geometry enhancement module constructs a K-nearest neighbor graph for each center point in the point cloud; Calculate the edge features of the edges connecting the center point and its neighboring points. The edge features include the relative coordinates and feature differences between the center point and its neighboring points. By using a learnable neural network to transform edge features, anisotropic messages can be obtained. Aggregate messages from all neighboring points to the central point to update the central point's characteristics.

[0009] In one possible embodiment, the step of the dynamic attention module performing group vector attention calculation on the enhanced local geometric features includes: The dynamic attention module maps input features into query vectors, key vectors, and value vectors, and divides the query vectors, key vectors, and value vectors into multiple groups; Within each group, calculate the relevance score between the query vector and the key vector; The position modulation term and density modulation term are introduced as bias terms or scaling terms into the calculation of the relevance score to generate modulated attention weights; The value vector is weighted and summed using the modulated attention weights to obtain the grouped attention output; The outputs of all groups are merged to obtain the final attention features.

[0010] In one possible embodiment, the position modulation term is obtained by mapping the relative coordinates between the center point and neighboring points through a multilayer perceptron and is implemented through learnable position multipliers and position bias terms.

[0011] In one possible embodiment, the density modulation term is obtained in the following manner: Calculate the distribution statistics of points within the local neighborhood of each center point; The density modulation factor of the center point is obtained by mapping the statistics through a multilayer perceptron.

[0012] In one possible embodiment, the distribution statistics of points within a local neighborhood include the average distance between neighboring points and the center point, the variance of the distance, and the number of neighboring points.

[0013] In one possible embodiment, the method further includes, before inputting the 3D point cloud data into the segmentation model: Map the original point cloud onto a regular 3D mesh; Weighted pooling is performed on the features of points within each non-empty grid to generate representative points; The set of representative points is used as input to the segmentation model.

[0014] In one possible embodiment, the feedforward network includes two or more fully connected layers and employs residual connections.

[0015] Secondly, this application proposes a three-dimensional point cloud semantic segmentation device, the device comprising: The acquisition module is used to acquire the 3D point cloud data to be segmented. The 3D point cloud data includes the 3D coordinates and initial features of multiple points. The input module is used to input 3D point cloud data into a pre-trained segmentation model to obtain the semantic label of each point; The segmentation model consists of a local geometry enhancement module, a dynamic attention module, and a feedforward network connected in sequence. The local geometry enhancement module uses a graph neural network to perform anisotropic graph message passing on the point cloud data to aggregate the geometric information of the adjacency graph and enhance the local geometric features of the point cloud data. The dynamic attention module performs grouped vector attention calculation on the enhanced local geometric features and introduces a position modulation term based on the relative position between points and a density modulation term based on the local density of points during the calculation process to dynamically adjust the attention weights. The output module is used to output the semantic segmentation results of point cloud data.

[0016] The main solution and its various further alternatives described above can be freely combined to form multiple solutions, all of which are solutions that can be adopted and are claimed in this application; furthermore, the (non-conflicting alternatives) can also be freely combined with each other and with other alternatives. Those skilled in the art, after understanding the solution of this application, will realize from the prior art and common general knowledge that there are many combinations, all of which are technical solutions to be protected in this application, and will not be exhaustively listed here.

[0017] This application discloses a 3D point cloud semantic segmentation method and apparatus, which acquires 3D point cloud data to be segmented and inputs it into a pre-trained segmentation model to obtain semantic labels. The segmentation model employs a sequentially connected local geometry enhancement module, a dynamic attention module, and a feedforward network. Its core lies in first explicitly enhancing local geometric features through anisotropic graph message passing via a graph neural network; then, through a grouped vector attention mechanism, and jointly introducing a position modulation term based on the relative position between points and a density modulation term based on local density, dynamically adjusting the attention weights, thereby achieving adaptive feature aggregation for heterogeneous point cloud structures and complex geometries, effectively improving segmentation accuracy and contour sharpness in sparse, dense mixed regions and high curvature boundaries. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 The diagram shows a flowchart of a three-dimensional point cloud semantic segmentation method proposed in an embodiment of this application.

[0020] Figure 2 A schematic diagram of the segmentation model proposed in an embodiment of this application is shown.

[0021] Figure 3 This diagram illustrates the segmentation results on the SemanticKITTI dataset. Figure 4 A schematic diagram of the segmentation results on the S3DIS dataset Area_5 is shown. Detailed Implementation

[0022] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0023] Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Real-world point clouds are characterized by disorder, sparseness, and strong non-uniformity. Most existing point cloud segmentation methods suffer from problems such as weak pre-attention geometric modeling, static self-attention with poor adaptability to density and structure, and the quadratic time-memory overhead of self-attention hindering scene-level deployment.

[0025] To address the aforementioned technical challenges, this application proposes a 3D point cloud semantic segmentation method that embeds an anisotropic graph neural network (GNN) between a multilayer perceptron (MLP) and an attention module to pre-aggregate geometric information from the adjacency graph. It improves grouped vector attention by adding learnable position multipliers / bias terms and density contrast terms to achieve adaptive weight levels. Furthermore, it reduces time and memory overhead through the synergistic use of Performer (operator-layer linearized attention) and GridPool (data-layer regular grid partitioning + pooling point reduction). Experiments show that compared to Point Transformer v2, GDPNet improves the mean intersection-union ratio (mIoU) by 2.2 and the mean accuracy (mAcc) by 1.9 on the SemanticKITTI dataset; improves mIoU by 1.5 and mAcc by 1.4 on the S3DIS (Area 5) dataset; and improves mAcc by 0.7 and overall accuracy (OA) by 0.6 on the ModelNet40 dataset. Ablation experiments demonstrate that by integrating prior geometric knowledge, location-density joint weight modulation, and dual-path efficiency optimization, segmentation boundaries can be sharpened, global consistency can be improved, and near-linear scaling can be achieved, making it suitable for large-scale scenarios and resource-constrained deployments.

[0026] Please refer to Figure 1 , Figure 1 This paper presents a flowchart illustrating a three-dimensional point cloud semantic segmentation method proposed in an embodiment of this application. The method includes: Obtain the 3D point cloud data to be segmented, which includes the 3D coordinates and initial features of multiple points; The 3D point cloud data is input into a pre-trained segmentation model to obtain the semantic label of each point; The segmentation model consists of a local geometry enhancement module, a dynamic attention module, and a feedforward network connected in sequence. The local geometry enhancement module uses a graph neural network to perform anisotropic graph message passing on the point cloud data to aggregate the geometric information of the adjacency graph and enhance the local geometric features of the point cloud data. The dynamic attention module performs grouped vector attention calculation on the enhanced local geometric features and introduces a position modulation term based on the relative position between points and a density modulation term based on the local density of points during the calculation process to dynamically adjust the attention weights. Output the semantic segmentation results of the point cloud data.

[0027] First, the 3D point cloud data to be segmented is acquired, including the 3D coordinates and initial features of multiple points. Then, the point cloud data is input into a pre-trained segmentation model, which outputs a semantic label for each point. This segmentation model employs a sequentially connected multi-stage processing structure, mainly including a local geometry enhancement module, a dynamic attention module, and a feedforward network. In the local geometry enhancement module, a graph neural network is used to perform anisotropic graph message passing on the point cloud data, explicitly aggregating geometric information within the neighborhood, thereby enhancing the expressive power of local geometric features. This module is located after the multilayer perceptron and before the attention module, enabling the pre-injection of geometric priors such as boundaries and curvature before entering global attention computation, effectively mitigating the boundary ambiguity problem caused by insufficient geometric information embedding.

[0028] In the dynamic attention module, geometrically enhanced features are grouped into vector attention calculations. The core innovation of this module lies in the introduction of position modulation and density modulation terms, which work together to dynamically adjust the attention weights. The position modulation term calibrates the attention kernel based on the relative positions of points using learnable multipliers and bias parameters; the density modulation term adaptively adjusts the aggregation intensity based on local density differences among points, suppressing excessive smoothing in dense regions and compensating for contextual loss in sparse regions. Finally, features are further integrated through a feedforward network, and residual connections and normalization operations are used to stabilize the training process, ultimately outputting the semantic segmentation results of the point cloud data.

[0029] The local geometry enhancement module performs anisotropic graph message passing on point cloud data through a graph neural network, including: The local geometry enhancement module constructs a K-nearest neighbor graph for each center point in the point cloud; Calculate the edge features of the edges connecting the center point and its neighboring points. The edge features include the relative coordinates and feature differences between the center point and its neighboring points. By using a learnable neural network to transform edge features, anisotropic messages can be obtained. Aggregate messages from all neighboring points to the central point to update the central point's characteristics.

[0030] A local adjacency graph is constructed for each center point in the point cloud. This step uses the K-nearest neighbor algorithm to dynamically find the K nearest neighbors in space for each center point, thereby defining its local neighborhood structure and forming a graph structure. Next, the edge features of each edge in the adjacency graph are calculated. The edge features are a composite vector, which contains two core pieces of information: first, the relative coordinates between the center point and its neighbors, used to accurately describe the local spatial geometric relationships; and second, the feature difference between the center point and its neighbors, used to capture local variation patterns in the feature space.

[0031] Then, a learnable neural network (multilayer perceptron) is used to perform a nonlinear transformation on the aforementioned edge features. This step is crucial, enabling the model to learn and generate direction-sensitive, i.e., anisotropic messages based on the geometry and feature context of the edges. This shows that messages from neighboring points in different directions are processed differentially according to their local structure, thereby enhancing the model's ability to perceive directionally significant regions such as boundaries and corners. Finally, an aggregation function is used to integrate the anisotropic messages passed from all neighboring points to the center point, and the aggregated result is used to update the features of the center point.

[0032] The dynamic attention module performs group vector attention calculations on the enhanced local geometric features, including: The dynamic attention module maps input features into query vectors, key vectors, and value vectors, and divides the query vectors, key vectors, and value vectors into multiple groups; Within each group, calculate the relevance score between the query vector and the key vector; The position modulation term and density modulation term are introduced as bias terms or scaling terms into the calculation of the relevance score to generate modulated attention weights; The value vector is weighted and summed using the modulated attention weights to obtain the grouped attention output; The outputs of all groups are merged to obtain the final attention features.

[0033] First, the module maps the input enhanced local geometric features into query vectors, key vectors, and value vectors through independent linear transformations. Then, these three vectors are evenly divided into multiple groups along the feature channel dimension, forming multiple sub-queries, sub-keys, and sub-values. This grouping mechanism aims to allow the model to learn diverse contextual dependencies in parallel across different sub-feature spaces.

[0034] Within each group, the correlation between the query vector and the key vector is calculated to form an initial attention relevance score. Key position modulation and density modulation terms are introduced, used as learnable bias or scaling terms, directly influencing the calculation of the initial relevance score. The position modulation term, based on the relative coordinates between points, enhances the model's perception of spatial geometry; the density modulation term, based on the local density statistics of points, enables the model to adaptively cope with the heterogeneity of sampling density. By jointly introducing these two terms, modulated attention weights that dynamically adapt to the local geometry and density environment are generated.

[0035] Next, using this modulated attention weight, the value vectors within the same group are weighted and summed to obtain the attention output for that group. Finally, the outputs of all groups are merged along the feature channel dimension and integrated through a linear projection layer to obtain the final attention features that contain rich global context and adapt to local structure.

[0036] The position modulation term is obtained by mapping the relative coordinates between the center point and neighboring points through a multilayer perceptron and is implemented through learnable position multipliers and position bias terms.

[0037] The generation process of the position modulation term is as follows: First, the relative coordinate vector between the center point and one of its neighboring points is used as the basic input. This vector directly describes the relative positional relationship between the two points in Euclidean space. Then, this relative coordinate vector is input into a multilayer perceptron (MLP). This MLP, acting as a general nonlinear function approximator, is responsible for mapping the raw, low-level geometric displacement to a high-level, task-relevant feature representation. This mapping process can capture complex, anisotropic geometric patterns in space. Finally, to provide finer and more adaptive control, the features output by the MLP are affinely transformed with a set of learnable position multipliers and learnable position bias terms. The multiplier terms scale the features, achieving directional adjustment of attention intensity; the bias terms provide basic displacement adjustments.

[0038] The density modulation term is obtained in the following way: Calculate the distribution statistics of points within the local neighborhood of each center point; The density modulation factor of the center point is obtained by mapping the statistics through a multilayer perceptron.

[0039] First, for each center point in the point cloud, a set of statistics describing the point distribution are calculated within its local neighborhood. These statistics are key indicators for quantifying local density and distribution patterns, including but not limited to: the number of points in the neighborhood, the average distance between neighborhood points and the center point, and the variance of these distances. Then, these calculated statistics are combined into a feature vector and input into a multilayer perceptron (MLP). This MLP acts as a nonlinear transformer, responsible for mapping the information extracted from the raw statistics into a context-dependent, learnable scalar or vector—the density modulation factor.

[0040] The distribution statistics of points within a local neighborhood include the average distance between neighborhood points and the center point, the variance of the distance, and the number of neighborhood points.

[0041] Average distance between neighboring points and the center point: This is the arithmetic mean of the distances between all points in the neighborhood and the center point. This value reflects the overall sampling density of the local area; a smaller average usually indicates a denser area, and vice versa.

[0042] Distance variance: Calculate the variance of the above distances. This value characterizes the uniformity of the distribution of neighborhood points around the center point. A high variance indicates that the point distribution may be concentrated in a specific direction, while a low variance indicates that the distribution is more uniform.

[0043] Number of neighborhood points: This is the most direct measure of density, directly counting the total number of points contained within a local neighborhood.

[0044] Before inputting the 3D point cloud data into the segmentation model, the method also includes: Map the original point cloud onto a regular 3D mesh; Weighted pooling is performed on the features of points within each non-empty grid to generate representative points; The set of representative points is used as input to the segmentation model.

[0045] The acquired raw point cloud is mapped onto a regular 3D mesh space. This process involves defining a 3D mesh with a specific stride to cover the space containing the point cloud, and assigning each raw point to a corresponding mesh cell based on its 3D coordinates. Next, for each non-empty mesh cell, the features of all points contained within it are aggregated to generate a representative point for that cell. The aggregation is not a simple average or max pooling, but rather employs a weighted pooling strategy. This strategy comprehensively considers the spatial relationship between each point and the centroid of its mesh cell, typically using a distance-based kernel function for weighting, ensuring that points closer to the cell center contribute more. Furthermore, local point density can be selectively introduced as a weighting factor, incorporating density priors into feature aggregation, giving higher weight to high-density regions. Through this weighted pooling operation, each non-empty mesh cell produces a representative point with a representative location and aggregated feature vector. Finally, the set of all these representative points serves as the input to the subsequent segmentation model, replacing the original large-scale point cloud.

[0046] In one possible embodiment, to preserve the spatial location information of the grid cells, the centroid of all points within the cell is used as the representative point coordinates of that cell: ,in express The number of elements (i.e., units) (Number of original points within the cell). For feature aggregation, a weighted pooling strategy is adopted to avoid boundary errors from traditional voxel quantization. First, the weighted normalization coefficient of each point within the cell is defined: ,in The spatial distance is normalized and weighted using a kernel function. This introduces a local density prior, giving higher weight to points in high-density regions during feature aggregation. This is based on a weighted normalization coefficient. Grid unit The representative point features are: The continuity of this weighted pooling method can effectively reduce the boundary / quantization error caused by voxel quantization, while also... The adjustment can flexibly control the degree of influence of density prior.

[0047] Feedforward networks consist of two or more fully connected layers and employ residual connections.

[0048] The segmentation model includes a feedforward network, which follows the dynamic attention module and is responsible for the final integration and nonlinear transformation of features that have undergone local geometric enhancement and global attention modulation. The feedforward network consists of a sequence of fully connected layers with two or more layers. Each layer performs a linear transformation and is coupled with a nonlinear activation function, thereby performing complex mapping and information fusion in the feature space, further improving the discriminative ability of the features. A residual connection mechanism is introduced into the feedforward network, directly adding the input of the feedforward network module to the module's output, forming a fast path.

[0049] In one possible embodiment, Figure 2 This illustration shows a schematic diagram of the segmentation model proposed in an embodiment of this application. In this embodiment, the segmentation model is referred to as GDPNet (GNN–Dynamic Attention + Performer & GridPool). Given 3D point cloud data P={ Each point has three-dimensional coordinates. With initial features The basic blocks of GDPNet are connected in the order of "MLP→GNN→Dynamic Attention→FFN": first, feature extraction is completed by MLP, then local geometry is explicitly enhanced by anisotropic graph message passing, then grouped vector attention with position / density joint modulation is used to complete global information distribution, and finally the training is stabilized by feedforward network (FFN) and residual normalization.

[0050] While the Transformer architecture excels at long-range dependencies, it falls short in characterizing explicit geometry such as boundary continuity, curvature, and normals. Directly feeding the original features into the attention module amplifies the problems of weak local geometry and blurred boundaries along the propagation path. Therefore, this application introduces a Graph Neural Network (GNN), which excels at local neighborhood modeling, and integrates it after the MLP layer and before the dynamic attention module. First, the input features undergo a non-linear transformation through the MLP layer, typically using multiple fully connected layers to transform... Convert to Through the formula: ,in, Represents the activation function (ReLU). It is a weighting function. It is a bias.

[0051] The graph neural network receives the output from the MLP layer. Graph convolution operations can be performed using the adjacency matrix A and the feature matrix H (here, X'), following the following rules: , It is the weight matrix of the graph convolutional layer, which are learnable parameters. It is the sigmoid activation function.

[0052] In Graph Attention Networks (GATs), weights are dynamically calculated through an attention mechanism: in This represents the vector concatenation operation. This represents a learnable weight vector. Represents node characteristics.

[0053] After passing through the GNN layer, the features This represents the vector concatenation operation. This represents a learnable weight vector, which is then passed to the GroupedVectorAttention layer. The attention mechanism utilizes these features to perform queries. ,key ,value Attention operations are used to select and integrate information, mathematically represented as: , This represents the dimension of the key vector, used for scaling and normalization.

[0054] This integrated framework utilizes MLP to initially extract features, GNN to aggregate local information, and finally improves feature representation capabilities through an attention mechanism.

[0055] In existing Transformer frameworks, standard (and even grouped) attention typically uses static weights, indirectly introducing geometry only through relative position encoding. In point clouds with highly heterogeneous density and large curvature variations, fixed neighborhoods and static weights struggle to simultaneously address de-obfuscation in dense regions and context compensation in sparse regions. To address this, we introduce learnable position multipliers / biases and density differences as dual modulations within Grouped Vector Attention (GVA), achieving regionalized and conditional adaptive aggregation at the weight level, elevating traditional neighborhood adaptation to weight adaptation.

[0056] First, a linear transformation is performed on the input features to obtain Query, Key, and Value. A linear layer is used to transform the features into these three: , as well as In the code, through , and This is implemented by grouping the features into multiple groups within the `GroupedVectorAttention` class. The `Value` features are grouped to accommodate further attention calculations, and then reshaped using `einops.rearrange` to store the features in groups. ,in, Group by features; For indexing, it indicates the specific dimension within the subgroup.

[0057] Secondly, considering that positional encoding is particularly important in calculating the relationship between Query and Key among the natural properties of point clouds, this paper calculates the relative position ( ), and enhance it by subtracting the Query from the Key to obtain the relative position, represented as If pe_multiplier and pe_bias are enabled, they are used to adjust the positional features, represented as... ,in The coordinates are the position coordinates of the feature in the point cloud.

[0058] Subsequently, by calculating density differences, the attention mechanism can dynamically adjust weights to adapt to the local characteristics of the data, using density values ​​calculated from distance. : ,in, It is a small constant.

[0059] In the code, density differences are calculated. This allows the attention mechanism to better adapt to the characteristics of local data. Therefore, density differences... The calculation makes attention weights more flexible, which can be mathematically represented as: ,in This represents the set of reference points.

[0060] Then, attention weights are calculated by combining positional relationships and density differences. This is then applied to feature grouping. Attention weights are calculated using the encoder / decoder WeightEncoding. Then, use Softmax normalization to transform the weights into a probability distribution: Finally, a dynamic attention mechanism is applied to the input features, and the output features are weighted and fused to capture a broader and more detailed relational structure. Dynamic aggregation: This model enhances its feature representation capabilities through a dynamic adjustment mechanism, enabling a better understanding of complex geometric structures. In point cloud tasks, it improves adaptability, maintaining or even improving performance across varying environmental complexities, effectively handling sparsity and density variations, and providing broad task adaptability.

[0061] To overcome the O(N²) time and space bottleneck of standard self-attention with N points, this paper simultaneously reduces costs and increases efficiency from both the operator and data sides: on the operator side, the Performer algorithm is used to kernelize the soft attention into linear complexity; on the data side, GridPool is used for regular grid partitioning and intra-grid aggregation, directly reducing the effective number of points N. These two approaches are decoupled yet complementary; the former ensures that attention computation scales linearly with N, while the latter controls peak memory usage and end-to-end latency, making it more robust for large-scale point cloud and edge deployments. On the attention operator side, FAVOR+ linear attention is used to approximate the softmax dot product as a positive feature map. Kernel regression: ,in It is a stabilization term that ensures numerical robustness during the normalization process and avoids explicit construction. Attention matrix.

[0062] On the data side, a GridPool method based on regular grids is proposed, which achieves data size compression through grid mapping and differentiable aggregation, while preserving key geometric and feature information.

[0063] For each point in the original point cloud The corresponding raster cell index is calculated as follows: ,in This indicates that the floor function is performed on each element of the vector. A three-dimensional integer vector, representing The grid cell number to which it belongs.

[0064] Based on raster index Each grid cell can be defined. The set of original point indices contained within, and the total number of valid cells after GridPool: ,in Indicates all cells belonging to the grid. The original point index, The total number of non-empty raster cells directly determines the "active point scale" of subsequent operators—compared to the original number of points N. The grid step size can be adjusted. Flexible control is a key indicator for achieving data volume compression.

[0065] In the end-to-end integration strategy, to satisfy the dependence of GNN and attention mechanisms on "local neighborhood", in the representative point set Up through The k-nearest neighbor (kNN) algorithm reconstructs the neighborhood of each representative point: ,in Indicates representative point The set of k nearest neighbor indices, This represents the index corresponding to the k largest values. By taking the negative of the Euclidean distance, finding the nearest neighbor is transformed into finding the maximum value, which facilitates efficient computation. In practice, the adjacency indices of adjacent network layers can be cached and reused to avoid redundant calculations and improve inference efficiency.

[0066] Because the location scale (such as object size) and density distribution of point clouds vary greatly in different scenarios, directly using the original location features is not feasible. With density characteristics This can easily lead to numerical instability. Therefore, a temperature parameter is introduced for automatic scaling. ,in and The temperature parameters are for location and density features, respectively. Through adaptive learning training, the numerical normalization of features at different scales can be achieved, thereby improving the model's generalization ability and training stability.

[0067] The time complexity of standard self-attention is (in Calculate the overhead for the attention matrix. (The feature dimension) varies with the number of original points. It exhibits a secondary growth pattern, making it unsuitable for large-scale point clouds. GridPool reduces the original number of points... Compressed to the number of representative points Building upon this, Performer kernelizes the attention into a linear form through random feature mapping, which is effective in the subsampling domain (scale). The time complexity of ) is: This formula ignores the constant factor of the attention of the grouped vectors. Here, the random feature dimension is for Performer. In the grouped vector attention scenario, the channel dimension of each group is... At this point, the time complexity can be further reduced to The constant factor varies with the number of groups. It decreases linearly.

[0068] Standard self-attention requires explicit construction Attention matrix, memory usage is In large-scale point cloud scenarios, this can easily lead to memory overflow. However, Performer does not require explicit construction of the attention matrix; it operates within the subsampling domain (scale...). The upper bound of memory usage on ) is: in For the memory overhead of random feature mapping, The memory overhead for representative point features varies with... The results show linear growth. This design is consistent with the Full-Eff results in efficiency ablation, validating the deployability of this dual-path strategy without sacrificing accuracy.

[0069] In one possible embodiment, this application systematically evaluates GDPNet on three types of tasks: outdoor LiDAR scene segmentation (SemanticKITTI), indoor scene segmentation (S3DIS, Area 5), ​​and shape classification (ModelNet40). Unless otherwise specified, all results are repeated 10 times under the same training settings and averaged, and mIoU and mAcc are reported. Qualitative visualizations and three sets of ablation are also provided to verify the independent contributions and synergistic effects of each module.

[0070] First, semantic segmentation; on SemanticKITTI, GDPNet achieves 70.2% mIoU and 78.2% mAcc (Table II), an improvement of +2.2 mIoU over PTv2, and slightly higher mAcc than sparse convolutional baselines (such as SparseConvNet). By category, the model performs strongly on large-scale scene primitives and vehicle-related categories, including car (97.9%), truck (91.0%), other-vehicle (83.6%), and parking (59.4%). The road category is also competitive (94.9%). It performs well for the imbalanced "person" category, while results for the cyclist subclass (bicyclist / motorcyclist) are inconsistent. The gains for vehicle categories and drivable surfaces align with the design of GDPNet: firstly, the MLP→GNN front end enhances sensitivity to curvature and boundaries before entering attention, making the depiction of rigid object shells and road boundaries more reliable; secondly, the position × density dynamic attention modulates weights according to local sampling patterns, offsetting over-smoothing of features in dense areas (such as parked vehicles) and providing contextual compensation in sparse distances (such as distant LiDAR echoes). In contrast, vertically elongated structures (such as polls and trunks) are slightly less effective than PTv2. We hypothesize that (i) the grid pooling scale s occasionally merges extremely thin instances with the background; and (ii) density priors may cause aggregation to favor thicker regions. In practice, reducing s in high-curvature regions, employing category-aware reweighting, or adding fine-structure enhancements (such as random cylindrical expansion) can usually alleviate this problem without compromising global semantics. Error pattern analysis revealed fewer mislabeled profiles at the road–sidewalk and vehicle–vegetation interfaces compared to the baseline, consistent with the qualitative results in Figures 2 and 3. Mislabeling primarily occurred at distant, sparsely sampled cyclists near adjacent vehicles, suggesting that using adaptive k in kNN or multi-scale grouping in GVA could further improve recall of minimal human-related instances.

[0071] On S3DIS (Area 5), ​​GDPNet achieved 63.6% mIoU and 70.1% mAcc (Table III), surpassing PTv2 by +1.5 / +1.4 respectively. In terms of categories, GDPNet showed significant improvements in categories sensitive to boundaries and high curvature (e.g., column 24.8%, door 56.8%), and remained competitive in geometrically complex classes (e.g., window, table) and planar classes (ceiling / wall). In indoor scenes, occlusion and thin structures (pillars, door frames) frequently occur, making explicit local geometric enhancement crucial. By injecting neighborhood anisotropy into GNNs and modulating attention with position × density, GDPNet better maintains local continuity and reduces label leakage between adjacent instances. The slight decrease in chair / sofa values ​​is consistent with the changes in shape and material height in Area 5; we found that moderately increasing the number of GVA groups G and slightly decreasing the mesh stride s can narrow the gap, enhancing local interactions while keeping end-to-end overhead manageable. Qualitatively, the model generates cleaner segments at boundaries such as walls, doors, and windows. Figure 3 , Figure 4 Dynamic attention tends to upscale edges with high curvature contrast and sufficient local support, while downscaleing oversampled planar segments, thereby improving contour sharpness and region-level consistency.

[0072] like Figure 3 , Figure 4 As shown, GDPNet produces cleaner segmentation results at object contours and interfaces with adjacent categories. Misclassification and label leakage are significantly reduced at slender structures (such as pillars and thin plates) and openings (doors / windows), demonstrating the synergistic effect of geometric enhancement and dynamic attention based on GNN.

[0073] Second, shape classification experiments: On ModelNet40, GDPNet achieved 89.3% mAcc and 91.9% OA (Table IV), representing improvements of +0.7 and +0.6 percentage points respectively compared to PTv2. Although this task emphasizes global discrimination, performance remains sensitive to uneven sampling and fine-grained geometry. GDPNet delivers stable gains without significantly increasing computational overhead, indicating that the sequential design of "local geometric prior → global attention" benefits even in global recognition tasks.

[0074] From a design perspective, the MLP→GNN front end integrates component-level cues (such as wing vs. fuselage, chair back vs. chair leg) before global attention convergence, thereby reducing reliance on long-range interactions to resolve local ambiguities. Position × density modulation further mitigates the bias introduced by uneven sampling common in CAD datasets (different gridding strategies can create extremely high or low point density pockets locally). More importantly, Performer × GridPool maintains throughput, allowing for larger effective batches and resulting in more stable optimization dynamics; in practice, we observed fewer validation OA oscillations in later rounds when both are enabled. These factors collectively explain the improvements achieved without requiring deeper backbones or heavier data augmentation.

[0075] Third, ablation experiments: First, we examined the effect of inserting a GNN between the MLP and the attention layer. As shown in Table V, simply replacing local branches yields a stable gain as the inductive bias becomes more expressive. From no GNN to the full GDPNet, mIoU / mAcc improved from 67.9 / 76.8 to 70.2 / 78.2 (+2.3 / +1.4). General GCN already helped (+0.4 / +0.3), while anisotropic message passing (GAT / EdgeConv under fixed kNN) brought a greater improvement (+1.1 / +0.4 relative to GCN), highlighting the value of orientation-sensitive and boundary-aware aggregation. Enabling dynamic adjacency (“Full”) further improved the results (+0.8 / +0.7), demonstrating that local neighborhoods that adapt to geometry and density can produce more faithful boundary modeling and improve robustness to structural heterogeneity. Qualitatively, "fixed adjacency" settings tend to blur narrow gaps (such as door frames) and over-smooth high-curvature corners; dynamic adjacency mitigates both by redistributing edges along the main direction of change. This aligns with the motivation in Section III-A: to provide features that have been explicitly enhanced by local geometry before they enter attention.

[0076] Next, we replace only the attention module to isolate the effect of dynamic attention. The hierarchical evaluations in Tables VI and VII show that full "position + density" modulation (Full-DA) achieves the best overall performance and outperforms all hierarchies (dense / medium / sparse; low / medium / high curvature). Removing density modulation primarily harms performance in sparse and high curvature regions, which have limited contextual support and irregular sampling; removing position scaling / bias weakens boundary sensitivity and reduces gains at sharp interfaces. Compared to a completely static GVA, Full-DA achieves the largest mIoU improvement in the sparse (+2.6) and high curvature (+3.4) hierarchies, indicating that jointly utilizing density and position modulation at the weight level effectively suppresses confusion in dense clusters and compensates for contextual deficiencies in sparse regions.

[0077] From a signal processing perspective, the density term acts as an adaptive gain, downsampling locally oversampled planar segments while preserving informative discontinuities; the position multiplier / bias reparameterizes relative displacement, improving the numerical conditionality of the attention kernel near sharp edges. The two complement each other: the density term normalizes (regularizes) the amplitude of the interaction intensity, while the position term calibrates its direction and scale. We observe that increasing the number of groups G in the GVA amplifies the gains from position modulation, while denser neighborhoods (larger k) amplify the gains from the density term; both effects tend to saturate beyond a certain point, indicating a bias-variance tradeoff.

[0078] Finally, the efficiency paths are evaluated (Table VII). The full configuration (Performer + GridPool) performs best in terms of throughput, latency, and training step time (510 kpts / s, 90 ms / frame, 150 ms / step). Using only Performer (460 kpts / s, 110 ms / frame, 160 ms / step) or only GridPool with standard attention (420 kpts / s, 120 ms / frame, 170 ms / step) provides moderate gains, while removing both results in the worst performance (390 kpts / s, 130 ms / frame, 190 ms / step). This pattern is consistent with theory: Performer linearizes attention over the effective domain, while GridPool explicitly reduces the effective number of points M from N; the two are decoupled yet complementary—one reshapes the computational rules, the other controls the data scale—delivering predictable and robust speedups without sacrificing accuracy. In practice, GridPool sets an upper limit on M and stabilizes memory across different scan lengths; Performer maintains near-linear behavior on M, avoiding the double explosion of standard attention. Caches the kNN adjacency relationships of representative points across adjacent layers, reducing redundant search and contributing to the observed latency reduction. Notably, at a moderate grid stride s, these efficiency gains do not sacrifice the fidelity of fine structures; in high curvature regions, slightly reducing s recovers more details with negligible impact on end-to-end overhead.

[0079] In summary, GDPNet employs a combination of a GNN frontend and a Transformer, injecting explicit, anisotropic geometric cues before attention. Subsequently, attention is modulated at the weight level by a joint position-density signal. Simultaneously, the Performer and GridPool work together to keep computation nearly linear with respect to the number of effective points. Overall, these choices result in sharper boundaries, better long-range consistency, and stable throughput and latency across large-scale outdoor and indoor scenes, consistently achieving gains relative to PTv2 and remaining competitive with sparse convolutional baselines.

[0080] These results provide practical guidance: operator-side linearization works best when combined with data scaling control; GridPool provides a "cap" while preserving detail, and can be replaced with a more boundary-aware partition when fine structure dominates. Similarly, lightweight convolutional front-ends can enhance low-level priors without sacrificing the global context of attention, which is valuable for real-time road parsing.

[0081] Challenges remain: generalization across datasets, extremely sparse / high curvature regions, and deployment under tight memory and power budgets. Future work includes: (i) combining location-density modulation with feature decoupling, pruning / quantization, and distillation for end-to-end efficiency; (ii) incorporating multimodal alignment (intensity / image) with self-supervised pre-training; and (iii) learning strategies for grid stride, neighborhood size, and grouping to adapt to scale and density. These directions aim to improve robustness and scalability while maintaining the accuracy-efficiency balance of GDPNet.

[0082] Compared with the prior art, the embodiments of this application have the following beneficial effects: First, through anisotropic graph message passing in the local geometry enhancement module, this scheme can accurately capture the directional relationship between each point in the point cloud and its neighboring points. This deep perception of local geometry enables it to effectively distinguish points that are spatially adjacent but belong to different semantic objects, thereby achieving more accurate segmentation in complex scenes.

[0083] Second, the density modulation term introduced in the dynamic attention module enables the model to dynamically adjust the attention weights based on the local density around each point. This means that in sparse regions of the point cloud, the model enhances information aggregation to compensate for insufficient information; in dense regions, it suppresses redundant information, thereby significantly improving the model's generalization ability and segmentation stability on non-uniform point clouds scanned in the real world.

[0084] Third, a sequential closed-loop design is adopted, consisting of a local geometry enhancement module and a dynamic attention module. This design ensures that after acquiring rich local geometric features, global point relationships are established through an attention mechanism. This makes the classification decision of each point depend on its detailed local shape and also benefit from the contextual information of the entire scene, effectively reducing isolated misjudgments.

[0085] Fourth, through efficient computational design such as grouped vector attention, this scheme reduces the overhead of global attention calculations on massive point clouds. This optimization of computational efficiency makes the model not only highly accurate in segmentation but also easier to deploy in practical applications, handling large-scale 3D scenes.

[0086] The following provides a possible implementation of a 3D point cloud semantic segmentation device, which is used to perform the various execution steps and corresponding technical effects of the 3D point cloud semantic segmentation method shown in the above embodiments and possible implementations. The device includes: The acquisition module is used to acquire the 3D point cloud data to be segmented. The 3D point cloud data includes the 3D coordinates and initial features of multiple points. The input module is used to input 3D point cloud data into a pre-trained segmentation model to obtain the semantic label of each point; The segmentation model consists of a local geometry enhancement module, a dynamic attention module, and a feedforward network connected in sequence. The local geometry enhancement module uses a graph neural network to perform anisotropic graph message passing on the point cloud data to aggregate the geometric information of the adjacency graph and enhance the local geometric features of the point cloud data. The dynamic attention module performs grouped vector attention calculation on the enhanced local geometric features and introduces a position modulation term based on the relative position between points and a density modulation term based on the local density of points during the calculation process to dynamically adjust the attention weights. The output module is used to output the semantic segmentation results of point cloud data.

[0087] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for semantic segmentation of a three-dimensional point cloud, the method comprising: The method comprises: acquiring three-dimensional point cloud data to be segmented, the three-dimensional point cloud data comprising three-dimensional coordinates and initial features of a plurality of points; inputting the three-dimensional point cloud data into a pre-trained segmentation model to obtain a semantic label of each point; the segmentation model comprising a local geometry enhancement module, a dynamic attention module and a feedforward network connected in sequence; the local geometry enhancement module performing anisotropic graph message passing on the point cloud data by a graph neural network to aggregate adjacent graph geometry information and enhance local geometry features of the point cloud data; the dynamic attention module performing grouped vector attention calculation on the enhanced local geometry features and introducing a position modulation term based on relative positions between points and a density modulation term based on local density of points in the calculation process to dynamically adjust attention weights; outputting a semantic segmentation result of the point cloud data.

2. The method of claim 1, wherein, The step of the local geometry enhancement module performing anisotropic graph message passing on the point cloud data by a graph neural network comprises: the local geometry enhancement module constructing a K-nearest neighbor adjacent graph for each center point in the point cloud data; calculating edge features of edges between the center point and neighbor points based on the K-nearest neighbor adjacent graph, the edge features comprising relative coordinates and feature differences between the center point and the neighbor points; transforming the edge features by a learnable neural network to obtain anisotropic messages; aggregating all messages transmitted to the center point by the neighbor points to update features of the center point.

3. The method of claim 1, wherein, The step of the dynamic attention module performing grouped vector attention calculation on the enhanced local geometry features comprises: the dynamic attention module mapping input features into query vectors, key vectors and value vectors, and dividing the query vectors, the key vectors and the value vectors into multiple groups; in each group, calculating a correlation score between the query vectors and the key vectors; introducing the position modulation term and the density modulation term as bias terms or scaling terms into calculation of the correlation score to generate modulated attention weights; performing weighted summation on the value vectors by the modulated attention weights to obtain grouped attention outputs; merging outputs of all groups to obtain a final attention feature.

4. The method of claim 1, wherein, The position modulation term is obtained by mapping relative coordinates between the center point and the neighbor points by a multi-layer perceptron and is realized by a learnable position multiplier and a position bias term.

5. The method of claim 1, wherein, The density modulation term is obtained by: calculating distribution statistics of points in a local neighborhood of each center point; mapping the statistics by a multi-layer perceptron to obtain a density modulation factor of the center point.

6. The method of claim 5, wherein, The distribution statistics of the points in the local neighborhood of the center point comprise an average distance between the neighbor points and the center point, a variance of the distance and a number of the neighbor points.

7. The method of claim 1, wherein, Before inputting the three-dimensional point cloud data into the segmentation model, the method further comprises: mapping an original point cloud to a regular three-dimensional grid; performing weighted pooling on features of points in each non-empty grid to generate representative points; taking a set of the representative points as input of the segmentation model.

8. The method of claim 1, wherein, The feedforward network comprises two or more fully connected layers and adopts residual connection.

9. A three-dimensional point cloud semantic segmentation apparatus, characterized in that, The device comprises: an acquisition module configured to acquire three-dimensional point cloud data to be segmented, the three-dimensional point cloud data comprising three-dimensional coordinates and initial features of a plurality of points; an input module configured to input the three-dimensional point cloud data into a pre-trained segmentation model to obtain a semantic label of each point; and a semantic segmentation module configured to output a semantic segmentation result of the point cloud data. The segmentation model comprises sequentially connected local geometry enhancement modules, a dynamic attention module and a feedforward network; the local geometry enhancement modules perform anisotropic graph message passing on the point cloud data through a graph neural network to aggregate adjacent graph geometry information and enhance the local geometry features of the point cloud data; the dynamic attention module performs grouped vector attention calculation on the enhanced local geometry features, and introduces a position modulation term based on the relative positions between points and a density modulation term based on the local density of points in the calculation process to dynamically adjust the attention weights; The output module is configured to output the semantic segmentation result of the point cloud data.

Images