Point cloud primitive semantic segmentation method based on overall and local geometric attention fusion
By fusing global and local geometric attention, an improved Geometric Transformer network is developed, which solves the problems of high computational complexity and insufficient segmentation accuracy of existing models, and achieves efficient and high-precision point cloud primitive segmentation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-10
- Publication Date
- 2026-03-13
AI Technical Summary
Existing GeometricTransformer models have high computational complexity in point cloud processing, and their attention mechanisms are insufficient when fusing geometric information, which limits the segmentation accuracy on complex industrial point clouds.
We employ a fusion approach of global and local geometric attention. By using an improved Geometric Transformer network, we decompose the computation of local and global geometric word embeddings, optimize the geometric feature extraction process, and update point cloud features through a bidirectional attention mechanism.
It significantly improves computational efficiency and segmentation accuracy, especially for segmenting complex geometric primitives, reduces model computational complexity and inference time, and enhances the ability to capture local details and global context.
Smart Images
Figure CN121661345A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer graphics and 3D computer vision, and in particular to a point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention. Background Technology
[0002] Point cloud semantic segmentation is a core task in 3D vision, with wide applications in industrial inspection, reverse engineering, and autonomous driving. In industrial scenarios, point cloud data often lacks features such as color and texture, and its segmentation accuracy highly depends on geometric shape information. Furthermore, industrial parts are typically composed of basic geometric primitives (such as planes, cylinders, and cones), and accurate segmentation of these primitives is crucial for subsequent modeling and analysis.
[0003] Currently, Transformer-based models have demonstrated powerful capabilities in point cloud processing. Among them, GeometricTransformer enhances the model's ability to perceive geometric structures by introducing geometric features. However, the native GeometricTransformer has significant drawbacks: firstly, its geometric word embedding has high computational complexity, involving numerous K-nearest neighbor (KNN) searches and matrix operations, resulting in a large model size and slow inference speed; secondly, its attention mechanism is insufficient in fusing geometric information, especially in its ability to capture subtle local geometric structures and overall global contextual relationships, limiting the segmentation accuracy on complex industrial point clouds. Summary of the Invention
[0004] The purpose of this invention is to provide a point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention. This point cloud primitive segmentation method maintains high segmentation accuracy while significantly improving computational efficiency.
[0005] The technical solution of this invention: a point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention, comprising the following steps: Step 1: Obtain the point cloud data of the object to be segmented and preprocess the point cloud data; Step 2: Input the preprocessed point cloud into the trained primitive segmentation model to obtain the features at each point in the point cloud space; wherein, the primitive segmentation model is a model built based on an improved Geometric Transformer network, and the model optimizes the geometric feature extraction process through a two-way fusion mechanism of global and local geometric attention. Step 3: Input the point cloud features into the post-processing module to obtain the primitive segmentation result.
[0006] The above-mentioned point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention includes the following steps in step 1: preprocessing the point cloud data, which involves translating the point cloud center to move the origin of the point cloud coordinates to the geometric center of the point cloud, randomly flipping the point cloud to enhance data diversity, aligning the main direction of the point cloud with the coordinate axes, and then normalizing the point cloud coordinates.
[0007] The aforementioned point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention, wherein the improved Geometric Transformer network includes an encoder and a decoder, wherein: The encoder extracts local and global geometric word embeddings through downsampling operations and updates point cloud features based on a bidirectional attention fusion mechanism. The decoder restores the point cloud resolution through upsampling operations and fuses local and global geometric word embeddings to enhance feature representation capabilities.
[0008] The aforementioned point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention includes the calculation of local geometric word embeddings as follows: Obtain the downsampled point cloud by sampling from the farthest point; The K-nearest neighbor algorithm is used to find the nearest neighbors of each sampling point; Calculate the distance vector and scroll vector between the nearest neighbor and the center point, and generate local geometric word embeddings based on the angle and distance between them.
[0009] The aforementioned point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention includes the calculation of the global geometric word embedding, which comprises: Calculate the difference vector and its length between any two points in the downsampled point cloud; Extract the nearest neighbors of each point to form a local structure, and calculate its angular relationship with the global point; Generate holistic geometric word embeddings based on angle and distance features.
[0010] The aforementioned point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention is characterized in that the attention mechanism in the encoder is implemented through the following steps: The local geometric word embeddings are added to the point cloud features to update the feature representation; Calculate the query, key, and value matrix, and fuse the overall geometric word embeddings to generate attention weights; The updated features are output through Softmax normalization and feature weighting.
[0011] The aforementioned point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention, wherein the attention mechanism in the decoder is implemented through the following steps: The upsampled local geometric word embeddings are fused with the features; Calculate the query, key, and value matrix, and expand the attention receptive field using holistic geometric word embeddings; Attention weights are optimized using linear layers and pooling operations, ultimately outputting refined point cloud features.
[0012] The aforementioned point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention includes the following training process for the primitive segmentation model: Preprocess the labeled industrial point cloud dataset; Calculate the local and global geometric word embeddings sequentially during the downsampling and upsampling stages; Features are extracted using an improved Geometric Transformer network, and the loss function is calculated using a classification module; Iteratively optimize the model parameters until the loss function converges.
[0013] The aforementioned point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention includes primitive categories such as plane, cylinder, cone, torus, sphere and B-spline.
[0014] Compared with the prior art, the present invention has the following significant advantages: 1. Significantly Improved Operational Efficiency: By decomposing the massive global geometric word embedding calculation into local and global parts, a large amount of redundant KNN calculations and matrix operations are avoided, significantly reducing the computational complexity and inference time of the model. As shown in Table 5, under the same parameter scale, the time consumption of this invention is significantly lower than that of the unoptimized model.
[0015] 2. Improved Segmentation Accuracy: The bidirectional fusion mechanism of local and global geometric information enables the model to capture both local details and global context, enhancing its ability to discriminate complex geometric primitives. The method of this invention achieves higher mIoU (mean intersection-over-union ratio) across multiple primitive categories, with a particularly significant improvement in segmentation performance for complex primitives such as spheres and B-splines.
[0016] 3. Superior Model Structure: The improved attention mechanism is well-designed, which introduces rich geometric information while maintaining the model's trainability and stability, making it more suitable for practical industrial applications. Attached Figure Description
[0017] Figure 1 This is a schematic diagram of the overall process of the present invention. Figure 2 A schematic diagram of the encoder attention mechanism in an improved Geometric Transformer network; Figure 3A schematic diagram of the decoder attention mechanism in an improved Geometric Transformer network; Figure 4 This is a schematic diagram of the training process of the primitive segmentation model of the present invention. Detailed Implementation The present invention will be further described below with reference to the accompanying drawings and examples, but this should not be construed as limiting the present invention.
[0018] Example: A point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention, such as... Figure 1 As shown, it includes the following steps: Step 1: Obtain point cloud data of the object to be segmented and preprocess the point cloud data. In this step, an industrial point cloud dataset containing primitives such as planes, cylinders, cones, tori, spheres, and B-splines is obtained, and the primitive category to which each point belongs is labeled. The preprocessing process of the point cloud in this step is as follows: Step 1.1: By performing a center translation on the point cloud, the origin of the point cloud coordinate system is moved to the geometric center of the point cloud. The formula for calculating the center translation is as follows: ; in, For the first point cloud The spatial coordinates of the points The number of points contained in the point cloud. After translating from the center, the first The spatial coordinates of the points.
[0019] Step 1.2: Mirror the X-axis or Y-axis coordinates of the point cloud by random flipping to improve the diversity of the training data. The calculation formula for random flipping is as follows: or ; in, After translating from the center, the first The spatial coordinates of the points, After random flipping, the first The spatial coordinates of the points.
[0020] Step 1.3: Align the principal direction with the coordinate axes. Align the principal direction of the point cloud with the positive X-axis. The calculation formula is as follows: ; ; ; ; Among them, matrix It is composed of the coordinate values of all points in the point cloud. yes The transpose of the matrix, eigen() denotes the calculation of the eigenvalues and eigenvectors of the matrix. , , for eigenvalues, and It is the smallest eigenvalue. These are the corresponding feature vectors. To indicate the alignment direction of the point cloud, "·" represents the inner product operation, and "´" represents the cross product operation. Represents the magnitude of a vector. This represents the coordinate matrix of the point cloud after alignment with the coordinate axes.
[0021] Step 1.4: Normalize the point cloud coordinates based on the maximum values of the coordinates.
[0022] Step 2: Input the preprocessed point cloud into the trained primitive segmentation model to obtain the features at each point in the point cloud space; wherein, the primitive segmentation model is a model built based on an improved Geometric Transformer network, and the model optimizes the geometric feature extraction process through a two-way fusion mechanism of global and local geometric attention. In this step, the improved Geometric Transformer network includes an encoder, a decoder, and a feature interaction layer connecting the two. The encoder extracts local and global geometric word embeddings through downsampling and updates point cloud features based on a bidirectional attention fusion mechanism. The decoder restores point cloud resolution through upsampling and fuses local and global geometric word embeddings to enhance feature representation. Furthermore, the initial local geometric feature extraction stage of the encoder is implemented using Mini-PointNet: this Mini-PointNet uses a multilayer perceptron (MLP) to encode the features of the local point set selected by the K-nearest neighbor algorithm, outputting a local point cloud feature vector with uniform dimensionality, providing basic feature support for subsequent calculations of local geometric word embeddings. Compared to the traditional PointNet, the Mini-PointNet used in this invention simplifies the number of fully connected layers, further reducing model computational complexity while maintaining local feature capture capabilities, achieving synergistic optimization of efficiency and accuracy with the global-local attention fusion mechanism.
[0023] Because the original method requires calculating a huge amount of data. The word embedding matrix, due to the numerous KNN operations involved in its calculation and the fact that its queries expand across all points after upsampling or downsampling, suffers from extremely high computational cost. Therefore, this invention decomposes it into local geometric word embeddings and global geometric word embeddings, thereby reducing computational time.
[0024] In this embodiment, the encoder calculates the local geometric word embedding in the downsampling as follows: Step 2.1.1: Set the number of groupings for point cloud coordinates. and the size of each group According to the number of groups The farthest point of the point cloud is sampled to obtain the downsampled point cloud. The process can be represented as follows: ; Step 2.1.2: Using KNN, calculate... In the original point cloud The front of the middle The nearest neighbor points are grouped and pieced together, and the resulting shape is as follows: That is, the batch size is B, and the number of groups after downsampling is Each group contains Nearest neighbor points.
[0025] Step 2.1.3: Calculate the distance vector from each nearest neighbor point to its center point. Its shape remains unchanged. ;exist The third dimension, namely the nearest neighbor dimension, performs a forward scrolling operation to form... The relationship between the two can be expressed as: ; Step 2.1.4, Calculation , and Together constitute Angle of the vertex : ; Its shape is .
[0026] Step 2.1.5: Calculate the distance from each nearest neighbor point to the center point: ; Step 2.1.6, Adjust the angle and distance Input into the Geometric Transformer's native geometric word embedding generator to generate local angular word embeddings. and local distance word embedding Both are of the following shapes .
[0027] Step 2.1.7: Calculate and obtain the local geometric features during the downsampling stage: ; The Linear layer is a linear layer whose output feature length remains consistent with the input. All shapes .
[0028] In this embodiment, the encoder calculates the overall geometric word embedding in the downsampling as follows: Step 2.2.1: Calculate the downsampled point cloud The difference vector of coordinates of any two points in the vector Its shape is
[0029] Step 2.2.2: Calculate the difference vector The lengths of each vector in Its shape is
[0030] Step 2.2.3: Calculate using KNN The first three nearest neighbors of each point are grouped and then stitched together. The resulting shape is as follows: The third dimension represents the number of nearest neighbors, and the last dimension represents the coordinates of the point after normalization.
[0031] Step 2.2.4: Calculate the distance vector from each nearest neighbor point to its center point, the shape of which is... Repeat the following three dimensions This time received Its shape is ,in .
[0032] Step 2.2.5: Difference vector Repeat the last dimension 3 times to obtain... Its shape is .
[0033] Step 2.2.6: Take the sampled point cloud Hit the mark , and its three nearest neighbors These four points can be compared with the downsampled point cloud. Hit the mark , forming Let the three angles of the vertex be: ; Its shape is .
[0034] Step 2.2.7, Adjust the angle Inputting it into the Geometric Transformer's native geometric word embedding generator yields angular word embeddings, the shape of which is... And take the maximum value in its fourth dimension to obtain the overall angle word embedding. Its shape is .
[0035] Step 2.2.8, will The word embeddings are fed into the native geometric word embedding generator of the Geometric Transformer to generate global distance word embeddings. Its shape is .
[0036] Step 2.2.9: Calculate and obtain the overall geometric word embedding during the downsampling stage: ; The Linear layer is a linear layer whose output feature length remains consistent with the input. All shapes .
[0037] In this embodiment, the decoder calculates the local geometric word embedding corresponding to the upsampling process as follows: Step 2.3.1: Calculate the sampling point cloud based on KNN. Before any point in the middle Its nearest neighbor points have the following shape: .
[0038] Step 2.3.2: Calculate the distance vector from each nearest neighbor point to its center point. Its shape remains unchanged. ,exist The third dimension, namely the nearest neighbor dimension, is formed by performing a forward scrolling operation. The relationship between the two can be expressed as: .
[0039] Step 2.3.3, Calculation , and Together constitute Angle of the vertex :
[0040] Step 2.3.4: Calculate the distance from each nearest neighbor point to the center point: .
[0041] Step 2.3.5: Angle and distance Input into the Geometric Transformer's native geometric word embedding generator to generate local angular word embeddings. and local distance word embedding Both are of the following shapes .
[0042] Step 3.6: Calculate and obtain the local geometric features during the downsampling stage:
[0043] Where Linear is a linear layer, the output feature length remains consistent with the input, and the shape of RL is... .
[0044] In this embodiment, the decoder calculates the overall geometric word embedding corresponding to the upsampling process as follows: Step 2.4.1: Calculate the downsampled point cloud using KNN. The first three nearest neighbors of each point are grouped and then stitched together. The resulting shape is as follows: .
[0045] Step 2.4.2: Calculate the distance vector from each nearest neighbor point to its center point, and copy it. Secondly, its shape is , recorded as ,in .
[0046] Step 2.4.3: Calculate the original point cloud Each point in the middle The distance vector of each point in the vector is copied 3 times, and its shape is as follows: , recorded as ,in .
[0047] Step 2.4.4 Each point in Each point in, and Each point in the array has three nearest neighbors, which can form three groups of three points. The angle of the vertex is calculated as follows: ; Its shape is .
[0048] Step 2.4.5, Adjust the angle Inputting it into the Geometric Transformer's native geometric word embedding generator yields angular word embeddings, the shape of which is... And take the maximum value in its fourth dimension to obtain the overall angle word embedding. Its shape is .
[0049] Step 2.4.6, Calculation Each point in the middle The length of the distance vector to each point in the vector is denoted as . Its shape is .
[0050] Step 2.4.7, will The word embeddings are fed into the native geometrical word embedding generator of the Geometrical Transformer to generate global distance word embeddings. , shape is .
[0051] Step 2.4.8: Calculate the overall geometric word embeddings for the downsampling stage: .
[0052] In this embodiment, as Figure 2 As shown, the attention mechanism in the encoder is implemented through the following steps: The local geometric word embeddings are added to the point cloud features to update the feature representation; Calculate the query, key, and value matrix, and fuse the overall geometric word embeddings to generate attention weights; The updated features are output through Softmax normalization and feature weighting.
[0053] Specifically, the attention calculation process corresponding to encoder downsampling is described as follows: Step 2.5.1: Perform max pooling on the local geometric word embeddings. After pooling... Shape becomes .
[0054] Step 2.5.2, will Updated to: and sum: .
[0055] Step 2.5.3, will Feed it into a linear layer to obtain the attention mechanism. , and Values, and adjust their shapes to ,in For the number of attention heads.
[0056] Step 2.5.4, Calculation and Matrix multiplication Its shape is . Step 2.5.5: Embed the overall geometric word Feeding into the linear layer: ; After the update, its shape was adjusted to . Step 2.5.6, will Shape adjustment and calculate and Einstein product: .
[0057] After calculation, adjust its shape to .
[0058] Step 2.5.7, Calculation and The sum of: .
[0059] Step 2.5.8, for Perform Softmax: . Step 2.5.9, will Updated to and The matrix product in the last two dimensions: ; After calculation, their shapes were all adjusted to .
[0060] Furthermore, KNN is used to find point clouds. At each point in In 1 nearest neighbor, thus obtaining Its shape is And by using pooling operations, the features are reduced and further condensed into This completes the upsampling of features.
[0061] like Figure 3 As shown, the attention mechanism in the decoder is implemented through the following steps: The upsampled local geometric word embeddings are fused with the features; Calculate the query, key, and value matrix, and expand the attention receptive field using holistic geometric word embeddings; Attention weights are optimized using linear layers and pooling operations, ultimately outputting refined point cloud features.
[0062] Specifically, the attention calculation process corresponding to decoder upsampling is described as follows: Step 2.6.1: Perform max pooling on the local geometric word embeddings. After pooling... Shape becomes .
[0063] Step 2.6.2, will Updated to and sum: .
[0064] Step 2.6.3, will Feed it into a linear layer to obtain the attention mechanism. , and Value, and adjust their shapes to ,in For the number of attention heads.
[0065] Step 2.6.4, Calculation and Matrix multiplication Its shape is .
[0066] Step 2.6.5: Embed the overall geometric words Feeding into the linear layer: ; After the update, its shape was adjusted to .
[0067] Step 2.6.6, will Shape adjustment and calculate and Einstein product: .
[0068] After calculation, adjust its shape to .
[0069] Step 2.6.7: Update the... The data is fed into a linear layer to expand and amplify its features. ; The linear layer shape is Reshape it to .
[0070] Step 2.6.8, Pooling is performed, and its shape becomes... .
[0071] Step 2.6.9, Calculation and The sum of: ; Its shape is .
[0072] Step 2.6.10, regarding Perform Softmax: ; Its shape is .
[0073] Step 2.6.11, will Updated to Product with Value: ; After calculation, their shapes were all adjusted to ; Then, by using skip connections, the final features of each point are expressed as follows: Its shape is .
[0074] Step 3: Input the point cloud features into the post-processing module (i.e., the prediction layer) to obtain the primitive segmentation result. In this step, the prediction layer can perform a linear transformation on the feature vector of each point in the point cloud, and then use a Softmax function to convert it into a probability distribution for each class. Finally, the class with the highest probability is taken as the segmentation result for that point.
[0075] In this embodiment, as Figure 4 As shown, the training process of the primitive segmentation model includes: S1. Obtain the industrial point cloud dataset and label the categories of each primitive in the point cloud dataset.
[0076] Preprocess the labeled data; S3. Improved geometric word embedding in computational point cloud downsampling; S4. Calculate the improved geometric word embedding in point cloud upsampling; S5. Embed each geometric word into the improved Geometric Transformer network to obtain the features at each point in the point cloud space; S6. Input the features of each point in the point cloud into the classification module and calculate the model loss function; S7. Adjust the model parameters to optimize the model. The training of the model is completed when the loss function converges.
[0077] The method of this invention was used to perform semantic segmentation tests on industrial point cloud primitives. The test results are shown in Table 1 below. Here, "before optimization" refers to the original Geometric Transformer network method, and "after optimization" refers to the improved Geometric Transformer network method of this invention.
[0078] Table 1
[0079] The test results show that the method of the present invention achieves improved segmentation accuracy and reduced running time under different grouping parameter settings. The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A point cloud primitive semantic segmentation method based on the fusion of global and local geometric attention, characterized in that: Includes the following steps: Step 1: Obtain the point cloud data of the object to be segmented and preprocess the point cloud data; Step 2: Input the preprocessed point cloud into the trained primitive segmentation model to obtain the features at each point in the point cloud space; wherein, the primitive segmentation model is a model built based on an improved Geometric Transformer network, and the model optimizes the geometric feature extraction process through a two-way fusion mechanism of global and local geometric attention. Step 3: Input the point cloud features into the post-processing module to obtain the primitive segmentation result.
2. The point cloud primitive semantic segmentation method based on global and local geometric attention fusion according to claim 1, characterized in that: In step 1, the preprocessing of the point cloud data includes translating the point cloud center to move the origin of the point cloud coordinates to the geometric center of the point cloud, randomly flipping the point cloud to enhance data diversity, aligning the main direction of the point cloud with the coordinate axes, and then normalizing the point cloud coordinates.
3. The point cloud primitive semantic segmentation method based on global and local geometric attention fusion according to claim 1, characterized in that: The improved Geometric Transformer network includes an encoder and a decoder, wherein: The encoder extracts local and global geometric word embeddings through downsampling operations and updates point cloud features based on a bidirectional attention fusion mechanism. The decoder restores the point cloud resolution through upsampling operations and fuses local and global geometric word embeddings to enhance feature representation capabilities.
4. The point cloud primitive semantic segmentation method based on global and local geometric attention fusion according to claim 3, characterized in that, The calculation of the local geometric word embedding includes: Obtain the downsampled point cloud by sampling from the farthest point; Use the K-nearest neighbor algorithm to find the nearest neighbors of each sampling point; Calculate the distance vector and scroll vector between the nearest neighbor and the center point, and generate local geometric word embeddings based on the angle and distance between them.
5. The point cloud primitive semantic segmentation method based on global and local geometric attention fusion according to claim 3, characterized in that, The calculation of the overall geometric word embedding includes: Calculate the difference vector and its length between any two points in the downsampled point cloud; Extract the nearest neighbors of each point to form a local structure, and calculate its angular relationship with the global point; Generate holistic geometric word embeddings based on angle and distance features.
6. The point cloud primitive semantic segmentation method based on global and local geometric attention fusion according to claim 3, characterized in that, The attention mechanism in the encoder is implemented through the following steps: The local geometric word embeddings are added to the point cloud features to update the feature representation; Calculate the query, key, and value matrix, and fuse the overall geometric word embeddings to generate attention weights; The updated features are output through Softmax normalization and feature weighting.
7. The point cloud primitive semantic segmentation method based on global and local geometric attention fusion according to claim 3, characterized in that, The attention mechanism in the decoder is implemented through the following steps: The upsampled local geometric word embeddings are fused with features; Calculate the query, key, and value matrix, and expand the attention receptive field using holistic geometric word embeddings; Attention weights are optimized using linear layers and pooling operations, ultimately outputting refined point cloud features.
8. The point cloud primitive semantic segmentation method based on global and local geometric attention fusion according to claim 1, characterized in that, The training process of the primitive segmentation model includes: Preprocess the labeled industrial point cloud dataset; Calculate the local and global geometric word embeddings sequentially during the downsampling and upsampling stages; Features are extracted using an improved Geometric Transformer network, and the loss function is calculated using a classification module; Iteratively optimize the model parameters until the loss function converges.
9. The point cloud primitive semantic segmentation method based on global and local geometric attention fusion according to claim 1, characterized in that, The primitive categories include plane, cylinder, cone, torus, sphere, and B-spline.