Point cloud semantic segmentation methods based on global and progressive attention

By employing a point cloud semantic segmentation method with global and progressive attention, this approach directly processes point cloud data and leverages an encoder-decoder network to enhance global context and geometric features. This addresses the issues of low accuracy and compromised structural integrity in existing technologies, achieving highly efficient point cloud segmentation.

CN122336299BActive Publication Date: 2026-11-17HEILONGJIANG INST OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202610665385.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-05-14
Publication Date
2026-11-17
Estimated Expiration
2046-05-14

AI Technical Summary

Technical Problem

Existing methods require intermediate data conversion steps such as voxelization and multi-view projection, resulting in low point cloud semantic segmentation accuracy and damage to the integrity of the global spatial structure.

Method used

We adopt a point cloud semantic segmentation method based on global and progressive attention. We directly process point cloud data through an encoder-decoder network and use a global feature attention module, an encoder neighborhood construction module, and a progressive attention convolution module to capture global contextual information and enhance geometric features, thus avoiding intermediate transformation steps.

Benefits of technology

It improves the accuracy of point cloud semantic segmentation, enables efficient segmentation of large-scale complex scenes, and maintains the integrity of the global spatial structure.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122336299B_ABST
    Figure CN122336299B_ABST
Patent Text Reader

Abstract

This paper proposes a point cloud semantic segmentation method based on global and progressive attention, which relates to the field of computer vision technology. It addresses the shortcomings of existing point cloud transformation and direct processing methods, which are insufficiently adaptable to disordered and unevenly distributed point cloud structures, making it difficult to fully extract the potential geometric structure information of point clouds and resulting in low segmentation accuracy for some features. The technical solution proposed in this application is an end-to-end encoder-decoder network that enables direct processing of raw point cloud data without intermediate data transformation steps such as voxelization and multi-view projection. This avoids the loss of spatial information in the raw point cloud data and improves the accuracy of point cloud semantic segmentation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computer vision technology, specifically to a point cloud semantic segmentation method based on global and progressive attention. Background Technology

[0002] Semantic segmentation is a core task in computer vision, aiming to assign precise semantic category labels to each point in a point cloud. As a crucial preprocessing step in cutting-edge applications such as autonomous driving, robot navigation, urban planning, virtual reality, and smart city digital twins, its segmentation accuracy directly determines the reliability of subsequent decision-making and control. The widespread adoption of active sensing technologies such as LiDAR has made the acquisition of large-scale, high-density 3D point clouds increasingly convenient, thus placing higher demands on point cloud semantic segmentation methods in terms of processing efficiency, accuracy, and adaptability to complex scenes.

[0003] Existing methods require intermediate data transformation steps of voxelization and multi-view projection. These two intermediate transformation steps continuously weaken the spatial correlation of the neighborhood, resulting in damage to the integrity of the global spatial structure, i.e., loss of spatial information, leading to low semantic segmentation accuracy of the final point cloud. Summary of the Invention

[0004] The purpose of this invention is to address the problem that existing methods require intermediate data transformation steps of voxelization and multi-view projection. These two intermediate transformation steps continuously weaken the neighborhood spatial correlation, resulting in damage to the integrity of the global spatial structure, i.e., loss of spatial information, and thus low accuracy of the final point cloud semantic segmentation. This invention provides a point cloud semantic segmentation method based on global and progressive attention.

[0005] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0006] A point cloud semantic segmentation method based on global and progressive attention includes the following steps:

[0007] Step 1: Obtain the point cloud of the target city area;

[0008] Step 2: Input the point cloud into the encoder network to obtain the feature vector output by the encoder network. 1;

[0009] Step 3: Convert the feature vector output by the encoder network 1. Input the decoder network to obtain the feature vector output by the decoder network. 2;

[0010] Step 4: Convert the feature vector 2. Input the fully connected layer to obtain the final semantic labels of the original point cloud.

[0011] Furthermore, the feature attributes of the point cloud include three-dimensional coordinates, and optionally one or more of reflection intensity and color information.

[0012] Furthermore, the encoder network includes a global feature attention module, an encoder neighborhood construction module, and a progressive attention convolution module;

[0013] The encoder network specifically performs the following steps:

[0014] First, a global feature attention module is used to globally enhance the point cloud. Then, an encoder neighborhood construction module is used to construct the neighborhood of the globally enhanced point cloud. Finally, the point cloud and its corresponding neighborhood points are input into a progressive attention convolution module to obtain the output features of the current layer. After repeating the above operation 5 times, the feature vector output by the encoder network is obtained. 1.

[0015] Furthermore, the global feature attention module specifically performs the following steps:

[0016] For each point in the point cloud, calculate the Euclidean distance between that point and all other points in the point cloud to obtain a distance vector. Use this distance vector as a global distance feature channel. Then, concatenate all global distance feature channels with the original feature channels of the point cloud to obtain a global feature vector. .

[0017] Furthermore, the encoder neighborhood construction module specifically performs the following steps:

[0018] Step 221: Sample the globally enhanced point cloud to obtain M sampling points;

[0019] Step 222: Take each sampling point as a reference point, and construct a spherical neighborhood within a set radius with each reference point as the center of the sphere;

[0020] Step 223: Perform a random sampling operation within the spherical neighborhood to obtain K neighborhood points corresponding to each reference point, thereby constructing the neighborhood point set of the reference point.

[0021] Furthermore, in step 221, the farthest point sampling algorithm is used to sample the globally enhanced point cloud.

[0022] Furthermore, the progressive attention convolution module specifically performs the following steps:

[0023] Step 231: Based on the reference point and its neighboring points, input the set of neighboring points into a multilayer perceptron (MLP) to obtain the feature vector. ;

[0024] Step 232: Directly extract coordinate feature vectors from the neighborhood point set, namely the x-coordinate feature vector, y-coordinate feature vector, and z-coordinate feature vector;

[0025] Step 233: Convert the feature vector Local augmentation attention based on x-coordinate feature vector input Module, Local Enhanced Attention The module will use feature vectors After concatenating the x-coordinate feature vector, a multilayer perceptron (MLP) is used to process the resulting feature vector to obtain the feature vector. ;

[0026] Step 234: Convert the feature vector Local augmentation attention based on y-coordinate feature vector input Modules, Local Enhanced Attention The module will use feature vectors After concatenating the y-coordinate feature vector, a multilayer perceptron (MLP) is used to process the resulting feature vector to obtain the feature vector. ;

[0027] Step 235: Convert the feature vector Local augmentation attention based on z-coordinate feature vector input Module, Local Enhanced Attention The module will use feature vectors After concatenating the z-coordinate feature vector, a multilayer perceptron (MLP) is used to process the resulting feature vector to obtain the feature vector. ;

[0028] Step 236: Convert the feature vector After being input into a multilayer perceptron (MLP), the progressive attention weights are obtained through a softmax function. ;

[0029] Step 237: Process the point cloud after neighborhood construction using a multilayer perceptron (MLP) to obtain feature vectors. ;

[0030] Step 238: Convert the feature vector and progressive attention weights Element-wise multiplication yields the feature vector. ;

[0031] Step 239: Convert the feature vector Input the feature vector into a multilayer perceptron (MLP) and perform max pooling. .

[0032] Furthermore, the decoder network includes a decoder neighborhood construction module, a progressive attention convolution module, and a multilayer perceptron (MLP).

[0033] The decoder network specifically performs the following steps:

[0034] During each layer of decoding, the features output from the previous layer are first input into the decoder neighborhood construction module. The points in the current layer's midpoint set are used as reference point positions, and the reference point position is used as the center of a sphere. Within a set radius, neighborhood points corresponding to the reference point position are searched from the previous layer's midpoint set to construct the neighborhood point set of the reference point.

[0035] Subsequently, the neighboring point set is processed using a progressive attention convolution module to obtain the processed features; the processed features are then averaged using an average pooling function to obtain the recovered features of the current layer; then, the recovered features of the current layer are connected to the features of the same layer of the encoder through skip connections to obtain the connected features; finally, the connected features are input into a multilayer perceptron (MLP) for feature refinement to obtain the decoded output features of the current layer.

[0036] After repeating the above operation 5 times, the output feature vector of the decoder network is obtained. 2.

[0037] Furthermore, within the set radius, the neighboring points corresponding to the reference point are searched from the previous layer point set using a ball query algorithm.

[0038] Furthermore, the point cloud includes point clouds of buildings, roads, power lines, low vegetation, and trees.

[0039] The beneficial effects of this invention are:

[0040] The technical solution of this application is an end-to-end encoder-decoder network that enables direct processing of raw point cloud data without the need for voxelization and multi-view projection transformation steps. This avoids the problem of weakening the neighborhood spatial correlation caused by two intermediate transformation steps and improves the accuracy of point cloud semantic segmentation.

[0041] This application can effectively capture geometric details, further enhance geometric features, and simulate global context dependencies to capture global context information. It can handle semantic segmentation tasks of large-scale, irregular, and complex terrain features, providing a more practical technical solution for the field of 3D point cloud analysis. Attached Figure Description

[0042] Figure 1 The flowchart shows a point cloud semantic segmentation method based on global and progressive attention.

[0043] Figure 2 This is a diagram illustrating the overall structure of a point cloud semantic segmentation method based on global and progressive attention.

[0044] Figure 3 Here is a structural diagram of the global feature attention module;

[0045] Figure 4 This is a structural diagram of the progressive attention module;

[0046] Figure 5 This is a structural diagram of the progressive attention convolution module. Detailed Implementation

[0047] It should be noted that, where there is no conflict, the various embodiments disclosed in this application can be combined with each other.

[0048] Specific Implementation Method 1: The point cloud semantic segmentation method based on global and progressive attention described in this implementation method includes:

[0049] The process includes the data acquisition stage, encoder network stage, decoder network stage, and semantic tag generation stage, as follows: Figure 1 As shown. The specific technical solution is as follows:

[0050] Step 1: Obtain point cloud data with a consistent data format. Represent the acquired point cloud as follows: ,in This represents the number of channels for each point, including 3D coordinates and characteristic attributes such as reflection intensity and color information. When there are no additional characteristic attributes... .

[0051] Point cloud data of the target urban area is acquired using airborne LiDAR. This point cloud data includes buildings, roads, power lines, low vegetation, trees, etc. After acquiring the raw point cloud data, it is preprocessed to obtain point cloud data that can be input into a semantic segmentation model. Subsequently, using a point cloud semantic segmentation model based on global and progressive attention, point-by-point semantic segmentation is performed on the point cloud data to obtain semantic objects such as buildings, roads, power lines, low vegetation, and trees that can be used by the smart city digital twin platform.

[0052] Step 2: Input the point cloud data into the encoder network, and repeat the process 5 times in the order of global feature attention module, encoder neighborhood construction module, and progressive attention convolution module to extract multi-scale features from the point cloud data, such as... Figure 2 The encoder network section is shown.

[0053] The encoder network consists of a global feature attention module, an encoder neighborhood construction module, and a progressive attention convolution module. The encoder network first uses the global feature attention module to perform global augmentation on the point cloud data, then uses the encoder neighborhood construction module to construct neighborhoods for the globally augmented point cloud, and finally employs a farthest point sampling algorithm to further enhance the globally augmented point cloud. Sampling is performed, and M sampling points are used as reference points. , with reference point Using a sphere as the center, construct a spherical neighborhood within a specific radius. Perform random sampling within this spherical neighborhood to obtain K neighboring points corresponding to each reference point, thus forming the reference point. neighborhood point set Then, the reference point and its neighboring points are input into the progressive attention convolution module. The global feature attention module simulates global context dependence, capturing global context information; the progressive attention module captures geometric details, further enhancing geometric features.

[0054] Global feature attention module, such as Figure 3 As shown, the progressive attention module is as follows: Figure 4 As shown, the progressive attention convolution module is as follows: Figure 5 As shown.

[0055] 1) Global Feature Attention Module

[0056] For any point in the point cloud Calculate the Euclidean distance between the point and all points in the point cloud (including itself) to obtain the distance vector. ,in for and European distance, This distance vector is used as the global distance feature channel and concatenated with the original feature channels of the point cloud to obtain the global feature vector. .

[0057] 2) Progressive Attention Module

[0058] ① Localized enhancement of attention

[0059] By learning the coordinates and features of a reference point and its neighboring points, local augmentation attention can capture geometric details and further enhance geometric features. The module will use feature vectors After concatenating the x-coordinate feature vector, a multilayer perceptron (MLP) is used to process the resulting feature vector to obtain the feature vector. , Localized enhancement of attention The module will use feature vectors After concatenating the y-coordinate feature vector, a multilayer perceptron (MLP) is used to process the resulting feature vector to obtain the feature vector. , Localized enhancement of attention The module will use feature vectors After concatenating the z-coordinate feature vector, a multilayer perceptron (MLP) is used to process the resulting feature vector to obtain the feature vector. , .

[0060] ② The structure of the progressive attention module

[0061] Based on the reference point and its neighboring points, the set of neighboring points is input into a multilayer perceptron (MLP) to obtain feature vectors. ; Extract coordinate feature vectors directly from the neighborhood point set, and then... The x-coordinate feature vector is input into the local augmentation attention. The module obtains the feature vector. ; to feature vector The y-coordinate feature vector is input into the local augmentation attention. The module obtains the feature vector. ; to feature vector The z-coordinate feature vector is input into the local augmentation attention. The module obtains the feature vector. . The feature vector The input is fed into a multilayer perceptron (MLP) and then into a softmax function to obtain progressive attention weights. .

[0062] 3) Progressive attention convolution module

[0063] The point cloud after neighborhood construction is processed by a multilayer perceptron (MLP) to obtain feature vectors. , to feature vector Attention weights output by the progressive attention module Element-wise multiplication is performed to obtain the feature vector. . The feature vector Input the multilayer perceptron (MLP) (this MLP has C2 input channels and C3 output channels) and perform max pooling to obtain the output feature vector of the progressive attention convolutional module. .

[0064] After repeating the above operation 5 times, the feature vector output by the encoder network is obtained. 1.

[0065] The feature channels C1, C2, and C3 of the progressive attention convolution module were set to (32, 32, 64), (64, 64, 128), (128, 128, 256), (256, 256, 512), and (512, 512, 1024) respectively in the 5 executions.

[0066] Step 3: Input the output of the encoder network from Step 2 into the decoder network, such as... Figure 2 The decoder network is shown in the diagram. Following the order of the decoder neighborhood construction module, the progressive attention convolution module, and the multilayer perceptron (MLP), and combined with skip connections, this process is repeated five times to restore and refine the features from coarse-scale to the original scale, ultimately obtaining a feature vector with the same number of points as the original point cloud. During each decoding layer, the features output from the previous layer are first input into the decoder's neighborhood construction module. Using a point in the current layer's midpoint set as a reference point and the reference point as the center of a sphere, a sphere query algorithm is used to search for neighboring points corresponding to the reference point within a specific radius, constructing a neighborhood point set for the reference point. Subsequently, the neighborhood point set is processed using a progressive attention convolution module to obtain the processed features. Average pooling is then applied to the processed features to obtain the recovered features for the current layer. Skip connections between corresponding layers connect the recovered features of the current layer to features at the same level as the encoder, resulting in connected features. Finally, the connected features are input into a multilayer perceptron (MLP) (this MLP has 2*C3 input channels and C2 output channels) for feature refinement, yielding the decoded output features for the current layer. This process is repeated five times to obtain the output feature vector of the decoder network. 2.

[0067] The feature channels C3 and C2 of the multilayer perceptron (MLP) of the decoder network were set to (1024, 512), (512, 256), (256, 128), (128, 64), and (64, 64) respectively in 5 executions.

[0068] Step 4: Input the feature vector output by the decoder network in Step 3 into the fully connected layer (this fully connected layer has 64 input channels and the number of output channels is the number of categories in the point cloud) to obtain the final semantic labels of each point in the original point cloud.

[0069] It should be noted that the specific embodiments are merely explanations and illustrations of the technical solution of the present invention and should not be used to limit the scope of protection. Any modifications made in accordance with the claims and specification of the present invention that are only partial should still fall within the protection scope of the present invention.

Claims

1. A point cloud semantic segmentation method based on global and progressive attention, characterized in that... Includes the following steps: Step 1: Obtain the point cloud of the target city area; Step 2: Input the point cloud into the encoder network to obtain the feature vector output by the encoder network. 1; Step 3: Convert the feature vector output by the encoder network 1. Input the decoder network to obtain the feature vector output by the decoder network. 2; Step 4: Convert the feature vector 2. Input the fully connected layer to obtain the final semantic labels of the original point cloud; The encoder network includes a global feature attention module, an encoder neighborhood construction module, and a progressive attention convolution module; The encoder network specifically performs the following steps: First, a global feature attention module is used to globally enhance the point cloud. Then, an encoder neighborhood construction module is used to construct the neighborhood of the globally enhanced point cloud. Finally, the point cloud and its corresponding neighborhood points are input into a progressive attention convolution module to obtain the output features of the current layer. After repeating the above operation 5 times, the feature vector output by the encoder network is obtained. 1; The decoder network includes a decoder neighborhood construction module, a progressive attention convolution module, and a multilayer perceptron (MLP). The decoder network specifically performs the following steps: During each layer of decoding, the features output from the previous layer are first input into the decoder neighborhood construction module. The points in the current layer's midpoint set are used as reference point positions, and the reference point position is used as the center of a sphere. Within a set radius, neighborhood points corresponding to the reference point position are searched from the previous layer's midpoint set to construct the neighborhood point set of the reference point. Subsequently, the neighboring point set is processed using the progressive attention convolution module to obtain the processed features; the processed features are then averaged using the average pooling function to obtain the recovered features of the current layer. Then, by using skip connections, the features recovered from the current layer are connected with the features at the same level of the encoder to obtain the connected features; finally, the connected features are input into a multilayer perceptron (MLP) for feature refinement to obtain the decoded output features of the current layer. After repeating the above operation 5 times, the output feature vector of the decoder network is obtained.

2.

2. The point cloud semantic segmentation method based on global and progressive attention as described in claim 1, characterized in that... The point cloud's characteristic attributes include three-dimensional coordinates, and optionally one or more of reflection intensity and color information.

3. The point cloud semantic segmentation method based on global and progressive attention as described in claim 2, characterized in that... The global feature attention module specifically performs the following steps: For each point in the point cloud, calculate the Euclidean distance between that point and all other points in the point cloud to obtain a distance vector. Use this distance vector as a global distance feature channel. Then, concatenate all global distance feature channels with the original feature channels of the point cloud to obtain a global feature vector. .

4. The point cloud semantic segmentation method based on global and progressive attention as described in claim 3, characterized in that... The encoder neighborhood construction module specifically performs the following steps: Step 221: Sample the globally enhanced point cloud to obtain M sampling points; Step 222: Take each sampling point as a reference point, and construct a spherical neighborhood within a set radius with each reference point as the center of the sphere; Step 223: Perform a random sampling operation within the spherical neighborhood to obtain K neighborhood points corresponding to each reference point, thereby constructing the neighborhood point set of the reference point.

5. The point cloud semantic segmentation method based on global and progressive attention as described in claim 4, characterized in that... In step 221, the farthest point sampling algorithm is used to sample the globally enhanced point cloud.

6. The point cloud semantic segmentation method based on global and progressive attention as described in claim 5, characterized in that... The progressive attention convolution module specifically performs the following steps: Step 231: Based on the reference point and its neighboring points, input the set of neighboring points into a multilayer perceptron (MLP) to obtain the feature vector. ; Step 232: Directly extract coordinate feature vectors from the neighborhood point set, namely the x-coordinate feature vector, y-coordinate feature vector, and z-coordinate feature vector; Step 233: Convert the feature vector Local augmentation attention based on x-coordinate feature vector input Module, Local Enhanced Attention The module will use feature vectors After concatenating the x-coordinate feature vector, a multilayer perceptron (MLP) is used to process the resulting feature vector to obtain the feature vector. ; Step 234: Convert the feature vector Local augmentation attention based on y-coordinate feature vector input Module, Local Enhanced Attention The module will use feature vectors After concatenating the y-coordinate feature vector, a multilayer perceptron (MLP) is used to process the resulting feature vector to obtain the feature vector. ; Step 235: Convert the feature vector Local augmentation attention based on z-coordinate feature vector input Module, Local Enhanced Attention The module will use feature vectors After concatenating the z-coordinate feature vector, a multilayer perceptron (MLP) is used to process the resulting feature vector to obtain the feature vector. ; Step 236: Convert the feature vector After being input into a multilayer perceptron (MLP), the progressive attention weights are obtained through a softmax function. ; Step 237: Process the point cloud after neighborhood construction using a multilayer perceptron (MLP) to obtain feature vectors. ; Step 238: Convert the feature vector and progressive attention weights Element-wise multiplication yields the feature vector. ; Step 239: Convert the feature vector Input the data into a multilayer perceptron (MLP) and perform max pooling to obtain the feature vector. .

7. The point cloud semantic segmentation method based on global and progressive attention as described in claim 6, characterized in that... Within a set radius, a search is performed on neighboring points corresponding to the reference point from the upper-level point set using a ball query algorithm.

8. The point cloud semantic segmentation method based on global and progressive attention as described in claim 7, characterized in that... The point cloud includes point clouds of buildings, roads, power lines, low vegetation, and trees.

Citation Information

Patent Citations

  • Point cloud semantic segmentation method based on multi-view structural feature attention convolution

    CN115965788A

  • Lidar point cloud segmentation method, device, apparatus, and storage medium

    US20240212374A1