Three-dimensional point cloud semantic segmentation system for enhancing vehicle boundary perception and global segmentation capability

Abstract

The invention provides a three-dimensional point cloud semantic segmentation system for enhancing vehicle boundary perception and global segmentation capability, which is used for solving the problems of fuzzy boundary division, confusion of global semantics and the like when adjacent objects or similar objects are segmented. The method is based on a novel hierarchical Superpoint Transform architecture, and mainly comprises five parts: (1) a multi-layer perceptron (MLP) for processing hierarchical initial point clouds and a relative position code; (2) a local feature enhancement module; (3) a double attention feature extraction module; (4) a large kernel convolution hierarchical fusion network used for multi-scale feature fusion; and (5) a linear layer classification head for 3D point cloud segmentation. Wherein the double attention module comprises space coordinate attention, sparse self-attention and multiple partial convolution. According to the method, S3DIS, KITTI-360 and DALES data sets are used for verification, and compared with the most advanced algorithm, it is shown that the average segmentation precision is remarkably improved.

Description

Technical Field

[0001] The present invention relates to technical fields such as computer vision processing technology, deep learning, and three-dimensional point cloud. Background Art

[0002] Recently, the Policy and Economic Research Institute and the Artificial Intelligence and Economic and Social Research Center of the China Academy of Information and Communications Technology (CAICT) jointly released a special report titled "Autonomous Driving Strategy and Policy Observation (2024) - Policies and Regulations Helping Highly Automated Driving Accelerate Industrialization". The report deeply analyzes the current development status of the autonomous driving industry, summarizes the important role of policies and regulations in promoting the industrialization of autonomous driving, and looks forward to future development trends.

[0003] As a product of the deep integration of artificial intelligence, information communication and other technologies with automobiles, autonomous driving technology is accelerating the profound changes in the automobile, transportation and other industries. The report pointed out that in recent years, with the rapid evolution of artificial intelligence, large models, information communication and other technologies, the application scenarios of autonomous driving have been continuously enriched, from Robotaxi (autonomous driving taxis), Robobus (autonomous driving buses) to unmanned delivery vehicles, urban NOA (urban advanced assisted driving navigation), autonomous parking, etc., which are gradually integrated into the daily life of the people and bring unprecedented service experience. In the process of realizing autonomous driving, environmental perception is one of the key links. The vehicle needs to accurately perceive the surrounding environment, accurately segment and identify the categories of surrounding objects, and make safe driving decisions. The perception process of autonomous driving vehicles usually requires the participation of three-dimensional point clouds, and the corresponding semantic labels are given to objects within the radiation range of the car through point cloud segmentation. The accuracy of point cloud segmentation will directly affect the subsequent decision-making of autonomous driving vehicles, the normal progress of path planning and other tasks, and the safety of drivers.

[0004] At present, image processing technology and 3D point cloud technology have become the core support of existing point cloud segmentation algorithms. Through computer vision algorithms, vehicles can extract key information such as lane lines, traffic signs, pedestrians and other vehicles from images. Deep learning models, especially convolutional neural networks (CNNs), perform well in tasks such as image classification, object detection and semantic segmentation. However, two-dimensional images have limitations in depth perception and spatial understanding, and it is difficult to accurately estimate the distance and three-dimensional shape of objects. To make up for the shortcomings of two-dimensional images, vehicles are usually equipped with sensors such as laser radar (LiDAR) to directly obtain three-dimensional point cloud data of the environment. Point clouds are composed of a large number of three-dimensional coordinate points that accurately describe the spatial position and shape of surrounding objects. By processing point cloud data, vehicles can achieve high-precision three-dimensional target detection, object classification and scene understanding.

[0005] The accuracy of autonomous driving scene segmentation depends largely on the performance of the point cloud segmentation algorithm, and the most important thing is whether the point cloud segmentation algorithm is suitable for large-scale point cloud segmentation in autonomous driving scenarios. With the continuous iteration of point cloud algorithms and the technical support of sensors such as lidar, vision-based Transformers have been widely used in the field of point cloud segmentation. When Transformers segment large-scale point cloud data, their unique parallel mechanism can greatly improve the segmentation efficiency, and they are highly flexible and scalable, and can effectively segment irregular point cloud data. This just meets the complex needs of large-scale autonomous driving scenarios.

[0006] Initially, people focused on various variants of Transformer self-attention, which led to multi-head self-attention, sparse self-attention, spatial channel self-attention, adaptive self-attention, etc. However, due to the limitations of self-attention itself, it focuses on global context information and is difficult to learn important local details of objects. Later, people introduced hierarchical structures into Transformer. By introducing multi-level feature representations, self-attention can be calculated at different resolution levels, allowing the model to simultaneously capture multi-scale global context features and local detail information.

[0007] Recently, the super-point mechanism was added to the Transformer. By imitating the clustering method, point cloud data is aggregated into super-points, which enhances the sensitivity to local boundary information and reduces the computational complexity and memory requirements exponentially, making the computation of the self-attention mechanism more efficient.

[0008] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0009] The purpose of the present invention is to provide a 3D point cloud semantic segmentation system that enhances vehicle boundary perception and global segmentation capabilities, so as to solve the problems of local boundary fuzziness and global segmentation confusion in the process of large-scale point cloud segmentation.

[0010] The present invention is based on a new hierarchical superpoint transformer framework, which mainly consists of five parts:

[0011] (1) Multi-layer perceptron (MLP) and relative position encoding module for hierarchical initial point cloud processing;

[0012] (2) Local feature enhancement module for improving boundary sensitivity;

[0013] (3) Dual attention module for super-point feature extraction;

[0014] (4) Large kernel convolution multi-scale fusion module for hierarchical feature fusion;

[0015] (5) Linear classifier for semantic mapping of 3D point clouds.

[0016] As a further configuration of the above invention, a multi-layer perceptron MLP and a relative position encoding module are used to extract 3D initial point cloud features acquired by a laser radar, and the steps include:

[0017] (1) Relative Position Encoding (RPE) constructs the encoding by calculating the relative coordinate difference or distance between points to better capture the spatial dependency and geometric structure between points.

[0018] (2) MLP performs nonlinear transformation through multiple hidden layers and activation functions to learn more complex feature representations.

[0019] As a further configuration of the above invention, due to the multi-layer encoder-decoder structure processing, local point cloud information will be lost to a large extent. The local feature enhancement module focuses on the fine-grained features in the point cloud, and learns the tiny detail differences of these similar object categories very well, achieving a good distinction effect. At the same time, when processing the segmentation of adjacent objects, it can effectively ensure the integrity of the object and solve the problem of local boundary segmentation ambiguity. By performing F on the input feature X 1X1 and F 3×3 The convolution operation is performed and batch normalized to obtain features U1 and U2. At the same time, all spatial positions of the input feature X are summed up, global statistics are extracted, and the statistical information is input into the fully connected layer W, and converted into weights by the Sigmoid function and multiplied by the input features to obtain the locally enhanced feature U en :

[0020]

[0021] Finally, the SiLU activation function is used to activate U1, U2, U en Working together, we get the local feature enhancement module output Y:

[0022] Y=SiLU(U1+U2+U en )

[0023] As a further configuration of the above invention, in order to solve the problem of point cloud semantic segmentation errors caused by insufficient spatial coordinate information and insufficient long-distance point relations under single self-attention, the present invention proposes a dual attention module, which consists of three parts: spatial coordinate attention mechanism, sparse self-attention mechanism and multiple partial convolutions.

[0024] As a further setting of the above invention, since the three-dimensional point cloud itself has a very complex spatial structure, it has an important impact on the accurate identification of the point cloud category. The spatial coordinate attention in the dual attention module fully captures the directional differences of the local boundary detail features, providing richer and more accurate directional information for subsequent feature processing. The spatial coordinate attention extracts the node features in the horizontal and vertical directions through an adaptive mean pooling operation. The average output in the vertical and horizontal directions is as follows:

[0025]

[0026] Then the channels are concatenated and passed through a lightweight convolutional layer F 1X1 , a directional attention map is generated through the nonlinear activation function δ, and then the vertical attention weight is formed through the lightweight convolution layer and the sigmoid activation function respectively. and horizontal attention weights Finally, the coordinate attention output of the present invention can be expressed as:

[0027]

[0028] As a further setting of the above invention, paying too much attention to local information may lead to the loss of global features, and fail to accurately segment similar points across long distances, resulting in local overfitting, which in turn leads to the problem of blurred local boundaries. The sparse self-attention in the dual attention module is used to further learn local boundary information while still ensuring the acquisition of global semantic information. The attention map of the sparse self-attention can be expressed as follows:

[0029]

[0030] Among them, ⊙ represents element-by-element multiplication, Represents the neighbor set of the current node, using the softmax function and V T The weighted attention map is obtained. The final output after neighbor feature aggregation and multi-head shared normalized weights is as follows:

[0031]

[0032] Among them, k represents k attention heads, Represents the index of the neighbor node.

[0033] As a further configuration of the above invention, multiple partial convolutions in the dual attention module can select important features through channel selection, while also avoiding feature information redundancy caused by the dual attention mechanism. The formula for a single partial convolution is as follows:

[0034]

[0035] Among them, K and L represent the height and width of the convolution kernel, w k,l represents the convolution kernel weight, is the input feature, m i,j The mask value is used to determine whether the input feature is valid. The final output after multiple partial convolutions.

[0036] As a further configuration of the above invention, the hierarchical feature fusion large kernel convolution multi-scale fusion module strengthens the dependency between super-points by hierarchically fusing super-point information of different resolutions, which helps the model maintain the consistency of object semantics and avoid global semantic segmentation confusion. And the maximum pooling feature P max Combine them and convert them into attention weights through the convolutional layer Different feature weight information is extracted through multiple attention heads, which are fused and then nonlinearly mapped F, and combined with the input feature X to form the final output feature Y.

[0037]

[0038] As a further configuration of the above invention, the linear classifier for 3D point cloud segmentation maps features to the number of categories through a fully connected layer, and outputs a probability distribution of each superpoint belonging to each category.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) This paper proposes a new 3D point cloud semantic segmentation framework based on hierarchical superpoint transformer, which can effectively solve the problems of unclear boundary division and global semantic division confusion when segmenting adjacent objects or similar objects.

[0041] (2) The present invention proposes a local feature enhancement module. When segmenting the boundaries between objects, it can perform local enhanced learning on the fine boundary features of similar objects. When processing the segmentation of adjacent objects, it can effectively ensure the integrity of the objects and solve the problem of local boundary segmentation fuzziness.

[0042] (3) The present invention proposes a dual attention feature extraction module to solve the problem of point cloud semantic segmentation errors caused by insufficient spatial coordinate information under single self-attention.

[0043] (4) The present invention proposes a hierarchical feature fusion mechanism to fully capture the relationship between super-points between layers and solve the problem of global semantic confusion caused by the loose and insufficient fusion between super-points.

[0044] (5) The present invention has been extensively experimentally verified on three large-scale challenge datasets S3DIS, KITTI-360, and DALES, proving that the 3D point cloud segmentation model of the present invention has excellent performance in processing local boundary segmentation and global large-scale segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a new 3D point cloud segmentation system based on hierarchical superpoint transformer of the present invention;

[0046] Figure 2 It is a schematic diagram of the structure of the dual attention mechanism;

[0047] Figure 3 It is a schematic diagram of the structure of the large kernel convolution multi-scale feature fusion module;

[0048] Figure 4 It is a schematic diagram of the structure of the local feature enhancement module;

[0049] Figure 5 It is a visual comparison diagram of the proposed model and the original baseline superpoint transformer (SPT) on the S3DIS dataset;

[0050] Figure 6 It is a visual comparison diagram of the model of the present invention and the original baseline SPT on the KITTI-360 dataset;

[0051] Figure 7 It is a visual comparison diagram of the model of the present invention and the original baseline SPT on the DALES dataset;

[0052] Figure 8 It is a schematic diagram comparing the evaluation index curve and loss curve of the model of the present invention with the original baseline SPT;

[0053] Fig. 9 This is a schematic diagram of the visual comparison of the ablation experiment;

[0054] Fig.10 It is a schematic diagram of the new 3D point cloud segmentation system based on hierarchical superpointtransformer. DETAILED DESCRIPTION

[0055] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.

[0056] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.

[0057] As attached Figures 1 to 10 As shown, a 3D point cloud semantic segmentation system for enhancing vehicle boundary perception and global segmentation capabilities according to a specific embodiment of the present invention is based on a new hierarchical superpoint transformer framework, which is mainly composed of five parts:

[0058] (1) Multi-layer perceptron (MLP) and relative position encoding module for hierarchical initial point cloud processing;

[0059] (2) Local feature enhancement module for improving boundary sensitivity;

[0060] (3) Dual attention module for super-point feature extraction;

[0061] (4) Large kernel convolution multi-scale fusion module for hierarchical feature fusion;

[0062] (5) Linear classifier for semantic mapping of 3D point clouds.

[0063] The multi-layer perceptron MLP and relative position encoding module are used to extract the 3D initial point cloud features acquired by the laser radar. The steps include:

[0064] (1) Relative Position Encoding (RPE) constructs the encoding by calculating the relative coordinate difference or distance between points to better capture the spatial dependency and geometric structure between points.

[0065] (2) MLP performs nonlinear transformation through multiple hidden layers and activation functions to learn more complex feature representations.

[0066] In the point cloud segmentation scenario of autonomous driving, there are a series of closely related objects, such as sidewalks and roads, trees and low plants, low walls and fences, and buildings and road edges. These objects are highly similar and difficult to segment. The local feature enhancement module focuses on the fine-grained features in the point cloud, and learns the tiny details of these similar object categories to achieve a good distinction effect. At the same time, when processing the segmentation of adjacent objects, it can effectively ensure the integrity of the object and solve the problem of local boundary segmentation ambiguity. By performing F on the input feature X, the local feature enhancement module can effectively ensure the integrity of the object and solve the problem of local boundary segmentation ambiguity. 1X1 and F 3×3The convolution operation is performed and batch normalized to obtain features U1 and U2. At the same time, all spatial positions of the input feature X are summed up, global statistics are extracted, and the statistical information is input into the fully connected layer W, and converted into weights by the Sigmoid function and multiplied by the input features to obtain the locally enhanced feature U en :

[0067]

[0068] Finally, the SiLU activation function is used to activate U1, U2, U en Working together, we get the local feature enhancement module output Y:

[0069] Y=SiLU(U1+U2+U en )

[0070] In order to solve the problem of point cloud semantic segmentation errors caused by insufficient spatial coordinate information and insufficient long-distance point relations under single self-attention, the present invention proposes a dual attention module, which consists of three parts: spatial coordinate attention mechanism, sparse self-attention mechanism and multiple partial convolutions.

[0071] (1) Spatial coordinate attention mechanism. Since the three-dimensional point cloud itself has a very complex spatial structure, it has an important impact on the accurate identification of the point cloud category. The spatial coordinate attention in the dual attention module fully captures the directional differences of local boundary detail features and provides richer and more accurate directional information for subsequent feature processing. The spatial coordinate attention extracts node features in the horizontal and vertical directions through an adaptive mean pooling operation. The average output in the vertical and horizontal directions is as follows:

[0072]

[0073] Then the channels are concatenated and passed through a lightweight convolutional layer F 1X1 , a directional attention map is generated through the nonlinear activation function δ, and then the vertical attention weight is formed through the lightweight convolution layer and the sigmoid activation function respectively. and horizontal attention weights Finally, the coordinate attention output of the present invention can be expressed as:

[0074]

[0075] (2) Sparse self-attention mechanism. Paying too much attention to local information may lead to the loss of global features, and fail to accurately segment similar points across long distances, resulting in local overfitting, which in turn leads to the problem of blurred local boundaries. The sparse self-attention in the dual attention module is used to further learn local boundary information while still ensuring the acquisition of global semantic information. The attention map of sparse self-attention can be expressed as follows:

[0076]

[0077] Among them, ⊙ represents element-by-element multiplication, Represents the neighbor set of the current node, using the softmax function and V T The weighted attention map is obtained. The final output after neighbor feature aggregation and multi-head shared normalized weights is as follows:

[0078]

[0079] Among them, k represents k attention heads, Represents the index of the neighbor node.

[0080] (3) Multiple partial convolutions. The multiple partial convolutions in the dual attention module can select important features through channel selection, while also avoiding the feature information redundancy brought about by the dual attention mechanism. The formula for a single partial convolution is as follows:

[0081]

[0082] Among them, K and L represent the height and width of the convolution kernel, w k,l represents the convolution kernel weight, is the input feature, m i,j The mask value is used to determine whether the input feature is valid. The final output after multiple partial convolutions.

[0083] The large kernel convolution multi-scale fusion module of hierarchical feature fusion strengthens the dependency between super-points by hierarchically fusing super-point information of different resolutions, which helps the model maintain the consistency of object semantics and avoid global semantic segmentation confusion. And the maximum pooling feature P max Combine them and convert them into attention weights through the convolutional layer Different feature weight information is extracted through multiple attention heads, which are fused and then nonlinearly mapped F, and combined with the input feature X to form the final output feature Y.

[0084]

[0085] The linear classifier for 3D point cloud segmentation maps features to the number of categories through a fully connected layer, and the output is the probability distribution of each superpoint belonging to each category.

[0086] For the above invention, S3DIS, KITTI-360 and DALES datasets are used for verification and performance evaluation.

[0087] The S3DIS Stanford large-scale 3D indoor dataset provides three independent spatial buildings, 6 large indoor areas, and a total of more than 200 million annotated point cloud data and corresponding 13 semantic labels, such as offices, conference rooms, stairs, corridors, etc., involving a total of 271 rooms of point cloud data. Each point is represented by a six-dimensional vector, including coordinates and color information. The present invention uses the segmentation results in zone 5 to evaluate the model of the present invention to verify its generalization ability. Although the S3DIS dataset is not an autonomous driving dataset, its rich point cloud data and semantic categories can well detect the effectiveness of the present invention for 3D point cloud segmentation and verify the generalization ability of the model of the present invention.

[0088] KITTI-360 is a large-scale autonomous driving dataset with rich sensory information and complete annotations. It covers 320,000 images and 100,000 laser scans, covering a total driving distance of 73.7 kilometers. In addition to static scene elements, it also includes dynamic objects such as cars, trucks, pedestrians, etc. This dataset contains rich images, laser scan information, and detailed 3D and 2D annotations. 19 semantic label categories are provided. KITTI-360 is currently the authoritative and challenging benchmark in the field of autonomous driving.

[0089] The DALES large-scale outdoor dataset was released by the German Aerospace Center (DLR). It provides a new aerial perspective to scan outdoor building objects. It contains more than 10 square kilometers of area scans, which cover a variety of terrains such as forests, farmlands, and cities. The dataset provides more than 500 million manually labeled points, divided into 8 semantic categories, such as ground, buildings, vehicles, etc. This is very challenging for the segmentation capabilities of autonomous driving algorithms.

[0090] The present invention uses mIoU (mean Intersection over Union) as an evaluation indicator to measure the segmentation effect of the segmentation model for each category and the overall performance. The formula of mIoU is as follows:

[0091]

[0092] Where TP is the number of true positive samples, FP is the number of false positive samples, FN is the number of false negative samples, N is the number of semantic categories, and N is the total number of samples.

[0093] We use the same model configuration for all three datasets. We set the number of channels for S3DIS and DALES to 64 and the number of channels for KITTI360 to 128. We use a superpoint loss rate of 0.2 to reflect the training loss caused by each superpoint in different hierarchical structures, as well as random rotation, tilt, point jitter, and handcrafted features. The number of points in each superpoint ranges from 32 to 128. We use the ADAMW optimizer with its default parameters and a weight decay of 10. -4 ; Learning rate: DALES is 10 -2 , KITTI-360, S3DIS is 10 -1 The learning rate of the dual attention module is 10 times smaller than that of other modules. The present invention warms up the learning rate in the first 20 epochs of training each data set, with an initial value of 10 -6 Then use cosine annealing to gradually decay the learning rate to 10 -6 . All experiments were conducted on an NVIDIA Tesla P40 GPU. The present invention does not set a training random number seed to ensure that the model can adapt to different random conditions during each training, thereby improving its generalization ability to unknown data. All experiments are evaluated by the loss on the validation set, and then the model obtained under multiple epochs is tested, and the average intersection-over-union ratio of the last epoch is taken as the evaluation result.

[0094] The five best performing models in the field, PointNet++, ConvPoint, SPG, PointCNN and SPT, are compared with the model proposed in this invention to evaluate their performance. The average intersection-over-union statistics of the DALES dataset detection results are shown in the following table:

[0095]

[0096] The method proposed in the present invention has improvements over the baseline method in segmenting plants, cars, trucks, power lines, pillars and buildings. In particular, the segmentation of trucks and pillars is improved by 4.2% and 4.7% compared with the baseline SPT, and the mIoU value is relatively improved by 1.4%.

[0097] The average intersection-over-union statistics of the KITTI-360 autonomous driving data set detection results are shown in the following table:

[0098]

[0099]

[0100] The method proposed in the present invention has improvements compared with the baseline method when segmenting walls, traffic lights, traffic signs, ground, pedestrians, cars and motorcycles. In particular, the segmentation of traffic lights and motorcycles is improved by 8.8% and 8.2% compared with the baseline SPT, and the mIoU value is relatively improved by 1.2%.

[0101] At the same time, it can be seen from the visualization of three challenging scenes that the model of the present invention consistently produces highly accurate boundary segmentation effects and global segmentation capabilities, which shows that the model of the present invention has strong generalization capabilities for various types of data sets and can effectively process their large-scale point cloud data and perform accurate segmentation and recognition.

[0102] To verify the effectiveness of each component in the present invention, the present invention conducted three related ablation experiments and used DALES for evaluation. The steps include:

[0103] (1) Remove the dual attention feature extraction module, use the original model's single sparse self-attention mechanism experiment, and use the model to detect and record;

[0104] (2) Remove the local feature enhancement module and use the model to detect and record;

[0105] (3) Remove the large kernel convolution multi-scale fusion module and use the model for detection and recording;

[0106] Experimental results show that when the dual attention feature extraction module in step (1) is removed, the model's mIoU is only 78.4%. This is because there is no local coordinate information when extracting the global context of the superpoint, and the local features are not rich enough. When facing the segmentation of similar objects, the difference between the boundaries of objects cannot be well controlled, resulting in classification errors.

[0107] When the local feature enhancement module in step (2) is removed, the model's mIoU is only 78.2%. This is because the point cloud data is processed by the encoder-decoder structure layer by layer, and local information will be lost to a large extent. Therefore, local feature aggregation must be performed before global feature processing. The local feature enhancement module can focus on the fine-grained features in the point cloud and learn the tiny details of these similar object categories. The difference achieves a good distinction effect, which just shows the importance of the local feature enhancement module and its effectiveness in the model position.

[0108] When the large kernel convolution multi-scale fusion module in step (3) is removed, the model's mIoU is only 78.6%. This is because the hierarchical detail features are not organically combined with the high-level semantic features to enable the model to fully understand the shape and structure of the object.

[0109] Finally, the present invention also performs ablation experiment visualization on the S3DIS dataset. The modules of the present invention complement each other and work together to achieve significant segmentation effects on doors, bookshelves, and sundries, indicating that the semantic segmentation model proposed by the present invention has outstanding advantages in boundary segmentation and global segmentation. Obviously, the dual attention feature extraction module, local feature enhancement module, and large kernel convolution multi-scale fusion module used in the present invention are all conducive to improving the final performance of the three-dimensional point cloud segmentation model.

[0110] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A 3D point cloud semantic segmentation system with enhanced vehicle boundary perception and global segmentation capabilities, characterized in that: Based on a new hierarchical Superpoint Transformer framework, the framework mainly consists of five parts: (1) Multi-layer perceptron (MLP) and relative position encoding module for hierarchical initial point cloud processing; (2) Local feature enhancement module for improving boundary sensitivity; (3) Dual attention module for super-point feature extraction; (4) Large kernel convolution multi-scale fusion module for hierarchical feature fusion; (5) Linear classifier for semantic mapping of 3D point clouds.

2. A 3D point cloud semantic segmentation system for enhancing vehicle boundary perception and global segmentation capabilities according to claim 1, characterized in that: The multi-layer perceptron MLP and relative position encoding module of part (1) are used to extract the 3D initial point cloud features acquired by the laser radar, and the steps include: (1) Relative Position Encoding (RPE) constructs the encoding by calculating the relative coordinate difference or distance between points to better capture the spatial dependency and geometric structure between points. (2) MLP performs nonlinear transformation through multiple hidden layers and activation functions to learn more complex feature representations.

3. The 3D point cloud semantic segmentation system for enhancing vehicle boundary perception and global segmentation capabilities according to claim 1, characterized in that: The local feature enhancement module in part (2) focuses on the fine-grained features in the point cloud, and learns the small details of similar object categories to achieve a good distinction effect. At the same time, when processing the segmentation of adjacent objects, it can effectively ensure the integrity of the object and solve the problem of local boundary segmentation ambiguity. 1X1 and F 3×3 The convolution operation is performed and batch normalized to obtain features U1 and U2. At the same time, all spatial positions of the input feature X are summed up, global statistics are extracted, and the statistical information is input into the fully connected layer W, and converted into weights by the Sigmoid function and multiplied by the input features to obtain the locally enhanced feature U en : Y=SiLU(U1+U2+U en ) Finally, the SiLU activation function is used to activate U1, U2, U en Working together, we get the local feature enhancement module output Y.

4. The 3D point cloud semantic segmentation system for enhancing vehicle boundary perception and global segmentation capabilities according to claim 1, characterized in that: The dual attention module of part (3) consists of three parts: spatial coordinate attention mechanism, sparse self-attention mechanism and multiple partial convolutions.

5. The 3D point cloud semantic segmentation system for enhancing vehicle boundary perception and global segmentation capabilities according to claim 1, characterized in that: The spatial coordinate attention in the dual attention module of part (3) fully captures the directional differences of local boundary detail features, providing richer and more accurate directional information for subsequent feature processing. The spatial coordinate attention extracts node features in the horizontal and vertical directions through an adaptive mean pooling operation. The average output in the vertical and horizontal directions is as follows: Then the channels are concatenated and passed through a lightweight convolutional layer F 1X1 , a directional attention map is generated through the nonlinear activation function δ, and then the vertical attention weight is formed through the lightweight convolution layer and the sigmoid activation function respectively. and horizontal attention weights Finally, the coordinate attention output of the present invention can be expressed as 6. According to the three-dimensional point cloud semantic segmentation system with enhanced vehicle boundary perception and global segmentation capabilities as described in claim 1, the sparse self-attention in the dual attention module of part (3) is used to further learn local boundary information while still ensuring the acquisition of global semantic information. The attention map of the sparse self-attention can be expressed as follows: in, ⊙ represents element-by-element multiplication, Represents the neighbor set of the current node, using the softmax function and The weighted attention map is obtained. The final output after neighbor feature aggregation and multi-head shared normalized weights is as follows: Among them, k represents k attention heads, Represents the index of the neighbor node.

7. According to the three-dimensional point cloud semantic segmentation system with enhanced vehicle boundary perception and global segmentation capabilities as described in claim 1, the multiple partial convolutions in the dual attention module of part (3) can select important features through channel selection, while also avoiding information redundancy caused by dual attention. The formula of a single partial convolution is as follows: in, K and L represent the height and width of the convolution kernel, w k,l represents the convolution kernel weight, is the input feature, m i,j The mask value is used to determine whether the input feature is valid. The final output after multiple partial convolutions.

8. According to the three-dimensional point cloud semantic segmentation system for enhancing vehicle boundary perception and global segmentation capabilities as described in claim 1, the large-core convolution multi-scale fusion module of the hierarchical feature fusion of part (4) strengthens the dependency between super-points by hierarchically fusing super-point information of different resolutions, which helps the model maintain the consistency of object semantics and avoid global semantic segmentation confusion. And the maximum pooling feature P max Combine them and convert them into attention weights through the convolutional layer Then, different feature weight information is extracted through multiple attention heads, which are fused and nonlinearly mapped F, and combined with the input feature X to form the final output feature Y.

9. A three-dimensional point cloud semantic segmentation system for enhancing vehicle boundary perception and global segmentation capabilities as described in claim 1, wherein the linear classifier of the 3D point cloud segmentation in part (5) maps features to the number of categories through a fully connected layer, and the output is the probability distribution of each super point belonging to each category.

Images