A 3D point cloud semantic segmentation system that enhances vehicle boundary perception and global segmentation capabilities

Abstract

This invention proposes a 3D point cloud semantic segmentation system to enhance vehicle boundary perception and global segmentation capabilities, addressing challenges such as blurred boundary delineation and global semantic confusion during the segmentation of adjacent or similar objects. The invention is based on a novel hierarchical Superpoint Transformer architecture, consisting of five main parts: (1) a multilayer perceptron (MLP) and relative position encoding for processing the initial point cloud at each level; (2) a local feature enhancement module; (3) a dual-attention feature extraction module; (4) a large-kernel convolutional hierarchical fusion network for multi-scale feature fusion; and (5) a linear layer classification head for 3D point cloud segmentation. The dual-attention module includes spatial coordinate attention, sparse self-attention, and multiple partial convolutions. The invention is validated using the S3DIS, KITTI-360, and DALES datasets. Comparison with state-of-the-art algorithms shows a significant improvement in average segmentation accuracy.

Description

Technical Field

[0001] This invention relates to the fields of computer vision processing technology, deep learning, and 3D point cloud. Background Technology

[0002] Recently, the Policy and Economics Research Institute and the Artificial Intelligence and Economic and Social Research Center of the China Academy of Information and Communications Technology (CAICT) jointly released a special report entitled "Autonomous Driving Strategy and Policy Observation (2024) - Policy and Regulations Help Accelerate the Industrialization of Highly Automated Driving." This report provides an in-depth analysis of the current development status of the autonomous driving industry, summarizes the important role of policies and regulations in promoting the industrialization of autonomous driving, and forecasts future development trends.

[0003] Autonomous driving technology, as a product of the deep integration of artificial intelligence, information and communication technologies, and automobiles, is accelerating profound changes in the automotive and transportation industries. The report points out that in recent years, with the rapid evolution of technologies such as artificial intelligence, big data models, and information and communication, the application scenarios of autonomous driving have become increasingly diverse. From Robotaxi (autonomous taxis) and Robobus (autonomous buses) to unmanned delivery vehicles, urban NOA (Advanced Navigation Assistance) systems, and autonomous parking, it is gradually integrating into people's daily lives, bringing unprecedented service experiences. Environmental perception is one of the key aspects in achieving autonomous driving. Vehicles need to accurately perceive their surroundings, accurately segment and identify the categories of surrounding objects, and thus make safe driving decisions. The perception process of autonomous vehicles typically requires the participation of 3D point clouds. Through point cloud segmentation, objects within the vehicle's driving radius are assigned corresponding semantic labels. The accuracy of point cloud segmentation directly affects the subsequent decision-making, path planning, and driver safety of autonomous vehicles.

[0004] Currently, image processing and 3D point cloud technologies form the core support for existing point cloud segmentation algorithms. Through computer vision algorithms, vehicles can extract key information from images, such as lane lines, traffic signs, pedestrians, and other vehicles. Deep learning models, especially convolutional neural networks (CNNs), perform exceptionally well in tasks such as image classification, object detection, and semantic segmentation. However, two-dimensional images have limitations in depth perception and spatial understanding, making it difficult to accurately estimate the distance and 3D shape of objects. To compensate for the shortcomings of 2D images, vehicles are typically equipped with sensors such as LiDAR (Light Detection and Ranging) to directly acquire 3D point cloud data of the environment. Point clouds consist of a large number of 3D coordinate points, accurately describing the spatial position and shape of surrounding objects. By processing point cloud data, vehicles can achieve high-precision 3D object detection, object classification, and scene understanding.

[0005] The accuracy of point cloud segmentation in autonomous driving scenarios largely depends on the performance of the point cloud segmentation algorithm, primarily its suitability for large-scale point cloud segmentation in autonomous driving environments. With continuous iteration of point cloud algorithms and the support of technologies such as LiDAR sensors, vision-based Transformers have been widely applied in point cloud segmentation. When segmenting large-scale point cloud data, the Transformer's unique parallel mechanism significantly improves segmentation efficiency, and it also offers strong flexibility and scalability, effectively segmenting irregular point cloud data. This precisely meets the complex requirements of large-scale autonomous driving scenarios.

[0006] Initially, researchers focused on various variants of Transformer self-attention, leading to multi-head self-attention, sparse self-attention, spatial channel self-attention, and adaptive self-attention. However, due to the inherent limitations of self-attention—its focus on global contextual information—it struggles to learn important local details of objects. Later, hierarchical structures were introduced into Transformer. By incorporating multi-level feature representations, self-attention can be computed at different resolution levels, enabling the model to simultaneously learn both global contextual features and local details across multiple scales.

[0007] Recently, a superpoint mechanism has been added to Transformer. By mimicking clustering methods, point cloud data is aggregated into superpoints, thereby enhancing sensitivity to local boundary information and significantly reducing computational complexity and memory requirements, making the computation of the self-attention mechanism more efficient.

[0008] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0009] The purpose of this invention is to provide a 3D point cloud semantic segmentation system that enhances vehicle boundary perception and global segmentation capabilities, in order to solve the problems of local boundary ambiguity and global segmentation confusion in the process of large-scale point cloud segmentation.

[0010] This invention is based on a novel hierarchical superpoint transformer framework, which mainly consists of five parts:

[0011] (1) Multilayer perceptron (MLP) and relative position encoding module for hierarchical initial point cloud processing;

[0012] (2) Local feature enhancement module for improving boundary sensitivity;

[0013] (3) Dual attention module for superpoint feature extraction;

[0014] (4) Large kernel convolutional multi-scale fusion module for hierarchical feature fusion;

[0015] (5) A linear classifier for semantic mapping of 3D point clouds.

[0016] As a further provision of the above invention, a multilayer perceptron (MLP) and a relative position encoding module are used to extract the features of the initial 3D point cloud acquired by the lidar, the steps of which include:

[0017] (1) Relative Position Coding (RPE) constructs codes by calculating the relative coordinate differences or distances between points, which better captures the spatial dependencies and geometric structures between points.

[0018] (2) MLP learns more complex feature representations by performing non-linear transformations through multiple hidden layers and activation functions.

[0019] As a further improvement to the above invention, due to the multi-layer encoder-decoder structure, local point cloud information is significantly lost. The local feature enhancement module focuses on fine-grained features in the point cloud, effectively learning the subtle differences between similar object categories and achieving excellent discrimination. Simultaneously, when segmenting adjacent objects, it effectively ensures object integrity and resolves the problem of ambiguity in local boundary segmentation. This is achieved by performing F... 1X1 and F 3×3 The convolutional operation is performed and batch normalized to obtain features U1 and U2. Simultaneously, the summation is applied to all spatial locations of the input feature X to extract global statistics. These statistics are input into the fully connected layer W and transformed into weights using the Sigmoid function. These weights are then multiplied by the input features to obtain the locally enhanced feature U. en :

[0020]

[0021] Finally, the SiLU activation function was applied to U1, U2, and U... en The combined effect yields the output Y of the local feature enhancement module:

[0022] Y = SiLU(U1 + U2 + U en )

[0023] As a further improvement to the above invention, in order to solve the problem of point cloud semantic segmentation errors caused by insufficient spatial coordinate information and insufficient long-distance point relationships under single self-attention, this invention proposes a dual attention module, which consists of three parts: spatial coordinate attention mechanism, sparse self-attention mechanism and multiple partial convolution.

[0024] As a further improvement to the above invention, the highly complex spatial structure of 3D point clouds significantly impacts the accurate identification of point cloud categories. The spatial coordinate attention module in the dual attention module fully captures the directional differences in local boundary detail features, providing richer and more accurate directional information for subsequent feature processing. Spatial coordinate attention extracts node features in the horizontal and vertical directions through an adaptive mean pooling operation. The average outputs in the vertical and horizontal directions are as follows:

[0025]

[0026] Then, channel stitching is performed, and the result is passed through a lightweight convolutional layer F. 1X1 A directional attention map is generated using a non-linear activation function δ, and then vertical attention weights are formed by passing the map through lightweight convolutional layers and a sigmoid activation function. And horizontal attention weights Finally, the coordinate attention output of this invention can be expressed as:

[0027]

[0028] As a further improvement to the above invention, excessive focus on local information may lead to the loss of global features, resulting in inaccurate segmentation of similar points spanning long distances, causing local overfitting, and consequently, blurred local boundaries. Sparse self-attention in the dual attention module further learns local boundary information while still ensuring the acquisition of global semantic information. The attention map of sparse self-attention can be represented as follows:

[0029]

[0030] Here, ⊙ represents element-wise multiplication. Represents the set of neighbors of the current node, using the softmax function and V. T The attention map is obtained through weighted aggregation. The final output, after neighbor feature aggregation and multi-head shared normalized weights, is as follows:

[0031]

[0032] Where k represents k attention heads, The index representing the neighboring node.

[0033] As a further improvement to the above invention, the multiple partial convolutions in the dual attention module can select important features through channel selection, while also avoiding feature information redundancy caused by the dual attention mechanism. The formula for a single partial convolution is as follows:

[0034]

[0035] Where K and L represent the height and width of the convolution kernel, w k,l Represents the convolution kernel weights. The input features, m i,j The mask value is used to determine whether the input features are valid. The final output is after multiple partial convolutions.

[0036] As a further improvement to the above invention, the large-kernel convolutional multi-scale fusion module for hierarchical feature fusion strengthens the dependencies between superpoints by hierarchically fusing superpoint information at different resolutions. This helps the model maintain the consistency of object semantics and avoid global semantic segmentation confusion. Average pooling features... and max pooling feature P max They are combined and transformed into attention weights through convolutional layers. Different feature weights are extracted by multiple attention heads, fused together and nonlinearly mapped F, and combined with the input feature X to form the final output feature Y.

[0037]

[0038] As a further feature of the invention, the linear classifier for 3D point cloud segmentation maps features to the number of categories through a fully connected layer, and outputs the probability distribution of each superpoint belonging to each category.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) This invention proposes a new three-dimensional point cloud semantic segmentation framework based on hierarchical superpointtransformer, which can effectively solve the problems of unclear boundary division and global semantic segmentation confusion when segmenting adjacent or similar objects.

[0041] (2) This invention proposes a local feature enhancement module. When segmenting the boundaries between objects, it can perform local enhancement learning on the fine boundary features of similar objects. When processing the segmentation of adjacent objects, it can effectively ensure the integrity of the objects and solve the problem of blurred local boundary segmentation.

[0042] (3) This invention proposes a dual attention feature extraction module to solve the problem of point cloud semantic segmentation error caused by insufficient spatial coordinate information under single self-attention.

[0043] (4) This invention proposes a hierarchical feature fusion mechanism to fully capture the relationship between superpoints between layers, and solves the problem of global semantic confusion caused by the loose and insufficient fusion between superpoints.

[0044] (5) The present invention has been extensively verified on three large challenge datasets: S3DIS, KITTI-360, and DALES, proving that the three-dimensional point cloud segmentation model of the present invention has excellent performance in handling local boundary segmentation and global large-scale segmentation. Attached Figure Description

[0045] Figure 1 This invention is a novel 3D point cloud segmentation system based on a hierarchical superpoint transformer;

[0046] Figure 2 This is a schematic diagram of the dual attention mechanism;

[0047] Figure 3 This is a schematic diagram of the structure of a large-kernel convolutional multi-scale feature fusion module;

[0048] Figure 4 This is a schematic diagram of the local feature enhancement module structure;

[0049] Figure 5 This is a visual comparison diagram of the model of this invention and the original baseline superpoint transformer (SPT) on the S3DIS dataset;

[0050] Figure 6 This is a visual comparison diagram of the model of this invention and the original baseline SPT on the KITTI-360 dataset;

[0051] Figure 7 This is a visual comparison diagram of the model of this invention and the original baseline SPT on the DALES dataset;

[0052] Figure 8 This is a schematic diagram comparing the evaluation index curves and loss curves of the model of this invention with the original baseline SPT;

[0053] Figure 9 This is a visual comparison diagram of ablation experiments;

[0054] Figure 10 This is a schematic diagram of the process of a new 3D point cloud segmentation system based on hierarchical superpointtransformer. Detailed Implementation

[0055] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but it should be understood that the scope of protection of the present invention is not limited to the specific embodiments.

[0056] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprises" shall be understood to include the stated elements or components without excluding other elements or other components.

[0057] As attached Figures 1 to 10 As shown, a 3D point cloud semantic segmentation system for enhancing vehicle boundary perception and global segmentation capabilities according to a specific embodiment of the present invention is based on a novel hierarchical superpoint transformer framework, which mainly consists of five parts:

[0058] (1) Multilayer perceptron (MLP) and relative position encoding module for hierarchical initial point cloud processing;

[0059] (2) Local feature enhancement module for improving boundary sensitivity;

[0060] (3) Dual attention module for superpoint feature extraction;

[0061] (4) Large kernel convolutional multi-scale fusion module for hierarchical feature fusion;

[0062] (5) A linear classifier for semantic mapping of 3D point clouds.

[0063] A multilayer perceptron (MLP) and a relative position encoding module are used to extract features from the initial 3D point cloud acquired by the LiDAR. The steps include:

[0064] (1) Relative Position Coding (RPE) constructs codes by calculating the relative coordinate differences or distances between points, which better captures the spatial dependencies and geometric structures between points.

[0065] (2) MLP learns more complex feature representations by performing non-linear transformations through multiple hidden layers and activation functions.

[0066] In autonomous driving point cloud segmentation scenarios, there exist a series of closely related objects, such as sidewalks and roads, trees and low vegetation, low walls and fences, and buildings and road edges. These objects have high similarity, making segmentation difficult. The local feature enhancement module focuses on fine-grained features in the point cloud, effectively learning the subtle differences between these similar object categories, achieving good discrimination results. Simultaneously, when segmenting adjacent objects, it effectively ensures object integrity and solves the problem of ambiguity in local boundary segmentation. This is achieved by performing F... 1X1 and F 3×3The convolutional operation is performed and batch normalized to obtain features U1 and U2. Simultaneously, the summation is applied to all spatial locations of the input feature X to extract global statistics. These statistics are input into the fully connected layer W and transformed into weights using the Sigmoid function. These weights are then multiplied by the input features to obtain the locally enhanced feature U. en :

[0067]

[0068] Finally, the SiLU activation function was applied to U1, U2, and U... en The combined effect yields the output Y of the local feature enhancement module:

[0069] Y = SiLU(U1 + U2 + U en )

[0070] To address the point cloud semantic segmentation errors caused by insufficient spatial coordinate information and lack of long-distance point relationships under single self-attention, this invention proposes a dual attention module, which consists of three parts: a spatial coordinate attention mechanism, a sparse self-attention mechanism, and multiple partial convolutions.

[0071] (1) Spatial Coordinate Attention Mechanism. Due to the highly complex spatial structure of 3D point clouds, accurate point cloud classification is significantly affected. The spatial coordinate attention mechanism in the dual attention module fully captures the directional differences in local boundary details, providing richer and more accurate directional information for subsequent feature processing. Spatial coordinate attention extracts node features in the horizontal and vertical directions through an adaptive mean pooling operation. The average outputs in the vertical and horizontal directions are as follows:

[0072]

[0073] Then, channel stitching is performed, and the result is passed through a lightweight convolutional layer F. 1X1 A directional attention map is generated using a non-linear activation function δ, and then vertical attention weights are formed by passing the map through lightweight convolutional layers and a sigmoid activation function. And horizontal attention weights Finally, the coordinate attention output of this invention can be expressed as:

[0074]

[0075] (2) Sparse Self-Attention Mechanism. Overemphasizing local information can lead to the loss of global features, resulting in inaccurate segmentation of similar points spanning long distances, causing local overfitting and blurring of local boundaries. Sparse self-attention in the dual attention module further learns local boundary information while still ensuring the acquisition of global semantic information. The attention graph of sparse self-attention can be represented as follows:

[0076]

[0077] Here, ⊙ represents element-wise multiplication. Represents the set of neighbors of the current node, using the softmax function and V. T The attention map is obtained through weighted aggregation. The final output, after neighbor feature aggregation and multi-head shared normalized weights, is as follows:

[0078]

[0079] Where k represents k attention heads, The index representing the neighboring node.

[0080] (3) Multiple Partial Convolution. Multiple partial convolution in the dual attention module can select important features through channel selection, while also avoiding feature information redundancy inherent in the dual attention mechanism. The formula for a single partial convolution is shown below:

[0081]

[0082] Where K and L represent the height and width of the convolution kernel, w k,l Represents the convolution kernel weights. The input features, m i,j The mask value is used to determine whether the input features are valid. The final output is after multiple partial convolutions.

[0083] The hierarchical feature fusion module, a large-kernel convolutional multi-scale fusion module, strengthens the dependencies between superpoints by hierarchically fusing superpoint information at different resolutions. This helps the model maintain the semantic consistency of objects and avoids global semantic segmentation confusion. It also incorporates average pooling features. and max pooling feature P max They are combined and transformed into attention weights through convolutional layers. Different feature weights are extracted by multiple attention heads, fused together and nonlinearly mapped F, and combined with the input feature X to form the final output feature Y.

[0084]

[0085] A linear classifier for 3D point cloud segmentation maps features to the number of classes through a fully connected layer, and outputs the probability distribution of each superpoint belonging to each class.

[0086] The invention was validated and its performance evaluated using the S3DIS, KITTI-360, and DALES datasets.

[0087] The S3DIS Stanford Large-Scale 3D Indoor Dataset provides three independent architectural spaces, six large indoor areas, and over 200 million annotated point cloud data points with corresponding 13 semantic labels, such as offices, meeting rooms, staircases, and corridors, covering a total of 271 rooms. Each point is represented by a six-dimensional vector, including coordinates and color information. This invention uses the segmentation results in region 5 to evaluate the model and verify its generalization ability. Although the S3DIS dataset is not an autonomous driving dataset, its rich point cloud data and semantic categories can effectively test the effectiveness of this invention in 3D point cloud segmentation and verify the generalization ability of the model.

[0088] KITTI-360 is a large-scale autonomous driving dataset with rich sensory information and complete annotations. It comprises 320,000 images and 100,000 laser scans, covering a total driving distance of 73.7 kilometers. In addition to static scene elements, it includes dynamic objects such as cars, trucks, and pedestrians. This dataset contains rich image and laser scan information, as well as detailed 3D and 2D annotations. It provides 19 semantic label categories. KITTI-360 is currently the authoritative and challenging benchmark in the field of autonomous driving.

[0089] The DALES large-scale outdoor dataset, released by the German Aerospace Center (DLR), provides a novel aerial perspective for scanning outdoor structures. It encompasses over 10 square kilometers of scanned areas, covering diverse terrains such as forests, farmland, and urban areas. The dataset includes over 500 million manually labeled points categorized into eight semantic classes, such as ground, buildings, and vehicles. This presents a significant challenge to the segmentation capabilities of autonomous driving algorithms.

[0090] This invention uses mIoU (mean Intersection over Union) as an evaluation metric to measure the segmentation performance of the segmentation model for each class and its overall performance. The formula for mIoU is as follows:

[0091]

[0092] Where TP represents the number of true positive samples, FP represents the number of false positive samples, FN represents the number of false negative samples, N represents the number of semantic categories, and N represents the total number of samples.

[0093] This invention uses the same model configuration for all three datasets. The number of channels for S3DIS and DALES is set to 64, and the number of channels for KITTI360 is set to 128. A superpoint loss rate of 0.2 is used to reflect the training loss caused by different hierarchical structures, random rotations, skew, point jitter, and handcrafted characteristics of each superpoint. The number of points within each superpoint ranges from 32 to 128. This invention uses the ADAMW optimizer with its default parameters and a weight decay of 10. -4 Learning rate: DALES is 10. -2 KITTI-360, S3DIS is 10 -1 The dual attention module has a learning rate 10 times smaller than other modules. This invention warms up the learning rate during the first 20 epochs of training each dataset, with an initial value of 10. -6 Then, cosine annealing is used to gradually decay the learning rate to 10. -6 All experiments were conducted on an NVIDIA Tesla P40 GPU. This invention does not use a training random number seed to ensure the model can adapt to different random conditions during each training iteration, thereby improving its generalization ability to unknown data. All experiments were evaluated using loss on the validation set, and then the models obtained over multiple epochs were tested, with the average intersection-over-union ratio (IoU) of the last epoch used as the evaluation result.

[0094] The proposed model is compared with five of the best-performing models in the field—PointNet++, ConvPoint, SPG, PointCNN, and SPT—to evaluate its performance. The average intersection-over-union (IoU) ratio of the detection results on the DALES dataset is shown in the table below.

[0095]

[0096] The method proposed in this invention improves upon baseline methods when segmenting plants, carts, trucks, power lines, pillars, and buildings. In particular, it improves the SPT by 4.2% and 4.7% relative to the baseline when segmenting trucks and pillars, and the mIoU value is improved by 1.4%.

[0097] The average intersection-over-union (IoU) ratio of the detection results from the KITTI-360 autonomous driving dataset is shown in the table below:

[0098]

[0099]

[0100] The method proposed in this invention improves upon the baseline method when dividing walls, traffic lights, traffic signs, ground, pedestrians, cars, and motorcycles. In particular, it improves the SPT (Segmentation Point Test) by 8.8% and 8.2% respectively when dividing traffic lights and motorcycles, and the mIoU (middle IoU) value is improved by 1.2%.

[0101] Meanwhile, as can be seen from the three challenging scenario visualizations, the model of this invention consistently produces highly accurate boundary segmentation results and global segmentation capabilities. This demonstrates that the model of this invention has strong generalization ability for various types of datasets, and can effectively handle large-scale point cloud data and perform accurate segmentation and recognition.

[0102] To verify the effectiveness of each component in this invention, three related ablation experiments were conducted, and the steps for evaluating them using DALES included:

[0103] (1) Remove the dual attention feature extraction module, and use the original model's single sparse self-attention mechanism to conduct experiments, and use the model to detect and record;

[0104] (2) Remove the local feature enhancement module, and use the model to perform detection and record;

[0105] (3) Remove the large kernel convolutional multi-scale fusion module, and use the model to perform detection and recording;

[0106] Experimental results show that when the dual attention feature extraction module in step (1) is removed, the model's mIoU is only 78.4%. This is because there is no local coordinate information to support the global context extraction of superpoints, and the local feature richness is insufficient. When facing the segmentation of similar objects, it cannot control the difference between the object and the object boundary well, resulting in classification errors.

[0107] When the local feature enhancement module in step (2) is removed, the model's mIoU is only 78.2%. This is because point cloud data is processed by layers of encoder-decoder structures, resulting in a significant loss of local information. Therefore, local feature aggregation must be performed before global feature processing. The local feature enhancement module can focus on fine-grained features in the point cloud, effectively learning the subtle differences between similar object categories and achieving excellent differentiation. This precisely illustrates the importance of the local feature enhancement module and its effectiveness in the model.

[0108] When the large kernel convolutional multi-scale fusion module in step (3) is removed, the model's mIoU is only 78.6%. This is because the detailed features of the hierarchical level are not organically combined with the high-level semantic features, so that the model can fully understand the shape and structure of the object.

[0109] Finally, ablation experiments were visualized on the S3DIS dataset. The modules of this invention complement each other and work together, achieving significant segmentation results for doors, bookshelves, and clutter, demonstrating that the semantic segmentation model proposed in this invention has outstanding advantages in boundary and global segmentation. Clearly, the dual attention feature extraction module, local feature enhancement module, and large-kernel convolutional multi-scale fusion module used in this invention all contribute to improving the final performance of the 3D point cloud segmentation model.

[0110] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A three-dimensional point cloud semantic segmentation system that enhances vehicle boundary perception and global segmentation capabilities, the system comprising: Based on a new hierarchical Superpoint Transformer framework, which mainly consists of five parts: (1) Multi-layer Perceptron (MLP) and Relative Position Encoding (RPE) module for hierarchical initial point cloud processing; (2) Local feature enhancement module for improving boundary sensitivity; (3) Dual attention module for superpoint feature extraction, consisting of three parts: spatial coordinate attention mechanism, sparse self-attention mechanism, and multi-part convolution; The spatial coordinate attention learns the direction difference of local boundary detail features, and the spatial coordinate attention extracts horizontal direction node features through adaptive mean pooling operation and vertical direction node features The node feature expression formulas of the vertical and horizontal directions are as follows: Two features are channel-level spliced, and then pass through a lightweight convolutional layer F 1X1 After processing, the direction attention map is generated through a nonlinear activation function δ; then, it passes through a lightweight convolutional layer F 1X1 and a sigmoid activation function to form the vertical attention weight and the horizontal attention weight Finally, the coordinate attention output is obtained The formula is as follows: Sparse self-attention learns local boundary information and global semantic information; the attention graph of sparse self-attention is represented as follows: where denotes element-wise multiplication, denotes the neighbor set of the current node, and the softmax function and V T weighted attention map, and then through the neighbor feature aggregation and multi-head shared normalization weight; finally, the output of the sparse self-attention mechanism The formula is expressed as follows: wherein k represents the kth attention head, a set of indices representing neighbor nodes; Multi-part convolution selects important features through channel selection, and the formula of a single part convolution is as follows: where K and L represent the width and height of the convolution kernel, i and j are the traversal indexes on the voxel height and width; w k,l represents the convolution kernel weight, is the input feature; m i,j is the mask value, used to determine whether the input feature is valid; is the output of the multi-part convolution module; (4) Large kernel convolution multi-scale fusion module for hierarchical feature fusion; (5) Linear classifier for 3D point cloud semantic mapping.

2. The three-dimensional point cloud semantic segmentation system for enhancing vehicle boundary awareness and global segmentation capability of claim 1, wherein, The multi-layer perceptron (MLP) and relative position encoding module of part (1) is used to extract the 3D initial point cloud features obtained by laser radar, and the steps include: (1) Relative position encoding (RPE) constructs encoding by calculating the relative coordinate difference or distance between points, and learns the geometric structure between adjacent point clouds; (2) MLP learns more complex feature representation through multiple hidden layers and activation functions.

3. The three-dimensional point cloud semantic segmentation system for enhancing vehicle boundary awareness and global segmentation capability of claim 1, wherein, The local feature enhancement module of part (2) focuses on the fine-grained features in the point cloud; by performing F 1X1 and F 3×3 Two convolution operations and batch normalization get features U1 and U2; then sum all spatial positions of the input feature X, extract global statistics, input the statistical information to the fully connected layer W, and convert it to weights through the Sigmoid function σ to multiply the input feature, to get the locally enhanced feature U en , U en The formula is as follows: Finally, the SiLU activation function is used to U1, U2, U en The three enhanced features work together to obtain a local feature enhancement module output Y, and the formula expression of Y is as follows: Y = SiLU(U1+U2+U en ).

4. The three-dimensional point cloud semantic segmentation system for enhancing the vehicle boundary perception and global segmentation capability according to claim 1, wherein the large kernel convolution multi-scale fusion module of the hierarchical feature fusion part (4) fuses the super point information of different resolutions hierarchically; the average pooling features P and the maximum pooling features P max are combined, and they are converted into attention weights P Then extract different feature weight information through multiple attention heads, fuse them, and perform nonlinear mapping through a light convolution layer F, and multiply them with the input feature X to get the final output feature Y, the formula of Y is as follows:

5. The three-dimensional point cloud semantic segmentation system for enhancing the boundary perception and global segmentation capability of a vehicle according to claim 1, the linear classifier of 3D point cloud segmentation in part (5) maps the features to the number of categories through a fully connected layer, and the output is the probability distribution of each superpoint belonging to each category.

Images