A point cloud semantic segmentation method based on bilateral augmentation and attention-based collaborative enhancement

Abstract

This application provides a point cloud semantic segmentation method based on bilateral augmentation and attention-based collaborative enhancement. The method includes: firstly, a local bilateral augmentation module encodes relative positions and Euclidean distances in the geometric path and introduces feature differences in the semantic path to capture semantic mutations, achieving deep coupling between geometry and semantics; subsequently, a dual-attention regularization module employs spatial-channel collaborative weighting and bilinear higher-order interactions to suppress attention divergence and feature redundancy; finally, a residual subtraction connection module sharpens details through subtraction paths and compensates for information loss through residual compensation paths, enhancing feature discriminativeness while ensuring information integrity. Experimental results on the S3DIS dataset Area5 show that the average intersection-over-union ratio (IoU) of this invention reaches 66.2%, a 3.8% improvement over the baseline RandLA-Net, with a 21.8% improvement for the sofa category, significantly improving the segmentation accuracy for complex structures and easily confused categories.

Description

Technical Field

[0001] This invention relates to the fields of 3D point cloud processing and computer vision technology, and in particular to a point cloud semantic segmentation method based on bilateral augmentation and attention-based collaborative enhancement. Background Technology

[0002] Point cloud semantic segmentation is a core task in the field of 3D computer vision, aiming to assign predefined semantic category labels to discrete points in 3D space. It is a fundamental supporting technology for fields such as autonomous driving, robot navigation, digital twins, and augmented reality. With the large-scale application of 3D acquisition devices, point cloud data is growing exponentially. How to efficiently and accurately perform point-by-point semantic segmentation on large-scale irregular point clouds has become a core challenge of common concern to academia and industry.

[0003] In recent years, end-to-end methods based on deep learning have completely replaced traditional approaches and become the mainstream research method. Based on the different formats of 3D data organization, existing deep learning methods can be mainly divided into the following three categories: Multi-view projection-based methods project point clouds onto a series of two-dimensional planes (such as spheres, bird's-eye views, etc.), generating regular meshed images, which are then classified using two-dimensional convolutional networks. Representative methods include VPA-Net and RangeNet++. Their significant advantages are computational efficiency and the ability to leverage pre-trained models based on images. However, the 3D-to-2D projection process is essentially a dimensionality reduction mapping, inevitably leading to the loss of geometric information, such as depth discontinuities, occlusion relationships, and the difficulty in fully preserving fine structures. Furthermore, the choice of projection viewpoint is sensitive; a single viewpoint cannot cover the entire complex scene, limiting segmentation accuracy.

[0004] Voxelization-based methods discretize a continuous 3D space into a regular voxel mesh, converting the point cloud into a dense or sparse 3D tensor and employing a 3D convolutional neural network for feature learning. Representative methods include DRINet++ and 3D-UMamba. Voxelization preserves the spatial neighborhood structure, facilitating direct modeling of local geometric relationships in 3D convolution. However, it faces significant computational and memory bottlenecks: voxel resolution and storage requirements increase cubically, making high-resolution meshes unsustainable in terms of GPU memory and computing power. Reducing resolution introduces quantization errors, leading to a severe deterioration in the segmentation quality of thin-walled structures, small objects, and fine boundaries. While sparse convolution alleviates memory pressure, the quantization error problem remains unresolved.

[0005] Point-based methods: These methods directly define network operators on irregular point clouds to preserve the original geometric information to the greatest extent, and are the mainstream research direction for semantic segmentation. Since PointNet proposed point-wise MLP and symmetric aggregation, the ability to describe local features has been continuously improved: PointNet++ captures multi-scale geometric context through hierarchical sampling and local pooling; RS-CNN explicitly models the geometric topological relationships between points, enhancing local shape sensitivity; RandLA-Net replaces farthest point sampling with random sampling, and combined with a local feature aggregation module, it achieves real-time processing of millions of point clouds, becoming an industry benchmark; PointTransformer introduces self-attention into the neighborhood, generates dynamic weights through feature similarity, establishes long-range dependencies, and further improves feature representation capabilities.

[0006] Despite significant progress in local feature learning, the aforementioned methods still face key bottlenecks in feature aggregation: ① Insufficient synergy between geometry and semantics: Geometric modeling is often limited to relative coordinates and Euclidean distance, lacking explicit modeling of higher-order attributes such as curvature, radial decay, and normal vectors; semantic aggregation commonly employs weighted averaging or attention pooling, whose low-pass filtering characteristics easily smooth out semantic abrupt changes at boundaries, leading to blurred segmentation boundaries and positioning offsets for thin-walled structures and small objects. The geometric and semantic approaches have evolved independently, failing to form a deeply coupled enhancement mechanism. ② Single-dimensional attention mechanism: Existing attention mechanisms are mostly limited to a single spatial or channel dimension, lacking joint modeling and explicit constraints on weight distribution. This easily leads to attention divergence and weight homogenization, resulting in feature redundancy and decreased inter-class separability. This problem is particularly prominent in geometrically complex categories (such as cylinders and curved sofas) and categories with similar appearances (such as windows and walls, chairs and sofas). ③ Single feature fusion method: Skip connections and multi-feature fusion in encoder-decoder architecture generally adopt addition or concatenation. Addition strengthens commonalities and suppresses differences, while concatenation retains complete information but does not explicitly handle the relationship between features. Subsequent convolutions need to learn the filtering mechanism from scratch. Neither of these methods can highlight the uniqueness of local features relative to the global context, which seriously restricts the model's sensitivity to and ability to retain detailed information. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention constructs a geometric-semantic dual-path local augmentation mechanism to enhance input features at their source; it achieves adaptive recalibration and improved discriminative power of features through spatial-channel dual-dimensional attention-based weighting and higher-order interactions; and it achieves sharpening and complete preservation of local details through differential fusion and information compensation. These three modules work together organically to systematically solve the problems of information scarcity, filtering failure, and difference overwhelmance in the local feature aggregation process. While maintaining efficiency in large-scale scene processing, it significantly improves the segmentation accuracy for complex geometric structures, semantically easily confused categories, and small-scale objects.

[0008] The first aspect of this invention provides a point cloud semantic segmentation method based on bilateral augmentation and attention-based collaborative enhancement, comprising: S1. Obtain a large-scale point cloud dataset and divide the dataset into training and testing sets; S2. Construct a point cloud semantic segmentation network model based on bilateral augmentation and attention collaborative enhancement, abbreviated as BAACE-Net (Bilateral Augmentation and Attention Collaborative Enhancement for PointCloud Semantic Segmentation). The BAACE-Net model adopts an encoder-decoder architecture. Each layer of the encoder consists of a bilateral attention residual module and a random sampling layer, which gradually reduces the number of point clouds and increases the feature dimension to extract multi-scale hierarchical features. The decoder consists of a nearest neighbor interpolation upsampling layer and skip connections, which gradually restores deep features to the input point cloud resolution. Finally, a classification head composed of fully connected layers and Dropout layers outputs point-by-point semantic categories. The bilateral attention residual module includes a local feature aggregation module, a local bilateral expansion module, a dual attention regularization module, and a residual subtraction connection module. The local feature aggregation module is used to extract neighborhood features from the input point cloud and output the preliminary aggregated local features. The local bilateral augmentation module is used to receive the initially aggregated local features and enhance the representation of the local features through a dual-path collaborative augmentation mechanism of geometric path and semantic path. The dual-attention regularization module adaptively recalibrates the enhanced local features through the collaborative weighting of spatial attention branches and channel attention branches; and performs bilinear interactive operations on the recalibrated features to capture high-order nonlinear correlations in the spatial and channel dimensions, and outputs globally regularized features. The residual subtraction connection module is used to perform element-wise subtraction between the multi-dimensional enhanced local features and the global regularized features to obtain the difference features. At the same time, a learnable residual compensation path is introduced to dynamically compensate the difference features, and the final feature that integrates discriminative local details and global context information is output. S3. Train the BAACE-Net model using the input training set and optimize the model parameters; S4. Apply the optimized BAACE-Net model to the test set to obtain the semantic category prediction results for each point.

[0009] Furthermore, the local bilateral augmentation module enhances the representation of the initially aggregated local features through a dual-path collaborative augmentation mechanism of geometric and semantic paths, including: Geometric path based on center point coordinates coordinates of neighboring points Calculate the relative position vector as input. Euclidean distance , The calculation expression is: , The calculation expression is: In the formula, m represents the number of neighboring point coordinates obtained through the K-nearest neighbor algorithm. To express summation; to express the .... and After concatenation, the data is mapped using a multilayer perceptron to generate geometric feature codes. The expression is In the formula, Indicates a connector symbol; Semantic path with center point features Features of neighboring points Calculate the feature difference as input. ,Will and After concatenation, the dimensions are compressed using a lightweight convolutional network to generate semantic feature encodings. The expression is ; Encoding geometric features With semantic feature encoding After concatenation, the features are adjusted for dimensionality and fused nonlinearly using a multilayer perceptron, then aggregated using max pooling, and finally restored using another multilayer perceptron to output locally enhanced bilateral features. The expression is In the formula, This indicates global max pooling.

[0010] Furthermore, the geometric path employs a neighborhood adaptive sampling strategy, and the number of neighborhood samples for the geometric path is less than the number of neighborhood samples for the semantic path.

[0011] Furthermore, the dual-attention regularization module outputs a global regularization feature process, including: The spatial attention branch enhances the local bilateral features of the input. Features are obtained through channel compression. The expression is Subsequently, global max pooling and average pooling are performed along the feature dimension. The pooling results are concatenated and then passed through a convolutional layer with a sigmoid function to generate spatial attention weights. The expression is In the formula, Indicates global max pooling. Indicates global average pooling; Channel attention branches enhance the local bilateral features of the input. Features are obtained through channel compression. The expression is Subsequently, global max pooling and global average pooling are performed along the spatial dimension. The pooling results are concatenated and then passed through a convolutional layer with a sigmoid function to generate channel attention weights. The expression is ; Spatial attention weights and channel attention weights Local bilateral enhancement features for input Modulation is performed to obtain spatial modulation characteristics. With channel modulation characteristics ;right and Performing outer product operations to achieve bilinear interaction generates a space. The channel joint response matrix, which is compressed in dimension by a 1×1 convolution and activated by the ReLU function to obtain feature M, is expressed as follows: Finally, feature M and input local bilateral enhancement features The residuals are joined and summed to output the globally regularized features. The expression is .

[0012] Furthermore, the residual subtraction connection module enhances the discriminative power of local features and maintains information integrity through the complementary fusion of the subtraction path and the residual compensation path, including: Subtraction path for local bilateral enhancement features With global regularization features Perform element-wise subtraction to obtain differential features, thereby suppressing feature commonality and highlighting discriminative details; The residual path will enhance local bilateral features. With global regularization features The features are concatenated along the feature dimension and then transformed nonlinearly through a 1×1 convolutional layer to obtain adaptive compensation features, which dynamically compensate for the effective information that may be lost during the subtraction operation. The difference features output by the subtraction path and the adaptive compensation features output by the residual path are added and fused element by element, and then the refined local features F are output after activation by the Mish function.

[0013] A second aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor as described above in a point cloud semantic segmentation method based on bilateral augmentation and attention co-enhancement.

[0014] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the point cloud semantic segmentation method based on bilateral augmentation and attention co-enhancement as described above.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1) The local bilateral augmentation module explicitly encodes the relative position and Euclidean distance in the geometric path and introduces feature difference in the semantic path. The deep coupling of the two paths enables the local features to have rich geometric details and sharpened boundary response at the same time, which significantly improves the segmentation accuracy of thin-walled structures, slender objects and regions with blurred boundaries.

[0016] 2) The dual attention regularization module generates focused attention weights through a complementary dual pooling strategy in spatial and channel dimensions, and achieves high-order bilinear interaction through outer product operation. It explicitly models spatial-channel joint dependency, effectively suppresses attention divergence and feature redundancy, and enhances inter-class separability. It has obvious advantages for complex and easily confused categories such as cylinders, curved sofas, and window-wall surfaces. The residual connection ensures training stability and generalization at the same time.

[0017] 3) The subtraction path in the residual subtraction connection module captures the differences between local and global features, highlighting details and boundaries, and avoiding feature convergence caused by traditional fusion. The residual path uses concatenation and convolution to learn adaptive compensation terms, dynamically compensating for the information loss of the subtraction operation. The fusion of the two paths achieves the dual goals of improving feature discrimination and preserving information integrity.

[0018] 4) On the S3DIS Area5 dataset, the present invention achieves a mean Intersection over Union (mIoU), mean class accuracy (mAcc), and overall accuracy (OA) of 66.2%, 77.0%, and 87.3%, respectively, compared to RandLA. Net improvements of 3.8%, 5.6%, and 0.1%. It achieved best performance in difficult categories such as pillars, windows, and sofas, and maintained highly competitive performance in other categories.

[0019] 5) All three modules adopt a lightweight design, with controllable parameter count and computational overhead, and can be flexibly embedded into RandLA. It supports mainstream networks such as Net and PointNet++ without requiring modifications to the backbone structure and has good generalization and transfer capabilities. Attached Figure Description

[0020] Figure 1 The overall architecture diagram of the point cloud semantic segmentation network (BAACE-Net) enhanced by bilateral augmentation and attention collaboration.

[0021] Figure 2This is a diagram of the overall architecture of the local bilateral expansion module.

[0022] Figure 3 This is the overall architecture diagram of the dual attention regularization module.

[0023] Figure 4 This is the overall architecture diagram of the residual subtraction connection module. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0025] In one specific embodiment of this invention, a point cloud semantic segmentation network model with bilateral augmentation and attention-based collaborative enhancement is constructed, as follows: like Figure 1 As shown, the BAACE-Net proposed in this invention adopts an encoder-decoder architecture. This network combines the high efficiency of RandLA-Net in large-scale point cloud processing, and significantly improves the segmentation accuracy for complex geometric structures and easily confused categories by introducing three innovative modules: local bilateral augmentation, dual attention regularization, and residual subtraction connections. The input to BAACE-Net is... The large-scale point cloud data contains N points, each with three-dimensional coordinates (x, y, z) and optional additional attributes such as RGB color information and laser reflection intensity. The input feature dimension is... .

[0026] The encoder consists of L cascaded bilateral attention residual (BAR) modules and random sampling (RS) layers, alternating between them. Each BAR module enhances, calibrates, and refines the input features; the random sampling layers progressively reduce the number of points (typically downsampling by a factor of 4 per layer), while each BAR module significantly increases the feature dimension of each point through a multilayer perceptron (e.g., 8→32→128→256→512), thereby extracting hierarchical features with rich semantic information at different scales.

[0027] The decoder is symmetrical to the encoder and consists of L upsampling layers and skip connections. The upsampling layers use nearest neighbor interpolation to restore the low-resolution feature map to a high resolution. The skip connections concatenate the features of the corresponding layer of the encoder with the upsampled features and then fuse them through a multilayer perceptron to gradually restore the spatial details of the point cloud.

[0028] The classification head consists of a fully connected layer and a Dropout layer, which maps the high-dimensional features output by the decoder to logits of the number of categories (13 in the S3DIS dataset used in this embodiment), and outputs the semantic category probability of each point through the Softmax function.

[0029] The Bilateral Attention Residual Module (BAR) in BAACE-Net is the core feature extraction unit of this invention. It is composed of a Local Feature Aggregation (LFA) module, a Localized Bilateral Augmentation (LBA) module, a Dual Attention Regularization (DAR) module, and a Residual Subtraction Connection (RSC) module connected sequentially. This module addresses the problem of insufficient utilization of geometric and semantic information in existing local feature aggregation methods. Building upon the initial local features extracted by the LFA module, it enhances local feature representation through bilateral augmentation, improves feature discriminative power through dual attention collaborative weighting and higher-order interactions, and maintains information integrity while sharpening details through residual subtraction fusion. The modules work together organically to systematically improve the network's ability to perceive and segment complex structures.

[0030] In one specific embodiment of this example, the Local Bilateral Augmentation (LBA) module is used to enhance local features at the source through a dual-path collaborative augmentation mechanism of geometric and semantic paths, thereby addressing the lack of deep interaction between geometric information and semantic features in existing methods. The structure of the Local Bilateral Augmentation module is as follows: Figure 2 As shown, the specific implementation steps of this module are as follows: In the geometric feature augmentation path, this invention constructs a geometric descriptor based on relative position and distance metrics, providing the network with accurate spatial structure priors, suitable for objects with obvious geometric features. The center point coordinates are set as... The K-nearest neighbor algorithm is used to obtain the set of neighboring points in Euclidean space. Where m represents the number of neighboring point coordinates obtained through the K-nearest neighbor algorithm, and the relative position vector is calculated. Euclidean distance The expression is as follows: , (1) In the formula, m represents the number of neighboring point coordinates obtained through the K-nearest neighbor algorithm. This represents the m-th neighboring point of the i-th point. This indicates a summation.

[0031] Will (3D) and (1D) Concatenate along the feature dimension to form a 4D geometric descriptor. Input the geometric descriptor into a multilayer perceptron for nonlinear mapping to obtain a high-dimensional geometric feature code. The expression is as follows: (2) In the formula, Indicates a connector symbol.

[0032] In the semantic feature augmentation path, this invention designs a differential enhancement mechanism to capture local semantic changes and enhance the network's ability to segment regions with blurred boundaries. Given center point features... The features of neighborhood points are obtained through the nearest neighbor algorithm. Calculate the feature difference of the m-th neighborhood point. ,Will and After concatenation, the dimensions are compressed using a lightweight convolutional network (1×1 convolution) to generate semantic feature encoding. The expression is: (3) In the formula, m represents the number of neighborhood point coordinates obtained by the K-nearest neighbor algorithm.

[0033] Encoding geometric features With semantic feature encoding The feature dimension is concatenated along the feature dimension, and then dimensionality is adjusted and nonlinearly fused using a multilayer perceptron. This progressive dimensionality enables multi-level interaction between geometric and semantic information while reducing computational complexity and memory consumption. Subsequently, max pooling aggregation is performed on the neighborhood dimension to retain the most salient feature responses in local regions. Finally, the feature dimension is restored using a multilayer perceptron, outputting locally enhanced bilateral features that contain both spatial structure information and semantic contextual relationships. The expression is: (4) In the formula, This indicates global max pooling.

[0034] To balance computational efficiency and feature richness, this embodiment employs an asymmetric neighborhood sampling strategy for geometric and semantic paths: the semantic path uses the full number of neighborhood points (m=K), while the geometric path uses half the number of neighborhood points (m=K / 2). This strategy reduces the computational overhead of the geometric path by approximately 40% while maintaining geometric structure awareness.

[0035] In one specific implementation of this embodiment, such as Figure 3 As shown, the Dual Attention Regularization (DAR) module achieves adaptive recalibration and regularization of local features through the collaborative weighting of spatial attention branches and channel attention branches, as well as higher-order bilinear interactions, as detailed below: The spatial attention branch enhances the local bilateral features of the input. Features are obtained through channel compression (1×1 convolution). The expression is The purpose of dimensional compression is to reduce computational complexity while retaining sufficient information capacity. Subsequently, global max pooling and average pooling are performed along the feature dimension (i.e., the channel dimension) to capture the local saliency and global contextual information at each spatial location. The pooling results are concatenated and then passed through a convolutional layer (1×1 convolution) and a sigmoid function to generate spatial attention weights. The expression is shown in equation (5).

[0036] (5) In the formula, Indicates global max pooling. This indicates global average pooling.

[0037] Channel attention branches enhance the local bilateral features of the input. Features are obtained through channel compression. The expression is Subsequently, global max pooling and global average pooling are performed along the spatial dimension (i.e., the point dimension). The pooling results are concatenated along the feature dimension and then passed through a convolutional layer (1×1 convolution) and the Sigmoid function to generate channel attention weights. The expression is: (6) Spatial attention weights Local bilateral enhancement features for input Modulation is performed to obtain spatial modulation characteristics. The expression is In the formula, Represents the Hadamard product; with channel attention weights Local bilateral enhancement features for input Modulation is performed to obtain channel modulation characteristics. The expression is ;right and Performing outer product operations to achieve bilinear interaction generates a space. The joint response matrix of the channels is compressed in dimension by a 1×1 convolution and activated by the ReLU function to obtain feature M, as shown in equation (7). To prevent information loss and overfitting due to excessive compression, residual connections are introduced. Feature M is added to the input local features through residual connections, which preserves the original feature information while ensuring direct gradient backpropagation. Finally, the feature scale is harmonized through convolutional layers to obtain the globally regularized features. The expression is shown in equation (8).

[0038] (7) (8) In one specific implementation of this embodiment, such as Figure 4 As shown, the Residual Subtraction Connector (RSC) ensures information integrity while sharpening discriminative features through the complementary fusion of the subtraction path and the residual compensation path, as detailed below: The design of the residual subtraction connection module stems from considerations of the balance between information preservation and feature enhancement during the optimization of local point cloud features. Traditional feature fusion methods often employ addition or concatenation operations, which, while integrating multi-source information, struggle to highlight discriminative differences between features (additive fusion strengthens common components and suppresses discriminative information, while concatenation fusion preserves complete information but fails to explicitly process the relationships between features). Especially when dealing with geometrically complex regions, these fusion methods often lead to convergence of feature representations, reducing the model's segmentation sensitivity.

[0039] To address this, this invention introduces a subtraction operation to highlight the portions of local features not adequately represented by dual-attention regularization, thereby enhancing the model's discriminative ability. However, traditional direct subtraction operations enhance the discriminative power of local features by suppressing common features, resulting in irreversible loss of valuable information. Therefore, this invention introduces a learnable residual compensation path to dynamically compensate for over-suppressed feature information.

[0040] Specifically, the residual subtraction connection module contains the following two parallel paths: Subtraction path: Local bilateral enhancement features With global regularization features Perform element-wise subtraction to obtain differential features. This operation explicitly suppresses... and The common components highlight unique details in local features that are not covered by the global context, including discriminative features such as boundary responses and local geometric anomalies, thereby avoiding excessive smoothing that may occur during the dual attention regularization process.

[0041] Residual compensation path: local bilateral enhancement features With global regularization features The features are concatenated along the feature dimension and then transformed nonlinearly through a 1×1 convolutional layer to obtain adaptive compensation features. This path dynamically compensates for effective common information (such as stable background context and scene global priors) that may be oversuppressed by the subtraction operation in a learnable way, achieving differentiated fusion with "subtraction as the main method and compensation as the auxiliary method".

[0042] Fusion and Output: The difference features output by the subtraction path and the adaptive compensation features output by the residual compensation path are added element-wise and fused to obtain the fused feature Y, expressed as: (9) The fused feature Y is activated by the Mish activation function, and the refined local feature F is output. The expression is shown in Equation (10). The Mish function can retain the non-zero gradient characteristics in the negative region, which helps to maintain the information flow of the negative feature vector that may be generated by the subtraction operation, and is more suitable for the differentiated fusion scenario of this module.

[0043] (10) In the formula, Represents the hyperbolic tangent function; This represents a soft addition function.

[0044] In one specific implementation of this embodiment, the data and its source, as well as the network training and validation settings, include: This embodiment verifies the effectiveness of the proposed method on the Stanford University large-scale 3D indoor space dataset (S3DIS). This dataset contains six large indoor areas from three buildings, covering various scenes such as offices, meeting rooms, and corridors, totaling over 215 million 3D points. Each point contains XYZ coordinates and RGB color information, finely labeled with 13 semantic categories: ceiling, floor, walls, beams, columns, windows, doors, tables, chairs, sofas, bookshelves, wooden planks, and miscellaneous items. The domain-recognized Area5 is used as the independent test set, while the other five areas (Areas 1, 2, 3, 4, and 6) are used as the training set. This partitioning ensures that the training and test data come from different buildings to evaluate the model's generalization ability.

[0045] In this embodiment, the hardware environment uses an AMD Ryzen 7 5700X CPU (8 cores, 16 threads, 3.4GHz) and a single NVIDIA RTX 3080Ti graphics card (12GB VRAM), with Ubuntu 24.04.1 LTS operating system and CUDA version 12.4. The software environment uses the TensorFlow 2.6 deep learning framework and Python version 3.6.

[0046] During network training, the training parameters were set as follows: the optimizer was Adam, the initial learning rate was 0.01, the exponential decay factor was 0.95 / epoch, the batch size was 6, the number of training epochs was 100, and the number of neighborhood points K was 16. Data preprocessing involved dividing the data into 1m×1m blocks and randomly downsampling to 20480 points / block, while data augmentation strategies such as random rotation, scaling, and dithering were applied. The evaluation metrics used were mean intersection-over-union (mIoU), overall accuracy (OA), and mean class accuracy (mAcc).

[0047] Experimental Results and Comparative Analysis To better analyze the overall performance of the BAACE-Net network, this embodiment of the invention performs a quantitative performance evaluation of the BAACE-Net network on the S3DIS dataset Area5 and compares it with mainstream methods. The results are shown in Table 1.

[0048] Table 1 Comparison of results for Area5 on the S3DIS dataset. As shown in Table 1, the proposed BAACE-Net achieves the best performance in both mean Intersection over Union (mIoU) and mean class accuracy (mAcc), reaching 66.2% and 77.0%, respectively. Compared to the classic semantic segmentation method RandLA-Net, mIoU is improved by 3.8% and mAcc by 5.6%; compared to the latest method LSBE-Net, mIoU is improved by 0.1% and mAcc by 3.5%. In terms of overall accuracy (OA), the proposed method achieves 87.3%, slightly lower than BAF-LAC's 87.6% and LSBE-Net's 90.1%. This difference stems from the different design focuses and inherent sensitivities of the evaluation metrics: OA is significantly affected by the majority of categories with a large proportion of points (such as ceiling, floor, and wall), while BAF-LAC and LSBE-Net gain a slight advantage in these large planar regions through reverse attention fusion and semantic boundary enhancement mechanisms, respectively. In contrast, this invention focuses on the collaborative design of three modules: local bilateral augmentation, dual attention regularization, and residual subtraction connection. Its core advantage lies in improving the segmentation accuracy of complex geometric structures and small sample categories, thus demonstrating outstanding performance in the mIoU and mAcc metrics that treat all categories equally, reflecting a reasonable trade-off in design objectives.

[0049] Table 2 shows the comparison of the cross-intersection over union ratio (CUI) of each network across 13 semantic categories on the S3DIS dataset Area5.

[0050] Table 2 Comparison of IoU (%) for 13 categories on the S3DIS dataset Area5 As can be seen from Table 2, the present invention achieves the best performance in the three categories of columns, windows and sofas, which have complex geometric structures and are difficult to segment. In the other categories, it maintains the competitiveness of the first tier, which fully verifies the significant gain of the collaborative design of the three modules for complex geometric structures and easily confused categories.

[0051] For columnar objects, their slender, vertical geometry and complex local structures at their junctions with walls and floors pose challenges to segmentation. The local bilateral augmentation module of this invention explicitly introduces Euclidean distance and neighborhood coordinate difference as key dimensions for geometric description. Combined with a multi-level feature difference mechanism, this enables the network to keenly perceive the radial attenuation pattern of the column surface and the structural consistency in the vertical direction, significantly enhancing the understanding of the geometric integrity of slender structures. Ultimately, it achieves an IoU of 36.4%, far exceeding the second-place LSBE-Net's 30.7%.

[0052] For windows, as thin-walled planar structures embedded in the wall, the glass portion lacks significant texture and geometric undulations, making it easily confused with the wall surface. The dual-attention regularization module of this invention plays a crucial role here: the spatial attention branch guides the network to focus on the subtle area between the window frame and the wall through coordinated max pooling and average pooling, enhancing contour perception; the channel attention branch dynamically modulates feature responses, suppressing interfering features of large wall areas and amplifying key channels representing material and structural differences, achieving precise localization of low-contrast targets and obtaining an IoU of 62.6%, setting a new best record for this category.

[0053] For sofas, their irregular curved surface shape and high similarity to other categories like chairs increase the difficulty of segmentation. The residual subtraction connection module of this invention actively preserves the continuous geometric variation patterns and local details of the sofa surface by calculating the difference between local rich features and global regularized features. It also performs adaptive compensation through learnable residual paths, significantly improving inter-class discriminative power, ultimately achieving an IoU of 82.0%, a 21.8% improvement over the baseline RandLA-Net and a 5.9% improvement over SCF-Net.

[0054] In summary, this invention systematically solves the common problems of existing point cloud semantic segmentation methods, such as insufficient utilization of geometric and semantic information, insufficient feature discriminative power, and a single fusion method, through the organic synergy of three modules: local bilateral augmentation, dual attention regularization, and residual subtraction connection. Extensive experiments on the S3DIS dataset demonstrate that the method of this invention significantly improves the segmentation accuracy for complex geometric structures, semantically easily confused categories, and small-scale objects while maintaining efficiency in large-scale scene processing, exhibiting outstanding substantive features and significant progress.

[0055] This invention also provides a computer-readable storage medium storing a computer program thereon, the computer program being executed by a processor of steps of a point cloud semantic segmentation method based on bilateral augmentation and attention co-enhancement as described in any of the above embodiments.

[0056] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a point cloud semantic segmentation method based on bilateral augmentation and attention co-enhancement as described in any of the above embodiments.

[0057] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions conceived without inventive effort should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims.

[0058] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A point cloud semantic segmentation method based on bilateral augmentation and attention-based collaborative enhancement, characterized in that, include: S1. Obtain a large-scale point cloud dataset and divide the dataset into training and testing sets; S2. Construct a point cloud semantic segmentation network model based on bilateral augmentation and attention-based collaborative enhancement, referred to as the BAACE-Net model. The BAACE-Net model adopts an encoder-decoder architecture. Each layer of the encoder consists of a bilateral attention residual module and a random sampling layer, which gradually reduces the number of point clouds and increases the feature dimension to extract multi-scale hierarchical features. The decoder consists of a nearest neighbor interpolation upsampling layer and skip connections, which gradually restores deep features to the input point cloud resolution. Finally, a classification head composed of fully connected layers and Dropout layers outputs point-by-point semantic categories. The bilateral attention residual module includes a local feature aggregation module, a local bilateral expansion module, a dual attention regularization module, and a residual subtraction connection module. The local feature aggregation module is used to extract neighborhood features from the input point cloud and output the preliminary aggregated local features. The local bilateral augmentation module is used to receive the initially aggregated local features and enhance the representation of the local features through a dual-path collaborative augmentation mechanism of geometric path and semantic path. The dual-attention regularization module adaptively recalibrates the enhanced local features through the collaborative weighting of spatial attention branches and channel attention branches; and performs bilinear interactive operations on the recalibrated features to capture high-order nonlinear correlations in the spatial and channel dimensions, and outputs globally regularized features. The residual subtraction connection module is used to perform element-wise subtraction between the multi-dimensional enhanced local features and the global regularized features to obtain the difference features. At the same time, a learnable residual compensation path is introduced to dynamically compensate the difference features, and the final feature that integrates discriminative local details and global context information is output. S3. Train the BAACE-Net model using the input training set and optimize the model parameters; S4. Apply the optimized BAACE-Net model to the test set to obtain the semantic category prediction results for each point.

2. The method according to claim 1, characterized in that, The local bilateral augmentation module enhances the representation of the initially aggregated local features through a dual-path collaborative augmentation mechanism of geometric and semantic paths, including: Geometric path based on center point coordinates coordinates of neighboring points Calculate the relative position vector as input. Euclidean distance , The calculation expression is: , The calculation expression is: In the formula, m represents the number of neighboring point coordinates obtained through the K-nearest neighbor algorithm. To express summation; to express the .... and After concatenation, the data is mapped using a multilayer perceptron to generate geometric feature codes. The expression is In the formula, Indicates a connector symbol; Semantic path based on center point features Features of neighboring points Calculate the feature difference as input. ,Will and After concatenation, the dimensions are compressed using a lightweight convolutional network to generate semantic feature encodings. The expression is ; Encoding geometric features With semantic feature encoding After concatenation, the features are adjusted for dimensionality and fused nonlinearly using a multilayer perceptron, then aggregated using max pooling, and finally restored using another multilayer perceptron to output locally enhanced bilateral features. The expression is In the formula, This indicates global max pooling.

3. The method according to claim 1 or 2, characterized in that, The geometric path employs a neighborhood adaptive sampling strategy, and the number of neighborhood samples for the geometric path is less than the number of neighborhood samples for the semantic path.

4. The method according to claim 1, characterized in that, The dual-attention regularization module outputs a global regularized feature process, including: The spatial attention branch enhances the local bilateral features of the input. Features are obtained through channel compression. The expression is Subsequently, global max pooling and average pooling are performed along the feature dimension. The pooling results are concatenated and then passed through a convolutional layer with a sigmoid function to generate spatial attention weights. The expression is In the formula, Indicates global max pooling. Indicates global average pooling; Channel attention branches enhance the local bilateral features of the input. Features are obtained through channel compression. The expression is Subsequently, global max pooling and global average pooling are performed along the spatial dimension. The pooling results are concatenated and then passed through a convolutional layer with a sigmoid function to generate channel attention weights. The expression is ; Spatial attention weights and channel attention weights Local bilateral enhancement features for input Modulation is performed to obtain spatial modulation characteristics. With channel modulation characteristics ;right and Performing outer product operations to achieve bilinear interaction generates a space. The channel joint response matrix is ​​compressed in dimension by a 1×1 convolution and activated by the ReLU function to obtain features. M The expression is Finally, feature M and input local bilateral enhancement features The residuals are joined and summed to output the globally regularized features. The expression is .

5. The method according to claim 1, characterized in that, The residual subtraction connection module enhances the discriminative power of local features and maintains information integrity through the complementary fusion of the subtraction path and the residual compensation path, including: Subtraction path for local bilateral enhancement features With global regularization features Perform element-wise subtraction to obtain differential features, thereby suppressing feature commonality and highlighting discriminative details; The residual path will enhance local bilateral features. With global regularization features The features are concatenated along the feature dimension and then transformed nonlinearly through a 1×1 convolutional layer to obtain adaptive compensation features, which dynamically compensate for the effective information that may be lost during the subtraction operation. The difference features output by the subtraction path and the adaptive compensation features output by the residual path are added and fused element by element, and then the refined local features F are output after activation by the Mish function.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of a point cloud semantic segmentation method based on bilateral augmentation and attention co-enhancement as described in any one of claims 1-5.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a point cloud semantic segmentation method based on bilateral augmentation and attention co-enhancement as described in any one of claims 1-5.

Images