A point cloud semantic segmentation method and system based on selective extended attention

Abstract

This invention relates to the field of large-scale semantic segmentation technology based on deep learning, specifically to a point cloud semantic segmentation method and system based on selective extended attention, used to achieve efficient and high-precision semantic segmentation of large-scale 3D point cloud data. This invention constructs a selective extended attention mechanism composed of global selective attention, local extended attention, and block attention units. Global selective attention combines coarse-grained Top-k and local constraints to efficiently filter globally associated regions; local extended attention expands the range of local block key values ​​and introduces reflection filling to eliminate semantic gaps at window boundaries; block attention units adaptively evaluate the weights of each spatial block and fuse features through pooling compression and a shared multilayer perceptron. The selective extended attention module is integrated into a U-Net shallow encoder, adaptively switching between full attention and selective extended attention computation modes based on the point cloud sequence length.

Description

Technical Field

[0001] This invention relates to the field of large-scale semantic segmentation technology based on deep learning, and more specifically, to a point cloud semantic segmentation method based on selective extended attention. Background Technology

[0002] In recent years, with the rapid development of technologies such as autonomous driving, smart cities, robot navigation, augmented reality, and remote sensing mapping, semantic segmentation of large-scale outdoor LiDAR point clouds has become a core foundational task for 3D environment perception and scene understanding. 3D point cloud data possesses inherent characteristics such as being unstructured, spatially sparse, highly uneven in density, and massive in scale (a single frame typically contains hundreds of thousands to millions of unordered points), posing significant challenges to efficient and high-precision semantic segmentation.

[0003] Early point cloud processing methods primarily relied on projection mapping, voxel discretization, or traditional convolutional neural networks (CNNs), such as PointNet, PointNet++, DGCNN, and the U-Net architecture based on sparse convolution. These methods extracted geometric and texture features layer by layer through local receptive fields, achieving good results in small- to medium-scale scenes or indoor point clouds. However, their limitations became increasingly apparent when facing large-scale, complex outdoor scenes: on the one hand, the local inductive bias of CNNs made it difficult to capture long-range semantic dependencies across regions, and while deep networks could expand the receptive field, the computational graph expanded exponentially, easily leading to memory overflow and inference latency; on the other hand, point basis methods typically relied on farthest point sampling (FPS) and K-nearest neighbors (KNN) for neighborhood aggregation, a process involving a large amount of spatial distance calculation and index lookup, resulting in huge computational overhead under massive point clouds, making it difficult to meet the real-time requirements of in-vehicle or airborne devices.

[0004] To overcome computational bottlenecks and introduce global contextual interaction, researchers introduced the Transformer architecture into point cloud processing. The self-attention mechanism can dynamically model semantic relationships between arbitrary point pairs, significantly improving global feature representation capabilities. However, the standard full attention mechanism has both high time and space complexity. Directly applying this to large-scale point clouds would result in unacceptable computational and storage costs. Therefore, current mainstream solutions (such as SST, FlatFormer, OctFormer, and PointTransformerV3) generally employ space-filling curves (such as Z-order or Hilbert curves) to map 3D point clouds into a one-dimensional ordered sequence, utilizing sequence locality to maintain spatial proximity, and introducing non-overlapping window or sliding window attention mechanisms on this basis. This strategy strictly restricts attention computation to a fixed-size local window, successfully reducing the complexity to... or This significantly improves processing efficiency.

[0005] While window attention mechanisms are highly effective in reducing computational complexity, they still face inherent technical bottlenecks in large-scale point cloud semantic segmentation: First, fixed window partitioning rigidly limits the feature interaction range of shallow networks, making it difficult to model long-range semantic dependencies across windows and easily leading to semantic breaks in large-scale continuous targets; second, non-overlapping window strategies disrupt the local topological continuity of spatially adjacent points, resulting in missing feature interactions and blurred segmentation contours at boundaries; third, direct transfer of Top- Sparse attention mechanisms often overemphasize distant semantically similar regions due to a lack of local constraints, neglecting key local geometric priors, which leads to a decline in segmentation performance. Fourth, existing methods, when fusing global sparse blocks and local window features, mostly rely on static stitching or gating strategies, lacking adaptive evaluation of the contribution of different spatial regions to the context, making it difficult to effectively handle feature conflicts and category confusion in complex scenes. These shortcomings collectively restrict the further balance between computational efficiency and segmentation accuracy in existing methods.

[0006] Furthermore, although recent state-space models (such as Mamba / SSM) have their own advantages... Linear complexity shows potential in global sequence modeling, but its direct application in shallow networks can easily smooth out high-frequency boundary details, and it lacks explicit constraint mechanisms for the spatial locality of point clouds. How to balance global long-range semantic interaction, preservation of local geometric continuity, and adaptive weighted fusion of multi-region features while maintaining low computational complexity and linear memory usage has become a critical technical bottleneck that urgently needs to be overcome in the field of large-scale outdoor point cloud semantic segmentation.

[0007] Therefore, there is an urgent need in this field for a novel attention mechanism and network architecture that can break through the receptive field limitation of traditional window attention based on serialized point cloud representation, effectively solve the problems of boundary semantic discontinuity and local information loss, thereby achieving the optimal balance between computational efficiency and segmentation accuracy, and providing more reliable underlying perception support for autonomous driving and real-world 3D reconstruction. Summary of the Invention

[0008] The technical problem to be solved by this invention is to overcome the shortcomings of existing window attention mechanisms in long-range dependency modeling, boundary semantic continuity and local information preservation, and to provide a computationally efficient and semantically consistent selective extended attention point cloud segmentation method.

[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: a point cloud semantic segmentation method based on selective extended attention, comprising the following steps:

[0010] S1: Acquire large-scale outdoor lidar point cloud data, perform coordinate normalization and intensity feature long-tail truncation and normalization, and map the three-dimensional point cloud to a one-dimensional Hilbert index according to the voxel grid coordinates according to the three-dimensional space filling curve, thereby obtaining a one-dimensional ordered sequence. S2: Input the one-dimensional ordered sequence into a 3D sparse convolutional network to extract point-level embedding features; construct a U-Net type encoder-decoder architecture, where decoders perform upsampling through the inverse mapping index stored in the downsampling stage, and encoders and decoders in the same layer fuse features of the corresponding layer through skip connections; S3: A dynamic selection strategy is adopted based on the length of the feature sequence of the current layer. In the shallow layer stage of the encoder where the number of points is greater than the preset window size, the selective extended attention module is activated; in the deep layer stage of the encoder where the number of points is less than or equal to the preset window size, the standard fully self-attention is adopted. S4: The selective extended attention module sequentially performs global selective attention, local extended attention and block attention unit operations to generate an enhanced point cloud feature representation; S5: The output features of each encoder layer are upsampled and fused with skip connections through the decoder, and finally input into the semantic segmentation head to output the category prediction result of each point.

[0011] Preferably, the dynamic selection strategy in step S3 is as follows: when the length of the current layer feature sequence does not exceed the preset window size, a fully self-attention mechanism is adopted; when the length of the current layer feature sequence exceeds the preset window size, a selective extended attention module is activated to perform feature interaction.

[0012] Preferably, the global selective attention in step S4 includes: dividing the point cloud sequence into equal-sized, non-overlapping spatial blocks; extracting the coarse-grained query vector and key vector of each block through average pooling; and calculating a coarse-grained attention map; for each query block, selecting the block with the highest attention score from the coarse-grained attention map. A key block; introduce locality constraints to force each query block to include its corresponding local block in the key block; In a set composed of key blocks, ensure that local geometric information is not lost; [the following text appears to be incomplete and requires further context: "the set of key blocks, ensure that local geometric information is not lost; the set of key blocks, ... Each key block is mapped back to a fine-grained feature space, and window-level multi-head attention computation is performed.

[0013] Preferably, the locality constraint is implemented as follows: in the coarse-grained attention graph, the diagonal element corresponding to the query block is forcibly set to the maximum value of its row, ensuring... Each selected block must contain a local block, and this local block is denoted as... The final set of selected key blocks is represented as , The splicing function splices along spatial dimensions. For the first Selected blocks.

[0014] Preferably, the local extended attention in step S4 includes: based on the local blocks retained in the global selective attention, sampling partial boundary regions of adjacent preceding and following blocks along the serialization sequence direction; and concatenating the boundary regions of adjacent local blocks with the current local block to construct an extended local representation. , Indicates the first A local block, Indicates the first For adjacent local blocks, a reflection filling strategy is used for the terminal blocks at the beginning and end of the sequence, that is, the missing adjacent boundary regions are symmetrically filled by copying half of the region near or far of the current block.

[0015] Preferably, the block attention unit in step S4 includes: a Top-order block attention unit for the outputs of global selective attention and local extended attention. Key-value blocks are subjected to average pooling and max pooling respectively to compress space and channel dimensions; the pooled features are input into a shared multilayer perceptron mechanism, and the adaptive attention weights of each block are output through the SiLU activation function; the weights are multiplied element-wise with the corresponding key-value blocks to complete the adaptive weighted fusion of features from multiple regions.

[0016] Preferably, the weights of the block attention unit The calculation formula is: ; Where I is the input key-value tensor, Mean is the global average pooling operation, and Max is the global max pooling operation. To share a multilayer perceptron, it specifically consists of two linear layers with a scaling ratio of 4 and an intermediate activation layer. Using the SiLU activation function, the fused key values ​​are represented as follows: ; ; in, and These are the final selected key block set and value block set, respectively. and These are the keys and values ​​after weighting by the block attention unit, respectively. This indicates element-wise multiplication.

[0017] Preferably, the sparse convolutional network in step S2 uses a 3D sparse convolution with a kernel size of 5. The encoder contains multiple downsampling stages, the decoder structure is symmetrical with the encoder, and multi-scale features of corresponding levels are fused through skip connections.

[0018] A point cloud semantic segmentation system based on selective extended attention includes: a data serialization module for mapping input LiDAR point clouds into a one-dimensional sequence via spatial filling curves; a sparse feature extraction module for extracting local geometric and semantic features of the point cloud; a selective extended attention interaction module for performing global selective attention global selective filtering, local extended attention local boundary expansion, and adaptive weighted fusion of block attention units; a dynamic scheduling module for adaptively switching between full attention and selective extended attention computation paths based on sequence length; and a decoding and segmentation module for upsampling feature fusion and outputting point-level semantic segmentation results.

[0019] Compared with the prior art, the present invention has the following beneficial effects: (1) through coarse-grained top-level global selective attention. Filtering and locality constraints effectively capture global long-range semantic dependencies while maintaining low computational overhead, and avoid the pitfalls of traditional Top-order filtering. (1) Attention loses local geometric details; (2) By expanding the boundary of local attention and the reflection filling mechanism, the hard boundary of non-overlapping windows is broken, and smooth feature interaction between adjacent spatial blocks is achieved, significantly improving the semantic continuity at the window boundary; (3) By using the adaptive weighted fusion strategy of block attention units, the importance of different spatial regions is dynamically evaluated, avoiding feature conflicts caused by simple splicing or gated fusion, and improving the class discrimination in complex scenes; (4) The overall architecture is compatible with U-Net multi-scale feature flow, the shallow layer activates the selective expansion attention module on demand, and the deep layer maintains full attention, achieving the optimal balance between computational efficiency and segmentation accuracy, and achieving leading performance on large-scale outdoor datasets such as nuScenes and DALES. Attached Figure Description

[0020] Figure 1 This is a diagram showing the overall network architecture of the Selective Extended Attention-Net and the internal structure of the Selective Extended Attention module of this invention. Figure 2 A heatmap comparison of similarity between PTv3 and selective extended attention; Figure 3 A schematic diagram comparing the effective receptive fields of PTv3 and selectively expanded attention; Figure 4 This is a visualization comparison of the segmentation performance of this invention on the nuScenes dataset; Figure 5 This is a visual comparison of the segmentation performance of this invention on the DALES dataset. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0022] Example like Figures 1 to 4 As shown, this embodiment provides a large-scale outdoor LiDAR point cloud semantic segmentation method based on Selective ExtendAttention. Its overall process covers core steps such as data preprocessing, spatial serialization, sparse feature extraction, dynamic route activation, three-level attention interaction of Selective ExtendAttention, multi-scale decoding and reconstruction, and supervised training. The specific implementation is as follows: (1) Point cloud data acquisition and standardized preprocessing; Acquire raw point cloud data from vehicle-mounted or airborne LiDAR sensors. ,in Points per frame (typical outdoor scene) C represents the feature dimension (containing at least three-dimensional coordinates). With reflection intensity value (Optional extensions include normal vectors, echo counts, or timestamps). Perform the following preprocessing operations on the raw data: Coordinate normalization: Mapping the scene bounding box to... Standard cubes eliminate dimensional differences between different data acquisition devices and geographical scales; Intensity standardization: The reflection intensity is normalized using a zero-mean, unit-variance (Z-score). Standardize: ; Data augmentation: Random spatial transformations, including rotations around the Z-axis, are applied during the training phase. Isotropic scaling Horizontal / vertical flip (0.5 probability each), random drop (drop rate) ) and Gaussian position jitter ( (noise=0.01). The enhanced data is uniformly converted into .npy or .pth tensor format as network input.

[0023] (2) Space-filling curve serialization and sparse feature extraction; To transform unstructured 3D point clouds into a one-dimensional ordered representation suitable for sequence modeling, this embodiment employs Space-Filling Curve (SFC) for serialization encoding. Preferably, a Z-order curve (Morton Code) or a Hilbert curve is selected, whose inverse mapping function... To map 3D discrete mesh coordinates to a 1D sequence index: Assume a meshing step size g = 0.02m (aligned with the voxelization resolution), and discrete coordinates are... Then the sequence index ,in For bitwise OR operation, For left shift operator, To reserve the number of bits for batch encoding. The sequence is sorted to obtain... To ensure the continuity of spatially neighboring points in the sequence, the serialized point cloud is input into a 3D sparse convolutional network (SpConv) with a kernel size of 5 to extract point-level embedding features. ( D (This refers to the number of feature channels). The network adopts a U-Net encoder-decoder architecture. The encoder contains 5 stages (Stage 0 to Stage 4). Each stage uses sparse convolutions with a stride of 2 to halve the spatial size and double the number of channels (typical channel configuration: 32). 64 128 256 512).

[0024] (3) Dynamic attention routing mechanism; To balance computational efficiency and feature representation capability, dynamic computation path selection is set in the shallow layers of the encoder (Stage 1~Stage 2): a preset window threshold is used. W =1024. When the length of the current layer feature sequence... L ≤ W When, the standard full attention mechanism is used; when L > W When the feature map is large, the selective extended attention module proposed in this invention is automatically activated. This routing strategy ensures that full attention is avoided when the feature map is large. It exhibits explosive complexity while retaining global context modeling capabilities even when deep features are relatively small.

[0025] (4) Core implementation of selectively extended attention module; like Figure 1As shown, the selective extended attention module consists of three cascaded components: global selective attention, local extended attention, and patch attention units. This module aims to overcome the receptive field limitations of fixed non-overlapping windows, solve the semantic fragmentation problem at window boundaries, and achieve adaptive weighted fusion of features from multiple regions. The specific implementation steps are as follows: (4.1) Global Selective Attention Module; The global selective attention module is used to efficiently model long-range semantic dependencies in shallow, high-resolution feature maps, avoiding the quadratic computational complexity of the full attention mechanism. Its specific computational process includes: 1. Non-overlapping block segmentation and coarse-grained feature extraction: The serialized feature sequence... Divided into Each equal-sized, non-overlapping spatial patch, patch size The optimal value is 128. Average pooling is performed on the features within each block to extract a coarse-grained query vector. With key vector 2. Coarse-grained attention graph construction and Top- Filtering: Calculating coarse-grained attention maps For the first A query block, in The Select the row with the highest attention score Key block index (preferred) =8), denoted as the candidate set. 3. Locality Constraint Injection: For direct application of Top-level constraints... Since attention tends to overlook local geometric priors, this invention introduces a forced locality constraint: the diagonal element of the coarse-grained attention map is explicitly set to the maximum value of that row, i.e. This operation is implemented through hardware-friendly matrix masking or index reordering, without introducing additional trainable parameters, ensuring that each query block always includes its corresponding local block. 4. Fine-grained feature interaction: Incorporate the selected set; Each block is mapped back to a fine-grained feature space, constructing a Top- Key-value sets: ; ; Then, window-level multi-head attention calculation is performed. This design frees query blocks from spatial proximity constraints, enabling global retrieval of highly relevant semantic regions. Simultaneously, it preserves key fine-grained geometric details through local constraints, effectively solving the problems inherent in traditional Top-... The problem that the mechanism's performance degrades instead of improving in point cloud tasks.

[0026] It is particularly important to emphasize that local constraint is one of the key technical contributions of this invention. Ablation experiments quantitatively confirm that directly applying the original Top- When attention (i.e., without locality constraints) is applied to point cloud semantic segmentation, model A1 shows no improvement in mIoU compared to baseline PTv3 (both are 80.3%), verifying the severe damage to segmentation performance caused by the loss of local information. However, after introducing locality constraints, model A2's mIoU improves to 80.7%, an increase of 0.4 percentage points, fully demonstrating that this constraint is the most effective way to improve top-level semantic segmentation. The core step in making sparse attention effective in point cloud tasks lies in the fact that the geometric structure of a point cloud is essentially defined by local spatial neighborhood relationships. Relying solely on semantic similarity for long-distance block selection would cause query points to lose their awareness of the geometric details of their own neighborhoods. Locality constraints, by forcibly preserving local blocks, achieve an effective balance between retrieving globally semantically relevant regions and preserving local geometric priors. This constraint is implemented through a hardware-friendly matrix diagonal element overwrite operation. It does not introduce any additional trainable parameters, and the computational cost is negligible.

[0027] (4.2) Locally extended attention module; While global selective attention expands the effective receptive field, Top- Filtering compresses the interaction scale of local blocks, and non-overlapping partitioning means that window boundary points can only focus on one side of the neighborhood. The local extended attention module enhances local semantic continuity through a cross-block boundary extension mechanism, as implemented below: 1. Sampling of adjacent local block boundary regions: using local blocks retained in global selective attention. Centered on the sequence, the boundary regions of adjacent blocks before and after it are truncated along the direction of the serialized sequence. and Experimental verification shows that setting the sampling ratio to half the block length (i.e., (Each point) can achieve an optimal balance between receptive field expansion and computational overhead; 2. Extended Local Representation Construction: Concatenate adjacent boundary regions with the current local block to construct an extended local representation. ; ; 3. Reflection Padding Strategy: For the terminal blocks at the beginning and end of the sequence (i.e., or The missing adjacent boundary areas are filled using a symmetrical copying strategy. Specifically, the missing left side of the first block is filled by copying its right half. The missing area on the right side of the tail block is filled by copying its left half. This strategy avoids introducing boundary artifacts caused by zero-filling or random sampling. 4. Alternative computing and cross-block interaction: [This will...] and Replace the original and Participate in Top- The collection is constructed, and the updated collection is represented as follows: ; This design maintains non-overlapping partitions at the query level to control complexity, while achieving overlapping coverage at the key-value level. This allows boundary points to simultaneously consider their left and right neighbors, significantly eliminating PCA feature breaks and semantic discontinuities caused by window partitioning.

[0028] This reflection-filling strategy offers significant advantages over traditional zero-filling or random sampling: zero-filling introduces zero-value perturbations at block boundaries that are unrelated to the extrapolation context, leading to spurious low-relevance regions in attention calculations and interfering with subsequent feature aggregation; random sampling, on the other hand, disrupts the integrity of local geometric topology and introduces uncontrollable noise. Reflection-filling, by symmetrically replicating the proximal features of the current block, maintains a high degree of consistency between the filled region and the current local block in terms of geometric distribution and semantic features, effectively avoiding the introduction of boundary artifacts. Furthermore, while achieving overlapping coverage at the key-value level, local extended attention maintains non-overlapping partitioning at the query level—this asymmetric design enables natural information interaction between adjacent local blocks during attention calculations: queries for each block can focus on key-value points from adjacent local blocks, thereby breaking the rigid window boundaries that disrupt semantic continuity. The PCA feature visualization experiment provides intuitive verification: the building roof features of the baseline model show obvious pseudo-break lines at the window boundary. This phenomenon is due to the non-overlapping window blocking the interaction of features between adjacent points. Introducing only global selective attention cannot effectively eliminate this break. However, global selective attention + local extended attention greatly reduces the boundary break. The full selective extended attention almost completely eliminates this artifact, which fully demonstrates the core contribution of local extended attention in breaking the window boundary restrictions and enhancing the semantic continuity across windows.

[0029] (4.3) Block Attention Unit Module; Global selective attention and local extended attention select multi-region key-value blocks from different spatial scales and semantic contexts. Direct concatenation or static summation can easily lead to feature conflicts. The block attention unit, as a lightweight spatial mixer, achieves adaptive importance evaluation and weighted fusion. The specific process includes: 1. Spatial-channel dual-path compression: for each key-value block ( For the current number of blocks, perform average pooling and max pooling operations respectively to compress the spatial dimension while preserving distribution statistics: ; ; 2. Shared Multilayer Perceptron Weight Prediction: Pooled features are input into a shared multilayer perceptron with a hidden layer dimension equal to the number of channels, d. Activated by SiLU function Output scalar attention weights: ; in The introduction of the SiLU activation function gives the gating nonlinear and sparsity characteristics, which is superior to the Sigmoid or no activation function configuration; 3. Adaptive weighted fusion: Perform element-wise multiplication between the weights and the corresponding key-value blocks to achieve dynamic reweighting. ; ; Preferred Implementation Description: The block attention unit is explicitly deployed within the attention computation mechanism, rather than in the position encoding layer or feedforward network (FFN). Ablation experiments confirm that placing it in the PE or FFN compresses spatial information into channel representation, resulting in the loss of local topology; while embedding it in the attention layer allows for independent evaluation of the spatial importance of each patch, and subsequent linear layers automatically perform channel mixing, achieving a balance between structural fidelity and computational efficiency.

[0030] Regarding the placement strategy of block attention units as spatial mixers, this invention conducted detailed comparative ablation experiments, evaluating three configuration schemes: placing the mixer in the position encoding layer (as a channel mixer), placing it in the feedforward network layer (as a channel mixer), and placing it inside the attention mechanism (as a spatial mixer, i.e., the block attention unit scheme of this invention). Quantitative results show that the block attention unit scheme placed inside the attention mechanism achieves the best mIoU (81.4%), while placing it in the position encoding layer and the feedforward network layer reduces it to 81.0% and 80.5%, respectively. The fundamental reason is that the channel mixers in the position encoding and feedforward networks compress the spatial information of each block into a single representation vector along the channel dimension, inevitably losing the details of the local topological structure and the statistical features of the spatial distribution within the block; while the block attention unit independently processes the spatial information of each block, first performing pooling compression along the spatial dimension and then generating scalar weights through a shared multilayer perceptron, preserving richer local geometric prior information. Subsequent linear layers automatically complete the mixing and recombination of information along the channel dimension after multi-head attention output, achieving a balance between structural fidelity and computational efficiency. Furthermore, regarding the selection of activation functions in block attention units, this invention systematically compared three schemes: no activation function (linear gating), Sigmoid activation, and SiLU activation. Experimental results show that SiLU achieves the best performance (81.4% mIoU), followed by Sigmoid (81.3%), and no activation function achieves the lowest performance (81.2%). SiLU function The smooth, non-monotonic nature of the gating mechanism gives it stronger nonlinear expressive power and gradient flow efficiency, which is conducive to the model adaptively learning differentiated attention weights based on the contextual semantic contribution of each block, thereby achieving effective suppression of irrelevant background and enhanced response to the target region.

[0031] (4.4) Cascade fusion strategy, computational complexity analysis and parameter configuration; 1. Key / Value-Level Fusion Strategy: Unlike existing methods that calculate attention independently in each branch and then concatenate / sum / gated the output, this invention directly performs key / value-level fusion after weighting the block attention units. and By concatenating along the sequence dimension, only one multi-head attention computation is performed. This strategy reduces inference latency by approximately 10.9% and improves mIoU by 0.2%~0.4% while maintaining the same number of parameters (46.2M), completely avoiding multi-branch computation redundancy and feature alignment deviation. 2. Computational Complexity Analysis: The complexity of full attention is... The sliding window is Non-overlapping windows are The total complexity of the selectively extended attention module in this invention is approximately: ; The first item is the coarse-grained attention map calculation, and the second item is the fine-grained Top-... Interaction. Because and The overall complexity is relatively small, and increases approximately linearly. 3. Dynamic parameter balancing mechanism: Top- Number of selections Controlling the global-local attention allocation ratio. To maintain a fixed computational budget, this invention constrains... (Window size, preferably 1024). When As the number of patches increases, the decrease in the number of patches leads to fragmentation of region-aware processing and an increase in video memory usage; when When the value decreases, the flexibility of attention allocation declines. Experimental verification. It reaches the performance inflection point (mIoU 81.4%), which is the optimal engineering configuration.

[0032] To systematically verify the independent contributions and synergistic effects of each module in the selective attention extension, this invention designs a rigorous progressive ablation experimental scheme. Using PTv3 as the baseline model (A0, mIoU=80.3%), the ablation is constructed sequentially according to the progressive logic of module stacking: the A1 model directly introduces Top- Attention (without locality constraints) maintains an mIoU of 80.3%, which conversely reveals the offsetting effect of local information loss on performance; the A2 model performs well in Top- Based on this, local constraints are applied to form global selective attention, improving mIoU to 80.7% (+0.4 percentage points), positively verifying the key enabling role of local constraints in sparse attention of point clouds; the A3 model further introduces local extended attention to expand the boundaries of local blocks, improving mIoU to 81.1% (+0.4 percentage points), confirming the effectiveness of cross-block boundary interaction in enhancing semantic continuity; the A4 model, which is the complete selective extended attention module (global selective attention + local extended attention + block attention unit), achieves mIoU of 81.4% (+0.3 percentage points), verifying the effectiveness of the block attention unit adaptive weighted fusion mechanism in the integration of complex multi-region features. It is worth noting that stages A1 to A3 only introduce lightweight non-parametric operations such as index lookup, block boundary concatenation, and pooling, and the model size is exactly the same as the baseline (46.2M), without introducing any additional trainable parameters; the block attention unit introduced in stage A4 only contains an extremely lightweight shared multilayer perceptron with a hidden layer dimension of d, and the overall parameter increment is negligible with millions of parameters, achieving significant performance gains at the cost of extremely low parameters.

[0033] Those skilled in the art will understand that the coarse-grained pooling method for global selective attention described above can be replaced by max pooling or adaptive pooling; the boundary sampling ratio for locally extended attention can be dynamically adjusted according to the point cloud density distribution (preferred range). (Block length); the shared multilayer perceptron hidden layer dimension of the block attention unit can be... Equivalent substitutions within the scope of this invention. The cascading order of the above modules, parameter fine-tuning, or combinations with other global modeling operators (such as Mamba, GCN, sparse convolution) are all equivalent technical substitutions within the scope of protection of the claims of this invention.

[0034] (5) Multi-scale decoding reconstruction and supervised training; The model training uses multi-class cross-entropy loss as the main supervision signal: ; in For one-hot real labels, Predict probabilities using Softmax. Optional addition of label smoothing. =0.1) to alleviate class imbalance. The optimizer uses AdamW with an initial learning rate of 0.1. Weight decay This was achieved using a cosine annealing strategy (50 / 400 epochs). Training was performed on dual-GPU servers (NVIDIA RTX4090, 24GB×2), with nuScenes set to batchsize=8 for 50 epochs and DALES set to batchsize=12 for 400 epochs. In each round, the validation set was used to monitor mIoU and save the optimal weights.

[0035] (6) Experimental verification, performance indicators and complexity analysis; This embodiment is fully validated on the nuScenes and DALES datasets. Experimental results on nuScenes are shown in Table 1. Quantitative results show that the proposed method achieves 81.4% mIoU on the nuScenes validation set, a 1.1% improvement over the baseline PTv3, with significant advantages, especially in easily confused categories such as construction (+7.7%), truck (+3.0%), and car (+2.7%). Experimental results on DALES are shown in Table 2. The method achieves 83.6% mIoU on the DALES validation set, showing excellent performance in the truck (+5.0%) and fences (+0.5%) categories. Qualitative visualization (PCA feature distribution and attention heatmap) confirms that global selective attention effectively expands the effective receptive field (ERF), local extended attention eliminates window boundary artifacts, and block attention units dynamically suppress irrelevant backgrounds (such as the ground) and enhance the response of the target region. In terms of complexity, the full attention method achieves a [missing value]. Non-overlapping windows are The overall computational complexity of the selectively extended attention module in this invention is approximately: ,in =128 and =8, with a measured single-frame inference memory usage of approximately 6.1GB. The experimental environment configuration used in this invention is shown in Table 3.

[0036] Table 1: Comparison of the method of this invention with other methods on the nuScenes dataset.

[0037] Table 2: Comparison of the method of this invention with other methods on the Dales dataset

[0038] Table 3: Experimental Environment Configuration of the Invention

[0039] In terms of qualitative visualization analysis, this invention reveals the mechanism of selective extended attention through multi-dimensional feature visualization. Similarity heatmap analysis shows that regions with high feature similarity to the query point in the baseline model are basically limited to spatial proximity, and some highly similar neighborhood points actually belong to different targets (such as local ground areas near the building query point also showing high similarity), reflecting the semantic discrimination limitations of local window attention. After introducing global selective attention and local extended attention, high similarity regions are effectively propagated to a wider spatial range, the receptive field is significantly expanded, and the similarity of local regions is still maintained. After further introducing block attention units, the similarity distribution is more refined: the similarity of irrelevant backgrounds such as the ground is significantly suppressed, while the similarity of other structures belonging to the same building as the query point is enhanced, indicating that block attention units can effectively evaluate the differences in the contextual contribution of different spatial blocks to the current query and dynamically adjust the weights accordingly. Visualization of the Effective Receptive Field (ERF) further confirms that the ERF of selectively expanded attention significantly expands compared to the baseline. The block attention unit dynamically adjusts the attention weights of each block based on the semantic category of the query point—for example, when the query point is located in a vegetation area, vegetation-related global selection blocks receive relatively higher attention weights, verifying the category-aware adaptive weighting capability of the block attention unit. The aforementioned multi-dimensional visualization evidence collectively demonstrates that selectively expanded attention, by allocating attention to distant points semantically similar to the query point, promotes effective information exchange between the query point and similar targets, thereby significantly improving the discrimination accuracy for easily confused categories (such as construction, truck, and car in nuScenes, and truck and fences in DALES).

[0040] (7) Engineering deployment and equivalent parameter replacement; The method described in this embodiment can be executed by a software program on a general-purpose processor (CPU / GPU / TPU), or it can be embedded in an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or an edge computing module (such as NVIDIA Jetson AGXOrin, Huawei Ascend 310). The computer program can be stored in a computer-readable storage medium such as ROM, RAM, SSD, optical disc, or cloud server, and executes the steps described in any embodiment of the present invention when invoked by a processor. Those skilled in the art will understand that the above hyperparameters (such as block size)... ∈[64,256]、Top- value ∈[4,16], window threshold The parameters (∈[512,2048], space-filling curve type, pooling operator replacement with max / adaptive / learnable pooling, etc.) can all be reasonably adjusted according to actual hardware computing power, dataset density distribution, and task accuracy requirements. Selectively extended attention modules can also seamlessly replace corresponding components in existing window attention architectures, or be cascaded with other global modeling mechanisms (such as Mamba / SSM, Graph Attention Network GCN, and multi-scale dilated convolution). Their combinations, parameter fine-tuning, or module reorganization schemes all fall within the protection scope of this invention.

[0041] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, parameter adjustments, or module reorganizations made within the spirit and principles of the present invention should be included within the scope of protection of the present invention. Technical details, data flow formats, or underlying operator implementations not described in detail in the present invention can be directly implemented using common knowledge in the art, open-source deep learning frameworks (such as PyTorch, TensorFlow, PaddlePaddle), or existing hardware acceleration libraries, and do not constitute a limitation on the substantive content of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A point cloud semantic segmentation method based on selective extended attention, characterized in that, Includes the following steps: S1: Acquire large-scale outdoor lidar point cloud data, perform coordinate normalization and intensity feature long-tail truncation and normalization, and map the three-dimensional point cloud to a one-dimensional Hilbert index according to the voxel grid coordinates according to the three-dimensional space filling curve, thereby obtaining a one-dimensional ordered sequence. S2: Input the one-dimensional ordered sequence into a 3D sparse convolutional network to extract point-level embedding features; A U-Net-type encoder-decoder architecture is constructed, where decoders perform upsampling through the inverse mapping index stored in the downsampling stage, and encoders and decoders in the same layer fuse features of the corresponding layer through skip connections; S3: Based on the length of the current layer feature sequence, a dynamic selection strategy is adopted to activate the selective extended attention module in the shallow layer stage of the encoder where the number of points is greater than the preset window size. In the deep stages of the encoder where the number of points is less than or equal to the preset window size, standard fully self-attention is used; S4: The selective extended attention module sequentially performs global selective attention, local extended attention, and block attention unit operations to generate an enhanced point cloud feature representation; wherein, the global selective attention in step S4 includes: dividing the point cloud sequence into equal-sized, non-overlapping spatial blocks, extracting the coarse-grained query vector and key vector of each block through average pooling, and calculating a coarse-grained attention map; for each query block, selecting the one with the highest attention score in the coarse-grained attention map. A key block; introduce locality constraints to force each query block to include its corresponding local block in the key block; In a set composed of key blocks, ensure that local geometric information is not lost; [the following text appears to be incomplete and requires further context: "the set of key blocks, ensure that local geometric information is not lost; the set of key blocks, ... Each key block is mapped back to the fine-grained feature space, and window-level multi-head attention computation is performed. The specific implementation of the locality constraint is as follows: in the coarse-grained attention graph, the diagonal element corresponding to the query block is forcibly set to the maximum value of its row, ensuring... A selected block contains a local block, and this local block is denoted as... The final set of selected key blocks is represented as , The splicing function splices along spatial dimensions. For the first Selected blocks; the local extended attention includes: based on the local blocks retained in the global selective attention, sampling partial boundary regions of adjacent preceding and following blocks along the serialization sequence direction; concatenating the boundary regions of adjacent local blocks with the current local block to construct an extended local representation. , Indicates the first A local block, Indicates the first For adjacent local blocks, a reflection filling strategy is used for the terminal blocks at the beginning and end of the sequence, that is, the missing adjacent boundary regions are symmetrically filled by copying half of the region near or far of the current block. S5: The output features of each encoder layer are upsampled and fused with skip connections through the decoder, and finally input into the semantic segmentation head to output the category prediction result of each point.

2. The point cloud semantic segmentation method based on selective extended attention according to claim 1, characterized in that, The dynamic selection strategy described in step S3 is as follows: when the length of the current layer feature sequence does not exceed the preset window size, a fully self-attention mechanism is adopted; when the length of the current layer feature sequence exceeds the preset window size, a selective extended attention module is activated to perform feature interaction.

3. The point cloud semantic segmentation method based on selective extended attention according to claim 1, characterized in that, The block attention unit mentioned in step S4 includes: a Top-order block attention unit for the outputs of global selective attention and local extended attention. Key-value blocks are subjected to average pooling and max pooling respectively to compress space and channel dimensions; the pooled features are input into a shared multilayer perceptron, and the adaptive attention weights of each block are output through the SiLU activation function; the weights are multiplied element-wise with the corresponding key-value blocks to complete the adaptive weighted fusion of multi-region features.

4. The point cloud semantic segmentation method based on selective extended attention according to claim 3, characterized in that, Weights of block attention units The calculation formula is: Where I is the input key-value tensor, Mean is the global average pooling operation, and Max is the global max pooling operation. To share a multilayer perceptron, it specifically consists of two linear layers with a scaling ratio of 4 and an intermediate activation layer. The SiLU activation function is used, and the fused bond values ​​are represented as follows: in, and These are the final selected key block set and value block set, respectively. and These are the keys and values ​​after weighting by the block attention unit, respectively. This indicates element-wise multiplication.

5. The point cloud semantic segmentation method based on selective extended attention according to claim 1, characterized in that, The sparse convolutional network described in step S2 uses 3D sparse convolution with a kernel size of 5. The encoder contains multiple downsampling stages, the decoder structure is symmetrical with the encoder, and multi-scale features of corresponding levels are fused through skip connections.

6. A point cloud semantic segmentation system based on selective extended attention, used to perform the point cloud semantic segmentation method based on selective extended attention as described in any one of claims 1-5, characterized in that, include: The data serialization module is used to map the input LiDAR point cloud into a one-dimensional sequence through a space-filling curve; The sparse feature extraction module is used to extract local geometric and semantic features of point clouds; the selective extended attention interaction module is used to perform global selective attention global selective filtering, local extended attention local boundary expansion, and block attention unit adaptive weighted fusion. The dynamic scheduling module is used to adaptively switch between full attention and selective extended attention computation paths based on the sequence length; the decoding and segmentation module is used to fuse upsampled features and output point-level semantic segmentation results.

Images