A three-dimensional point cloud semantic segmentation method and system based on adaptive sequence window routing

Abstract

The application discloses a three-dimensional point cloud semantic segmentation method and system based on an adaptive sequence window routing, and the method comprises the following steps: acquiring three-dimensional point cloud data to be segmented, performing spatial quantization and spatial serialization on the point cloud data, generating sequence order, reverse sequence index and batch offset, and obtaining serialized features; constructing a segmentation network comprising a serialized point cloud Transformer encoder and a decoder; inputting the serialized features into the encoder, using an adaptive sequence window routing module in the encoder to perform multi-scale context enhancement and fusion on the features, and obtaining enhanced features; inputting the enhanced features into the decoder for up-sampling recovery, and outputting the semantic category of each point through a point-level prediction head. The application guides the adaptive sequence window routing through point cloud serialization, dynamically fuses multi-scale context features, effectively solves the problem that local details and global semantics are difficult to be considered at the same time while reducing the computational complexity, and significantly improves the precision and efficiency of three-dimensional point cloud semantic segmentation.

Description

Technical Field

[0001] This invention relates to the fields of 3D point cloud data processing and computer vision technology, and more specifically to a 3D point cloud semantic segmentation method and system based on adaptive sequence window routing. Background Technology

[0002] 3D point clouds, as an important data form for describing the geometry of 3D space, are widely used in fields such as autonomous driving, robot navigation, augmented reality, and 3D reconstruction. Semantic segmentation, as a core task of point cloud understanding, aims to assign a specific semantic label to each point in the point cloud. With the development of deep learning, point-based methods (such as the PointNet series) and graph-based methods (such as DGCNN) have gradually become mainstream. In order to capture richer contextual information, in recent years, Transformer-based architectures have been introduced into point cloud processing, using self-attention mechanisms to model long-distance dependencies between points.

[0003] However, existing 3D point cloud semantic segmentation technologies still have the following major drawbacks in practical applications:

[0004] 1. High computational complexity, making it difficult to handle large-scale scenes: The computational complexity of the standard self-attention mechanism increases quadratically with the number of point clouds. For indoor scenes containing tens of thousands of points or outdoor LiDAR scenes containing hundreds of thousands of points, directly applying the global attention mechanism will lead to memory overflow or extremely slow inference speed, making it difficult to meet real-time requirements.

[0005] 2. The dilemma of balancing local and global features: To reduce computational cost, existing technologies often employ local window attention mechanisms, limiting computation to the local neighborhood. While this reduces complexity, it limits the receptive field, making it difficult for the model to capture long-distance global contextual information and prone to misjudgments in semantically ambiguous regions.

[0006] 3. Lack of adaptive multi-scale fusion mechanism: Existing multi-scale feature extraction methods are usually static or based on predefined rules (such as fixed-radius sphere queries), and cannot dynamically adjust the attention to features at different scales according to the specific content of the input point cloud. For example, a larger receptive field may be needed in flat areas, while more refined local features are needed in edge areas. Existing technologies lack this content-aware dynamic routing capability.

[0007] Therefore, proposing a 3D point cloud semantic segmentation method that can balance computational efficiency and feature representation capability, and can adaptively fuse multi-scale contextual information, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] In view of the above problems, this invention is proposed to provide a 3D point cloud semantic segmentation method and system based on adaptive sequence window routing to overcome or at least partially solve the above problems. By serializing the point cloud and using batch offsets and reverse indexes to guide the adaptive sequence window routing module, multi-scale contextual features are dynamically fused. This effectively solves the problem of difficulty in balancing local details and global semantics while reducing computational complexity, and significantly improves the accuracy and efficiency of 3D point cloud semantic segmentation.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, embodiments of the present invention provide a 3D point cloud semantic segmentation method based on adaptive sequence window routing, comprising the following steps: S1. Obtain the three-dimensional point cloud data to be segmented, wherein the point cloud data includes the three-dimensional coordinates and point features of discrete points; S2. Spatial quantization and spatial serialization are performed on the point cloud data to generate sequence order, reverse order index and batch offset, and serialization features are obtained. S3. Construct a segmentation network that includes a serialized point cloud Transformer encoder and decoder; input the serialized features into the encoder, and use the adaptive sequence window routing module in the encoder to perform multi-scale context enhancement and fusion on the features to obtain enhanced features; S4. Input the enhanced features into the decoder for upsampling recovery, and output the semantic category of each point through the point-level prediction head.

[0010] Further, step S1 specifically includes: Raw 3D point cloud data is acquired by LiDAR, depth camera or 3D scanner. The raw 3D point cloud data includes the 3D coordinates, reflection intensity and color information of discrete points. The original 3D point cloud data is preprocessed, including removing invalid and duplicate points, voxel filtering downsampling, and removing discrete noise points. The preprocessed point cloud data is used as the three-dimensional point cloud data to be segmented, and the three-dimensional coordinates and point features of each discrete point are extracted. The point features include at least one of reflection intensity, color information and normal vector.

[0011] Further, step S2 specifically includes: Normalize and map the 3D coordinates of each discrete point to the grid, converting the physical coordinates into grid coordinates; Based on the grid coordinates, a spatial sorting code is calculated using a space-filling curve. All points are then sorted in ascending order according to the spatial sorting code to generate a sequence order. The original position of each point before sorting is recorded to generate a reverse index. For batch input point cloud data, the number of points contained in each sample is counted and accumulated, and the start and end range of each sample in the spliced ​​total sequence is recorded to obtain the batch offset, which is used to distinguish the boundaries of different samples. The serialized point cloud data is mapped to the target dimension feature space through the embedding layer to obtain the final serialized features.

[0012] Furthermore, in step S3, the encoder comprises multiple basic blocks, and the basic blocks include at least a conditional position coding layer, a serialization attention layer, an adaptive sequence window routing module, and a feedforward network; The conditional location coding layer receives the serialized features, maps the sequence order into a learnable location embedding vector, and adds it element-wise to the serialized features to obtain a serialized feature that fuses location information. The serialization attention layer receives serialized features with fused position information, uses the batch offset to segment the features according to the sample boundaries, and independently calculates self-attention within the continuous sequence window of each sample to obtain local context features. The adaptive sequence window routing module receives local context features, constructs fine-grained branches at the original scale and at least one coarse-scale context branch in parallel, generates dynamic routing weights based on the mean of feature responses and geometric statistics within a single sample sequence window, and performs adaptive weighted fusion of features from each branch to obtain multi-scale enhanced features. The feedforward network receives multi-scale enhancement features, performs point-by-point feature mapping through two layers of linear transformation and nonlinear activation functions, and outputs the enhancement features of the current base block.

[0013] Furthermore, the original-scale fine-grained branch maintains the input feature sequence length unchanged and extracts local geometric detail features through lightweight sequence mixing operations; the lightweight sequence mixing operations include layer normalization, first linear mapping, one-dimensional depthwise convolution, activation function, and second linear mapping.

[0014] Furthermore, the coarse-scale context branch performs downsampling pooling on the input feature sequence to expand the receptive field, extracts global context features, and then restores the original sequence length through upsampling; specifically including: The sequence range of the current point cloud sample is determined using the batch offset, and padding is performed at the end of the sequence range so that the sequence length can be divided by the pooling ratio. The sequence is divided into multiple pooling groups according to the pooling ratio, and the pooling groups do not cross the sample boundaries. Pooling operation is performed on the features within each pooling group to obtain a coarse-scale sequence. A lightweight sequence mixing operation is performed on the coarse-scale sequence. Each token in the mixed coarse-scale sequence is restored to the original sequence length by repeating the pooling ratio times, and the padding part is removed to obtain a coarse-scale candidate feature with the same length as the original input sequence.

[0015] Furthermore, the generation of dynamic routing weights based on the mean of feature responses and geometric statistics within a single sample sequence window specifically includes: Window-level feature statistics calculation: Based on the batch offset, the features are isolated according to the sample boundaries. Within the continuous sequence window of each sample, the mean of the feature response, spatial dispersion, spatial coverage, and local density of the points within the window are calculated to generate a statistical vector representing the feature distribution within the window. Dynamic routing weight generation: The statistical vector is input into a lightweight routing network consisting of two fully connected layers. The first fully connected layer reduces the dimensionality and introduces non-linear activation. The second fully connected layer maps the vector to the dimension corresponding to the number of branches. Finally, the vector is normalized by the Softmax function to obtain the dynamic routing weights of each scale branch.

[0016] Furthermore, step S4 specifically includes: The enhanced features are used as the initial input to the decoder, and the resolution is restored by upsampling step by step according to the hierarchical structure corresponding to the downsampling factor of the encoder. After upsampling at each level of the decoder, shallow features of the corresponding level of the encoder are fused through skip connections to supplement local geometric details; The fused features are subjected to nonlinear transformation and channel dimensionality reduction by a multilayer perceptron to obtain decoded features containing rich semantic and location information. The decoded features are input into the point-level prediction head, mapped to the category dimension through a fully connected layer and normalized by Softmax. The category with the highest probability is selected as the semantic label of each point, and the segmentation result is output.

[0017] In a second aspect, embodiments of the present invention provide a three-dimensional point cloud semantic segmentation system based on adaptive sequence window routing, employing the method described in any one of the first aspects, comprising the following modules: Point cloud data acquisition module: used to acquire the three-dimensional point cloud data to be segmented, wherein the point cloud data includes the three-dimensional coordinates and point features of discrete points; Spatial serialization module: used to perform spatial quantization and spatial serialization on the point cloud data, generate sequence order, inversion index and batch offset, and obtain serialization features; Feature enhancement module: used to construct a segmentation network containing a serialized point cloud Transformer encoder and decoder; input the serialized features into the encoder, and use the adaptive sequence window routing module in the encoder to perform multi-scale context enhancement and fusion on the features to obtain enhanced features; Semantic output module: used to input the enhanced features into the decoder for upsampling and recovery, and output the semantic category of each point through the point-level prediction head.

[0018] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a three-dimensional point cloud semantic segmentation method and system based on adaptive sequence window routing, which has the following beneficial effects: First, a balance is achieved between computational efficiency and global receptive field. This invention constructs fine-grained branches at the original scale and coarse-scale context branches in parallel, which significantly reduces computational complexity while effectively capturing long-distance global dependencies, solving the problem that traditional global attention mechanisms cannot handle large-scale point clouds in real time.

[0019] Secondly, it improves the adaptability and discriminative power of feature fusion. This invention introduces a dynamic routing mechanism based on window feature statistics, which dynamically adjusts the weights of multi-scale branches according to local geometric complexity, enabling the model to focus on details at edges and emphasize the global picture in flat areas, significantly enhancing its adaptability to complex scenes.

[0020] Finally, the detail reproduction of the segmentation results is enhanced. This invention employs a symmetrical hierarchical upsampling and cross-hierarchical skip connection strategy to effectively supplement the high-frequency geometric details lost in deep features, significantly improving the clarity of object boundaries and the segmentation accuracy of microstructures, thus meeting the requirements of high-precision 3D perception. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0022] Figure 1 This is a flowchart of the 3D point cloud semantic segmentation method based on adaptive sequence window routing provided in this embodiment of the invention; Figure 2 This is a schematic diagram of the 3D point cloud semantic segmentation method based on adaptive sequence window routing provided in this embodiment of the invention. Figure 3 This is a schematic diagram of the adaptive sequence window routing module structure provided in an embodiment of the present invention; Figure 4This is a schematic diagram of intra-sample independent sequence processing provided in an embodiment of the present invention; Figure 5 This is a block diagram of a 3D point cloud semantic segmentation system based on adaptive sequence window routing provided in an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Example 1 This invention discloses a 3D point cloud semantic segmentation method based on adaptive sequence window routing, referring to... Figure 1 As shown, it includes the following steps: S1. Obtain the 3D point cloud data to be segmented. The point cloud data includes the 3D coordinates and point features of discrete points. S2. Perform spatial quantization and spatial serialization on the point cloud data to generate sequence order, inversion index and batch offset, and obtain serialization features; S3. Construct a segmentation network that includes a Transformer encoder and decoder for serialized point clouds; input the serialized features into the encoder, and use the adaptive sequence window routing module in the encoder to perform multi-scale context enhancement and fusion on the features to obtain enhanced features; S4. Upsample and recover the enhanced features input to the decoder, and output the semantic category of each point through the point-level prediction head.

[0025] This embodiment is applied to the perception and decision-making scenarios of autonomous vehicles at complex urban intersections. When the vehicle enters the intersection, the LiDAR collects real-time 3D point cloud data of the surrounding environment. This data includes geometric information of vehicles, pedestrians, traffic signs, road markings, and surrounding buildings. This embodiment first establishes an input point set based on the 3D coordinates of the points and optional point features; then, spatial quantization, spatial serialization, sparsification, and feature embedding are performed to organize the disordered 3D points into a one-dimensional sequence with local spatial correlation, and the sequence order, inversion index, and offset of different point cloud samples in the batch sequence are recorded simultaneously; then, the sequence features are input into the serialized point cloud Transformer encoder.

[0026] In the encoder's basic block, point-level sequence features are sequentially processed through conditional position encoding, serialized attention, and a feedforward network. In this embodiment, an adaptive sequence window routing module is inserted after serialized attention and before the feedforward network. The adaptive sequence window routing module constructs a fine-grained branch at the original scale and K coarse-scale context branches in parallel within the sequence range of a single point cloud sample. It generates window-level dynamic routing weights based on the feature responses, spatial dispersion, spatial coverage, and local density within the continuous sequence window. Subsequently, the window-level routing weights are broadcast to the point level, and adaptive weighted fusion is performed on candidate features at multiple scales to obtain enhanced point-by-point features. Finally, the semantic category of each point is output through the decoder and the point-level prediction head.

[0027] Unlike voxel window reconstruction-based point cloud segmentation schemes, this embodiment does not reconstruct the point cloud into densely arranged 3D voxel windows, nor does it use multi-scale pooling results as the K and V of the attention mechanism to compress the attention sequence. This embodiment retains the original fixed-point-count window attention form of the serialized point cloud Transformer, only adding a residual adaptive sequence window routing module branch after the attention output, using window-level 3D geometric statistics to drive the dynamic fusion of multi-scale candidate branches. Unlike the multi-scale convolution branch fusion in 2D image semantic segmentation, the routing weights in this embodiment are derived from the continuous intra-sample windows after 3D point cloud spatial serialization, and do not rely on 2D pixel grids, 2D convolution kernels, or skip connection feature maps in the image decoder.

[0028] Reference Figure 2 As shown, Figure 2 The overall data flow and module connection relationships of the 3D point cloud semantic segmentation network in this embodiment are presented. The input point cloud first undergoes spatial serialization, sparsification / quantization, and feature embedding to obtain serialized point features. Then, it enters the encoding stages E0 to E4 for hierarchical feature extraction, and then goes through the decoding stages D3 to D0 corresponding to the encoding stages to restore the point-level resolution step by step, and fuses the shallow features of the same level at the encoding end, and finally outputs the point-by-point semantic prediction results.

[0029] Figure 2 In the diagram, E0, E1, E2, E3, and E4 represent five coding levels, with C=32, 64, 128, 256, and 512 indicating the number of feature channels at the corresponding level. D0, D1, D2, and D3 represent four decoding levels, with the corresponding C value indicating the number of output channels at that decoding level. B-BLOCK represents the standard base block, which internally contains Conditional Position Encoding (CPE), Serialization Attention (SA), and a Feedforward Network (MLP). E-BLOCK represents the enhanced base block, which adds the Adaptive Sequence Window Routing (ASWR) module between SA and MLP. "×n" in the diagram indicates that the corresponding base block is repeated n times consecutively, and FC represents a fully connected mapping layer.

[0030] Figure 2 Solid arrows in the diagram represent forward feature propagation within the same layer; downward connections between adjacent layers in the encoding stage represent sequence pooling or downsampling, while connections between adjacent layers in the decoding stage represent upsampling recovery; dashed connections from the encoding end to the decoding end represent cross-layer skip connections, used to supplement shallow geometric details into the decoded features. The magnified box on the right further compares the compositional differences between B-BLOCK and E-BLOCK, illustrating that ASWR is inserted after sequential attention and before MLP, and participates in feature update through residuals.

[0031] The specific implementation steps of this embodiment are described below: Step S1: Obtain the input point cloud.

[0032] Obtain the 3D point cloud data to be segmented. The point cloud consists of L discrete points, each containing at least three-dimensional coordinates. The input point cloud can be further denoted as tj, and may include color, intensity, normal vector, or other auxiliary features. This step outputs the original point coordinate sequence P and the point feature sequence T, which are used for subsequent spatial serialization and point-level semantic prediction.

[0033] Step S1: Obtain the 3D point cloud data to be segmented.

[0034] In this embodiment, step S1 is specifically used to acquire raw data containing the three-dimensional coordinates and point features of discrete points, and to clean and standardize the data to provide high-quality input for subsequent spatial serialization. The specific implementation process is as follows: 1. Data collection.

[0035] First, raw 3D point cloud data of the scene is acquired using hardware sensors. Sensors in this embodiment include, but are not limited to, LiDAR, depth cameras (such as RGB-D cameras), or high-precision 3D scanners.

[0036] If a lidar is used, the collected data includes the three-dimensional spatial coordinates of each discrete point and reflection intensity information; if a depth camera is used, the collected data includes color information in addition to three-dimensional coordinates. Based on the actual application scenario, the original point cloud data in this embodiment is denoted as... , This represents the original point cloud dataset. Indicates the first The three-dimensional coordinates of a point cloud. Indicates the first Point features of a point cloud, This represents the number of original discrete points.

[0037] 2. Data preprocessing.

[0038] Due to sensor noise and environmental interference, raw point cloud data often contains invalid data and redundant information. To reduce computational complexity and improve segmentation accuracy, this step performs the following preprocessing operations on the raw point cloud: Remove invalid and duplicate points: Traverse the original point cloud and remove invalid points with coordinate values ​​of NaN (not a number) or Inf (infinity); at the same time, detect and merge duplicate points that are completely overlapping in spatial location, keeping only one instance to ensure the uniqueness of the data.

[0039] Voxel filtering downsampling: A voxel grid filter is used to downsample the point cloud. Specifically, the voxel size is set to δ (e.g., 0.05 meters), and the 3D space is divided into several tiny cubic voxels with side length δ. For all points falling within the same voxel, its geometric center or centroid is calculated, and this center point replaces all the original points within the voxel. This operation significantly reduces the number of points while preserving the macroscopic geometry of the point cloud, achieving uniform downsampling.

[0040] Removing discrete noise points: A statistical outlier removal algorithm is used to remove noise points that are outside the main body. Specifically, the average distance from each point to its nearest neighbors is calculated. Assuming the result follows a Gaussian distribution, any point whose average distance exceeds the sum of the global mean and standard deviation is considered a noise point and is removed.

[0041] 3. Feature extraction and output.

[0042] After the above preprocessing, clean 3D point cloud data to be segmented is obtained. At this point, the 3D coordinates and point features of each discrete point are extracted to construct the final input data structure.

[0043] 3D coordinates: Extract the spatial location information of each point, denoted as... , ,in, They represent the first The coordinates of a point cloud on the X, Y, and Z axes. This represents the number of discrete points after preprocessing.

[0044] Point features: Based on the sensor type and task requirements, auxiliary features T are extracted. Point features include at least one of the following: reflection intensity, color information (RGB normalized value), and normal vector.

[0045] Optionally, if the original data lacks normal vectors, the normal vectors can be calculated by fitting local planes through principal component analysis to enhance the model's ability to perceive geometric structures.

[0046] Finally, the 3D point cloud data to be segmented output in step S1 can be represented as follows: , This represents the preprocessed point cloud dataset.

[0047] Step S2 involves spatial quantization, spatial serialization, and feature embedding of the point cloud data.

[0048] In this embodiment, the core objective of step S2 is to convert the disordered, discrete 3D point cloud into a 1D sequence with spatial locality, so that the Transformer network can process it efficiently subsequently. The specific implementation process is as follows: 1. Spatial quantization and mesh mapping.

[0049] In order to reduce computational complexity and introduce spatial structure information, this embodiment first performs spatial quantization on the three-dimensional physical coordinates of each discrete point.

[0050] Determine the baseline and step size: Set the minimum coordinate value of the current point cloud or the current scene to [value]. Use it as the coordinate reference and set v This indicates the quantization step size or voxel scale (e.g., 0.05 meters).

[0051] Calculating mesh coordinates: Spatial quantization is performed on the 3D coordinates of each point, converting the physical coordinates into integer mesh coordinates to obtain the mesh coordinates. ;in, Indicates the first Grid coordinates of discrete points This indicates a floor operation. This step discretizes the continuous physical space into a regular three-dimensional mesh system.

[0052] 2. Spatial serialization based on space-filling curves.

[0053] In order to preserve the neighborhood relationships in three-dimensional space as much as possible in the one-dimensional sequence, this embodiment uses space-filling curves to encode and sort the grid coordinates.

[0054] Calculate the spatial sort code: based on the grid coordinates obtained above. The spatial order code for each point is calculated using a space-filling curve algorithm. In this embodiment, the space-filling curve is preferably a Morton code (Z-order curve) or a Hilbert curve, or its transposed encoding form. The spatial order code can map multidimensional grid coordinates to one-dimensional scalar values, and ensures that adjacent points in space are as close as possible in value.

[0055] Generate sequence order: Sort all points in ascending order according to the calculated spatial sorting key. The sorted point index sequence is the "sequence order". After this step, the originally disordered point cloud is reorganized into a one-dimensional sequence with strong spatial correlation.

[0056] Generate an inverse index: During the sorting process, the original position index of each point before sorting is recorded synchronously, generating an "inverse index". This inverse index is used to accurately map the processed sequence features back to the original point positions in subsequent decoding stages or when it is necessary to restore the original point order.

[0057] 3. Generation of batch offsets.

[0058] To address the batch input pattern commonly encountered in deep learning training, batch offsets need to be generated to distinguish the boundaries of different samples and prevent feature confusion across samples.

[0059] Point count: Count the number of points contained in each point cloud sample in the current batch.

[0060] Accumulation and Recording: The number of points for each sample is accumulated, and the start and end positions of each sample in the concatenated sequence are recorded. These start and end ranges constitute the "batch offset". In subsequent attention calculations or window partitioning, this offset is used to strictly isolate features according to sample boundaries, ensuring that sequence processing is performed only within a single sample.

[0061] 4. Feature embedding.

[0062] Finally, this embodiment maps the serialized data to a high-dimensional feature space through a learnable embedding layer.

[0063] The mapping process is as follows: the sorted point coordinates and their corresponding point features (such as reflection intensity, color, etc.) are input into the embedding layer.

[0064] The output is as follows: The embedding layer maps the low-dimensional input to the target dimension C through a linear transformation, resulting in the final serialized features. Where L is the sequence length, i.e., the number of discrete points after preprocessing; and C is the number of feature channels, i.e., the target dimension. This serialized feature contains rich geometric and semantic information and possesses the structural characteristics of a one-dimensional sequence, which can be directly input into the subsequent Transformer encoder.

[0065] Step S3: Construct a segmentation network; specifically, it includes a serialized point cloud Transformer encoder and decoder. The serialized features are input into the encoder, and the adaptive sequence window routing module in the encoder performs multi-scale contextual enhancement and fusion on the features to obtain enhanced features.

[0066] The serialized point cloud Transformer encoder in this embodiment comprises multiple basic blocks, each of which includes at least a conditional location encoding layer, a serialized attention layer, an adaptive sequence window routing module, and a feedforward network connected in sequence. First, conditional location encoding is used to supplement local geometric location information; then, serialized attention is performed within a fixed-point window according to the spatial sequence order to obtain local context features; the adaptive sequence window routing module constructs fine-grained branches at the original scale and coarse-scale context branches, and adaptively weights and fuses the features of each branch to obtain multi-scale enhanced features; finally, the feedforward network is used to complete the nonlinear transformation of the channel dimension.

[0067] Regarding the conditional position coding layer, it receives serialized features. The sequence order is mapped to a learnable position embedding vector, and then added element-wise to the sequenced features to obtain sequenced features that fuse positional information; specifically, the following steps are included: 1) Construct sparse grid mapping relationships (MapGrid operation).

[0068] Based on the grid coordinates generated in step S2 Establish a mapping relationship from one-dimensional sequence index to three-dimensional sparse grid space.

[0069] Specifically, the MapGrid operation will convert the input one-dimensional serialization features The data is rearranged and mapped into a sparse 3D voxel mesh structure. During this process, the feature of each point is placed at its corresponding mesh coordinates (x, y, z), forming a sparse 3D feature tensor. Empty mesh positions without any points are typically filled with zeros or left undefined.

[0070] 2) Perform sparse 3D convolution (Conv3D operation).

[0071] A lightweight 3D sparse convolutional layer is applied to the constructed sparse 3D mesh.

[0072] The convolutional kernel (e.g., 3×3×3 in size) slides in 3D space, allowing each center point to exchange local information with its neighboring points within its adjacent grid locations. This step effectively captures the local geometry and neighborhood context information of the point cloud, generating grid features that incorporate local geometry awareness.

[0073] 3) Recover point sequence features (MapBack operation).

[0074] The MapBack operation is the inverse process of MapGrid. Based on the previously recorded mapping relationships, it re-extracts the grid features after convolution and maps them back to the original one-dimensional point-level sequence order.

[0075] At this point, the output features not only retain the original semantic information but also incorporate the local geometric location encoding information extracted through 3D convolution, denoted as... The formula is expressed as follows:

[0076]

[0077] in, This represents the set of grid coordinates for all valid points. Indicates basis The operation of mapping point-level sequence features to a sparse 3D mesh. This represents a local 3D sparse convolution operation performed on a sparse grid. This represents the operation of restoring grid features to point-level sequence features based on the original mapping relationship. Indicates by MapGrid , Conv3D and MapBack The conditional positional encoding transformation is formed together.

[0078] 4) Residual connectivity and feature fusion.

[0079] To ensure stable training and preserve the original features, a residual connection method is used to incorporate positional encoding into the features.

[0080] The specific calculation formula is as follows:

[0081] Wherein, Norm represents the normalization layer, used to adjust the feature distribution; Linear represents the linear projection layer, used to adjust the channel dimension to match the input features; This represents the serialization feature that integrates location information.

[0082] Finally, the processed location encoding information is compared with the original input features. By adding elements one by one, a serialized feature incorporating location information is obtained. This information is then fed into the subsequent serialization attention layer.

[0083] Regarding the serialization attention layer, it receives serialized features with fused positional information, segments the features according to sample boundaries using batch offsets, and independently computes self-attention within a continuous sequence window of each sample to obtain local context features; specifically, it includes the following steps: 1) Rearrangement and filling based on sequence order.

[0084] First, using the sequence order generated in step S2, the fused features processed by the conditional position encoding layer are... The arrangement is performed so that it is consecutively arranged in memory according to the space-filling curve, denoted as... ; This represents the serialized attention input features rearranged according to Order.

[0085] Next, window partitioning is prepared; a fixed window size of W is set. If the sequence length of the current sample is not divisible by W, padding is added to the end of the current sample range to make the sequence length an integer multiple of W.

[0086] Meanwhile, this embodiment generates a valid point mask to mark which points are original data and which are invalid points generated by padding, so that the influence of the padding part can be ignored in subsequent calculations.

[0087] 2) Independent self-attention calculation within the window.

[0088] rearranged features Divide into several continuous sequence windows of length W. For the first... i A window, whose input features are denoted as... , This represents the C-dimensional feature matrix corresponding to the W tokens within the window; i This represents the window index. If there are fewer than W tokens at the end of a sample, padding is only performed within that sample, and the padding tokens are masked using a valid dot mask.

[0089] The standard self-attention mechanism is executed independently within each window: Linear mapping: through a learnable parameter matrix , , The input features are mapped to query vectors respectively. Key vector Sum value vector The linear mapping formula is: , , .

[0090] Attention output: Calculate the attention scores within the window and sum them using a weighted average. The formula is as follows: ,in, d The dimension representing a single attention head. For the first i The relative position offset or geometric offset of each window. Indicates the first i The attention output results for each window.

[0091] 3) Feature splicing and reverse order recovery.

[0092] Once the attention calculations for all windows are complete, output the results for each window. The sequences are spliced ​​together according to the original window order to form a complete sequence feature.

[0093] Then, using the inverse index generated in step S2, the concatenated features are restored to the original physical point cloud order to obtain the final attention output. .

[0094] The serialized attention layer in this embodiment avoids the complexity problem caused by global attention calculating every pairwise for all points, reducing it to linear complexity. Furthermore, since the input sequence is sorted based on the space-filling curve, consecutive sequence windows often correspond to adjacent regions in three-dimensional space. Therefore, this in-window attention calculation can effectively maintain the feature interactions between spatially adjacent points, achieving efficient enhancement of the local context.

[0095] Regarding the adaptive sequence window routing module, refer to... Figure 3 As shown, Figure 3 Part (a) of the diagram shows the connection relationship of the enhanced base block E-Block: the input features pass through CPE, layer normalization, serial attention, layer normalization, ASWR, layer normalization and MLP in sequence, and residuals are added at the corresponding positions of CPE, serial attention, ASWR and MLP; the symbol “⊕” indicates element-wise residual addition, and LN indicates layer normalization.

[0096] Figure 3 Part (b) of the diagram includes (b1), (b2), and (b3), which gives the internal structure of ASWR. Figure 3 In part (b1), the input features H,P] enters the original scale branch B0 and the coarse scale branches B1 and B2 in parallel; B0 directly obtains the original scale candidate features through lightweight sequence mixing (LSM), while B1 and B2 perform Pool→LSM→Unpool with the preferred pooling ratios s1=2 and s2=3, respectively, to obtain the medium-scale and large-scale candidate features. Figure 3 In part (b2), the single sample sequence is divided into windows according to the window length W, and the statistical measures are composed of the characteristic mean s, spatial dispersion d, spatial coordinate range r and local density ρ of each window. The window-level weights of each scale branch are obtained through two layers of MLP and Softmax with temperature coefficient τ. Figure 3 In part (b3), the window-level weights are broadcast to each point within the window, multiplied point-by-point with the corresponding scale candidate features, summed, and then linearly projected by Proj to obtain the ASWR output. .

[0097] Specifically, the following steps are included: 1) Construct the input branch.

[0098] Serialization attention output After layer normalization, the features are used as input features for the adaptive sequence window routing module. And denote the three-dimensional coordinate sequence corresponding to H as P. The adaptive sequence window routing module uses... H As the object for constructing multi-scale candidate features, P is used as the basis for window geometric statistics. If the batch contains multiple point cloud samples, H and P are divided into single-sample sequences of each sample according to the batch offset. , , This indicates that the b-th sample enters the point-level feature sequence of ASWR. Indicates and A one-to-one correspondence of three-dimensional coordinate sequences.

[0099] 2) Construct fine-grained branches at the original scale.

[0100] The original-scale fine-grained branching employs a lightweight sequence mixing module, based on features. Input, output features The length of the input feature sequence remains constant. The lightweight sequence mixing module consists of layer normalization, first linear mapping, one-dimensional depthwise convolution, GELU activation, and second linear mapping; the calculation process is expressed by the formula:

[0101] in, This indicates a lightweight sequence mixing operation; Presentation layer normalization operation; This represents a one-dimensional depthwise convolution performed along the sequence direction to capture local context; This indicates that GELU is activated; This is the first linear mapping parameter, used to adjust the channel dimension of the feature; The second linear mapping parameter is used to map the activated features back to the original channel dimension.

[0102] 3) Construct coarse-scale context branches.

[0103] To capture the global structural information and long-range dependencies of the point cloud, this embodiment constructs one or more coarse-scale context branches. The core function of this branch is to downsample the input feature sequence to expand the receptive field, extract global context features, and then restore the original sequence length through upsampling. The specific implementation process is as follows: K coarse-scale context branches 3.1) Boundary determination and filling based on batch offset.

[0104] First, determine the sample range. Using the batch offset generated in step S2, determine the current sample range. b The start and end index range of each sample in the concatenated total sequence is denoted as . L b Then for the first... b Input sequence of one sample This step ensures that subsequent processing is strictly limited to the current sample, preventing information leakage between different samples.

[0105] Next, calculate the fill length. Set the pooling ratio (or stride) of the current branch to... ,and To ensure complete pooling grouping, the computation can be... Fill length that is divisible , L b Indicates the first b The effective sequence length of a point cloud sample before padding. Indicates not less than L b And can be The length of the smallest integer that is divisible.

[0106] Finally, perform tail padding. Padded ends of the current sample sequence. An invalid token was obtained. , This represents the feature obtained after padding. In this embodiment, the padding token is only used to form a complete pooling group, and is prevented from participating in the final valid output by using a mask.

[0107] 3.2) Group pooling that does not cross boundaries.

[0108] First, divide the pooling groups, and then fill them with... According to each Each consecutive token is divided into a pooling group. Since the padding and division are based on the sample length, this grouping method naturally ensures that the pooling group will not cross the sample boundary.

[0109] Features within each pooling group are aggregated to obtain a coarse-scale sequence. For the coarse-scale feature of the q-th pooling group, the calculation formula is:

[0110] in, q Indicates a pooled group index. q=0,1,…, ; Indicates the first b The sample at the th kNumber of pooling groups on each coarse-scale branch; u Indicates the relative position index within the same pooling group; Indicates the first q C-dimensional coarse-scale features after pooling group aggregation; Pool (·) indicates an aggregation operation within a group, which can be average pooling, max pooling, or weighted pooling with a mask.

[0111] Pool Average pooling, max pooling, or masked average pooling can be used; average pooling is preferred to obtain a stable context representation.

[0112] 3.3) Lightweight sequence mixing and feature recovery.

[0113] First, this embodiment obtains a coarse-scale sequence. Performing a lightweight sequence mixing operation yields a coarse-scaled mixed sequence, which can be expressed by the following formula:

[0114] in, Indicates the first k The coarse-scaled sequence is obtained by mixing the coarse-scaled branches with lightweight sequences. Indicates the first k The lightweight sequence mixing operator is used in the coarse-scale branches. This operation is similar to the lightweight sequence mixing operation in the original fine-grained branches, and is used to further fuse local neighborhood information at the coarse-grained level. It is worth noting that different coarse-scale branches can have independent LSM parameters, or they can share some parameters.

[0115] To restore the features to their original resolution, this embodiment uses the blended coarse-scale sequence. Each token in the pool is allocated according to the pooling ratio. The repetition is restored to the original sequence length, expressed by the formula:

[0116] in, Indicates rounding down; j Indicates the position index of the fine-scale sequence after padding; Used to determine the coarse-scale token index corresponding to this location; Let represent the C-dimensional candidate feature at the j-th position after repeated recovery.

[0117] Finally, trim away the invalid parts that were previously filled in for alignment, keeping only the front portion. L b Each token is used to obtain the final sequence H, which is the same as the original input sequence. b Coarse-scale candidate features of equal length The process can be expressed by the following formula:

[0118] in, This means that padding is only added to the end of the b-th sample. Integer multiples; Indicates by Aggregate consecutive tokens within a group; This represents a lightweight sequence mix of the k-th coarse-scale branch; This indicates that each coarse-scale token is repeated. Second-rate; Indicates that only the first part is retained. One valid position. Compared with the original input With consistent length, it can be fused point-by-point with candidate features from other scales.

[0119] 4) Generate dynamic routing weights based on the mean of feature responses and geometric statistics within a single sample sequence window.

[0120] To ensure computational efficiency during batch processing, multiple point cloud samples are typically concatenated into a long sequence. However, when performing local window statistics, it is essential to strictly distinguish the boundaries between different samples.

[0121] Reference Figure 4 The diagram illustrates the impact of sample boundaries on window partitioning under batch processing conditions and the intra-sample independent processing method of this invention. In the diagram, blue tokens represent sample A, light green tokens represent sample B, and the vertical dividing line between them represents the sample boundary determined by the offset. The red box indicates that if the concatenated global sequence is directly segmented using a fixed window, a cross-sample window may be formed that simultaneously contains the tail token of sample A and the head token of sample B, thus causing geometric statistics and routing weights to be interfered with by another independent sample.

[0122] Figure 4 The two independent boxes in the lower half represent the internal processing of sample A and sample B, respectively. Gray pads represent invalid padding positions added only at the end of their respective samples; all padding, pooling grouping, routing window partitioning, window statistics, weight broadcasting, and pruning are performed within the [o] boxes of the corresponding samples. (b 1) ,o b Complete independently within the specified range, and do not cross sample boundaries, where o b Let represent the cumulative number of valid tokens from the first b samples. Then, the index range of the b-th sample in the concatenated sequence is [o...]. (b 1) ,ob After processing, the valid outputs of each sample are spliced ​​together in the original batch order, thereby ensuring that the fusion of statistics and features has a clear physical meaning for each single sample.

[0123] First, this embodiment uses the batch offset generated in step S2 to determine the start and end positions of each sample in the overall sequence. All pooling group partitioning and window routing statistics are performed within a single sample.

[0124] Secondly, this embodiment explicitly prohibits a pooling group or routing window from crossing the boundary between two samples. When the end length of a sample is insufficient to form a complete pooling group or window, padding is only applied to the end of that sample, and the original length is used during subsequent output recovery. The process involves cropping. This mechanism effectively avoids incorrectly classifying the tail points of sample A and the head points of sample B into the same statistical range, ensuring the correctness of the physical meaning of the feature calculations.

[0125] For the b-th sample, its sequence is divided into consecutive sequence windows according to a preset window length W. For the i-th window, the set of point indices it contains is denoted as... The features of points within the window are 3D coordinates are This embodiment calculates the following five types of feature statistics within this window to comprehensively characterize the feature distribution of the region: (1) Mean of characteristic response The calculation formula is: This metric describes the average semantic response intensity of points within a window, reflecting the overall activation level of the region.

[0126] (2) Mean of window coordinates The calculation formula is: That is, the geometric center coordinates of all points within the window.

[0127] (3) Spatial dispersion The calculation formula is: This index describes the degree of dispersion of the point set within the window around the geometric center. The larger the value, the sparser or wider the point distribution.

[0128] (4) Spatial coverage area The calculation formula is: This index calculates the maximum span of the window in the x, y, and z directions, and is used to describe the physical size of the local area.

[0129] (5) Local density The calculation formula is: ,in ,in, This represents the window bounding box volume estimated from the spatial coverage area. , , These represent the maximum span of the window in the three spatial directions. This is a numerical stability constant to prevent division by zero. This index reflects the number of points per unit volume, i.e., the density of the point cloud.

[0130] Then, based on the above statistics, a statistical vector is formed by concatenating them. It can be expressed by the formula: , Statistical vector The input to a lightweight routing network R(·) consisting of two fully connected layers is first subjected to dimensionality reduction or feature transformation through the first fully connected layer, and the GELU nonlinear activation function is introduced, as shown in the formula: ;in, This represents the weight matrix of the first layer of the routing network. This represents the bias vector of the first layer. Indicates the first i The routing hidden features are obtained by transforming the statistical vectors of each window through a first fully connected layer and applying GELU activation. Then, a second fully connected layer maps these features to a dimension corresponding to the number of branches (K+1, corresponding to one original-scale branch and K coarse-scale branches), yielding the preference scores. ;in, This represents the weight matrix of the second layer of the routing network. This represents the bias vector of the second layer. Indicates the first i Each window corresponds to a fine-grained branch at the original scale and... K Unnormalized preference scores for each coarse-scale context branch.

[0131] Finally, through the temperature coefficient Normalization using the Softmax function yields the window. i Weights for the k-th scale branch: Where k=0 represents the original scale fine-grained branch, and k=1,...,K represents K coarse-scale branches. The weights satisfy... and .

[0132] 5) Adaptively weighted fusion of features from each branch is performed to obtain multi-scale enhanced features.

[0133] This embodiment utilizes the generated dynamic weights to perform weighted fusion of candidate features from different branches.

[0134] Window-level weights Broadcast to window All points within the window. For points belonging to the window... i The j The fusion feature is calculated from the given points. .in, For the original scale fine-grained branch in the th j Candidate features at each point For the first k The coarse-scale branch recovers to the first... j Candidate features after each point. These are then expressed using linear projection with unified channels to obtain multi-scale enhanced features, expressed by the formula: ; Indicates the first j The C-dimensional feature of each point after multi-scale dynamic weighting This represents the multi-scale augmentation features output by ASWR. This represents a projection layer that maintains the same number of points while performing a linear transformation on the channels.

[0135] Regarding the feedforward network, it receives multi-scale enhancement features, performs point-by-point feature mapping through two layers of linear transformation and nonlinear activation function, and outputs the enhancement features of the current base block.

[0136] In this embodiment, after the adaptive sequence window routing module outputs its output, it first adds the residuals of the serialized attention output features to obtain... , This indicates that the residual augmentation feature already incorporates both sequential attention local context and ASWR adaptive multi-scale context, with both the number of points and channel dimension being the same as... F sa Since the features are consistent, they can be directly used as input to the subsequent feedforward network. Subsequently, this feature enters the feedforward network for further nonlinear transformation and feature enhancement. This feedforward network is mainly used to process features independently along the channel dimension, and its specific structure includes layer normalization, a first linear layer, an activation function, a second linear layer, and a final residual connection.

[0137] The process of layer normalization, the first linear layer, the activation function, and the second linear layer operations in this embodiment is expressed by the following formula:

[0138] After that, with The residual join, performed element-wise, can be expressed by the following formula:

[0139] in, Indicates the feedforward network to The resulting channel transformation increment, , These represent the weight matrices of the first and second linear layers of the feedforward network, respectively. , These represent the bias vectors of the corresponding linear layers. This represents the final output feature of the current enhanced base block.

[0140] Step S4: Upsample and recover the enhanced features input to the decoder, and output the semantic category of each point through the point-level prediction head.

[0141] In this embodiment, the enhanced features output from the last stage of the encoder are used as the initial input to the decoder. The decoder employs a hierarchical structure, with the number of stages corresponding to the downsampling factor of the encoder. During each stage of decoding, the low-resolution feature map is mapped back to the high-resolution original sequence length using the inversion index or unpooling index recorded by the encoder during the forward propagation phase.

[0142] This embodiment introduces a skip connection mechanism. After upsampling at each level of the decoder, the features of that layer are concatenated or element-wise added to the shallow features of the corresponding level in the encoder. The shallow features of the encoder contain rich boundary information and fine-grained geometric structures, while the deep features of the decoder contain abstract global semantic context. Through this fusion method, the generated intermediate features possess both high-precision spatial localization capabilities and robust semantic discriminative capabilities.

[0143] The fused features have a high channel dimension. In this embodiment, to extract a more compact feature representation and reduce computation, the fused features are input into a multilayer perceptron. This multilayer perceptron includes convolutional layers (such as 1x1 convolutions), batch normalization, and activation functions (such as ReLU or GELU). After processing, the number of channels in the features is reduced to a preset dimension, and a nonlinear transformation is completed to obtain the final decoded features.

[0144] Finally, the decoded features are input into a point-level prediction head, which is essentially a fully connected layer (or a 1x1 convolutional layer), and its weight matrix is ​​denoted as... The bias is denoted as This layer maps feature dimensions to the number of categories. N cls The mapping result is normalized using the Softmax function, transforming it into a probability distribution. The calculation formula is as follows:

[0145] in, This represents the decoded feature vector of the j-th input point-level prediction head. Let represent the predicted probability that the j-th point belongs to the c-th semantic category, where c=1,…, Ncls , and These represent the weight matrix and bias vector of the category mapping layer, respectively.

[0146] Finally, the category with the highest probability is selected as the semantic label for the j-th point. The set of labels for all points constitutes the complete semantic segmentation result of the point cloud.

[0147] Finally, this embodiment also performs end-to-end training on the constructed segmentation network. This embodiment uses the standard point-level cross-entropy loss as the training objective function, specifically:

[0148] in, For point-level cross-entropy loss, N valid For valid points, For the first l The true semantic category of each point To represent the network prediction of the first l Each point belongs to the real category. The logarithm of the probability value.

[0149] This embodiment transforms disordered point clouds into ordered sequences using spatial serialization technology, effectively reducing the computational complexity of the Transformer architecture. It innovatively introduces an adaptive sequence window routing module, dynamically generating multi-scale fusion weights based on local geometric statistics. This achieves an adaptive balance between global semantic context and local geometric details without requiring manual labeling of routing data. Combined with an end-to-end joint optimization strategy, it significantly improves the network's segmentation accuracy and robustness for complex scenes and small targets. Ultimately, while ensuring real-time inference efficiency, it provides high-precision 3D environmental perception capabilities for applications such as autonomous driving.

[0150] Example 2 This invention also discloses a 3D point cloud semantic segmentation system based on adaptive sequence window routing, referring to... Figure 5 As shown, the method described in any of Embodiment 1 includes the following modules: Point cloud data acquisition module: used to acquire the 3D point cloud data to be segmented. The point cloud data includes the 3D coordinates and point features of discrete points. Spatial serialization module: used to perform spatial quantization and spatial serialization on point cloud data, generate sequence order, inversion index and batch offset, and obtain serialization features; Feature enhancement module: used to construct a segmentation network containing a serialized point cloud Transformer encoder and decoder; input the serialized features into the encoder, and use the adaptive sequence window routing module in the encoder to perform multi-scale context enhancement and fusion on the features to obtain enhanced features; Semantic output module: used to upsample and recover the enhanced features input to the decoder, and output the semantic category of each point through the point-level prediction head.

[0151] This embodiment is deployed on an onboard computing platform of a vehicle with Level 3 or higher autonomous driving capabilities to analyze road scene data collected by LiDAR in real time. Traditional point cloud processing systems often struggle to balance global contextual information (such as distant vehicle outlines) and local geometric details (such as nearby road surface textures). Furthermore, due to the sparsity and disorder of point cloud data, directly processing raw point cloud data results in enormous computational complexity, making it difficult to meet the real-time requirements of onboard chips. This embodiment's system transforms disordered point clouds into ordered sequences through spatial serialization and utilizes an adaptive sequence window routing module to automatically adjust the receptive field size based on the complexity of different road surface areas (e.g., flat roads vs. congested intersections), significantly reducing computational redundancy while maintaining accuracy.

[0152] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0153] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A 3D point cloud semantic segmentation method based on adaptive sequence window routing, characterized in that, Includes the following steps: S1. Obtain the three-dimensional point cloud data to be segmented, wherein the point cloud data includes the three-dimensional coordinates and point features of discrete points; S2. Spatial quantization and spatial serialization are performed on the point cloud data to generate sequence order, reverse order index and batch offset, and serialization features are obtained. S3. Construct a segmentation network that includes a serialized point cloud Transformer encoder and decoder; input the serialized features into the encoder, and use the adaptive sequence window routing module in the encoder to perform multi-scale context enhancement and fusion on the features to obtain enhanced features; S4. Input the enhanced features into the decoder for upsampling recovery, and output the semantic category of each point through the point-level prediction head.

2. The method as described in claim 1, characterized in that, Step S1 specifically includes: Raw 3D point cloud data is acquired by LiDAR, depth camera or 3D scanner. The raw 3D point cloud data includes the 3D coordinates, reflection intensity and color information of discrete points. The original 3D point cloud data is preprocessed, including removing invalid and duplicate points, voxel filtering downsampling, and removing discrete noise points. The preprocessed point cloud data is used as the three-dimensional point cloud data to be segmented, and the three-dimensional coordinates and point features of each discrete point are extracted. The point features include at least one of reflection intensity, color information and normal vector.

3. The method as described in claim 1, characterized in that, Step S2 specifically includes: Normalize and map the 3D coordinates of each discrete point to the grid, converting the physical coordinates into grid coordinates; Based on the grid coordinates, a spatial sorting code is calculated using a space-filling curve. All points are then sorted in ascending order according to the spatial sorting code to generate a sequence order. The original position of each point before sorting is recorded to generate a reverse index. For batch input point cloud data, the number of points contained in each sample is counted and accumulated, and the start and end range of each sample in the spliced ​​total sequence is recorded to obtain the batch offset, which is used to distinguish the boundaries of different samples. The serialized point cloud data is mapped to the target dimension feature space through the embedding layer to obtain the final serialized features.

4. The method as described in claim 3, characterized in that, In step S3, the encoder comprises multiple basic blocks, and the basic blocks include at least a conditional position coding layer, a serialization attention layer, an adaptive sequence window routing module, and a feedforward network; The conditional location coding layer receives the serialized features, maps the sequence order into a learnable location embedding vector, and adds it element-wise to the serialized features to obtain a serialized feature that fuses location information. The serialization attention layer receives serialized features with fused position information, uses the batch offset to segment the features according to the sample boundaries, and independently calculates self-attention within the continuous sequence window of each sample to obtain local context features. The adaptive sequence window routing module receives local context features, constructs fine-grained branches at the original scale and at least one coarse-scale context branch in parallel, generates dynamic routing weights based on the mean of feature responses and geometric statistics within a single sample sequence window, and performs adaptive weighted fusion of features from each branch to obtain multi-scale enhanced features. The feedforward network receives multi-scale enhancement features, performs point-by-point feature mapping through two layers of linear transformation and nonlinear activation functions, and outputs the enhancement features of the current base block.

5. The method as described in claim 4, characterized in that, The original-scale fine-grained branch maintains the same length of the input feature sequence and extracts local geometric details through a lightweight sequence mixing operation. The lightweight sequence mixing operation includes layer normalization, a first linear mapping, a one-dimensional depthwise convolution, an activation function, and a second linear mapping.

6. The method as described in claim 4, characterized in that, The coarse-scale context branch performs downsampling pooling on the input feature sequence to expand the receptive field, extracts global context features, and then restores the original sequence length through upsampling; specifically including: The sequence range of the current point cloud sample is determined using the batch offset, and padding is performed at the end of the sequence range so that the sequence length can be divided by the pooling ratio. The sequence is divided into multiple pooling groups according to the pooling ratio, and the pooling groups do not cross the sample boundaries. Pooling operation is performed on the features within each pooling group to obtain a coarse-scale sequence. A lightweight sequence mixing operation is performed on the coarse-scale sequence. Each token in the mixed coarse-scale sequence is restored to the original sequence length by repeating the pooling ratio times, and the padding part is removed to obtain a coarse-scale candidate feature with the same length as the original input sequence.

7. The method as described in claim 4, characterized in that, The generation of dynamic routing weights based on the mean of feature responses and geometric statistics within a single sample sequence window specifically includes: Window-level feature statistics calculation: Based on the batch offset, the features are isolated according to the sample boundaries. Within the continuous sequence window of each sample, the mean of the feature response, spatial dispersion, spatial coverage, and local density of the points within the window are calculated to generate a statistical vector representing the feature distribution within the window. Dynamic routing weight generation: The statistical vector is input into a lightweight routing network consisting of two fully connected layers. The first fully connected layer reduces the dimensionality and introduces non-linear activation. The second fully connected layer maps the vector to the dimension corresponding to the number of branches. Finally, the vector is normalized by the Softmax function to obtain the dynamic routing weights of each scale branch.

8. The method as described in claim 1, characterized in that, Step S4 specifically includes: The enhanced features are used as the initial input to the decoder, and the resolution is restored by upsampling step by step according to the hierarchical structure corresponding to the downsampling factor of the encoder. After upsampling at each level of the decoder, shallow features of the corresponding level of the encoder are fused through skip connections to supplement local geometric details; The fused features are subjected to nonlinear transformation and channel dimensionality reduction by a multilayer perceptron to obtain decoded features containing rich semantic and location information. The decoded features are input into the point-level prediction head, mapped to the category dimension through a fully connected layer and normalized by Softmax. The category with the highest probability is selected as the semantic label of each point, and the segmentation result is output.

9. A 3D point cloud semantic segmentation system based on adaptive sequence window routing, characterized in that, The method described in any one of claims 1-8 includes the following modules: Point cloud data acquisition module: used to acquire the three-dimensional point cloud data to be segmented, wherein the point cloud data includes the three-dimensional coordinates and point features of discrete points; Spatial serialization module: used to perform spatial quantization and spatial serialization on the point cloud data, generate sequence order, inversion index and batch offset, and obtain serialization features; Feature enhancement module: used to construct a segmentation network containing a serialized point cloud Transformer encoder and decoder; input the serialized features into the encoder, and use the adaptive sequence window routing module in the encoder to perform multi-scale context enhancement and fusion on the features to obtain enhanced features; Semantic output module: used to input the enhanced features into the decoder for upsampling and recovery, and output the semantic category of each point through the point-level prediction head.

Images