Semantic perception parallel mask strategy-based point cloud segmentation method

Abstract

The invention discloses a semantic perception parallel mask strategy-based point cloud segmentation method. The method comprises the following steps of: 1, calculating to obtain a standardized point cloud; 2, performing farthest point sampling on the standardized point cloud to extract a representative center point, and performing calculation to obtain a local point cloud group feature tokens sequence with rich spatial information; step 3, dividing the local point cloud group feature tokens sequence into K semantic related groups, implementing a parallel mask strategy on each group, dynamically adjusting loss weight by combining semantic similarity among the groups, and performing pre-training through a multi-path parallel mask auto-encoder framework to obtain semantic enhanced feature representation; 4, extracting multi-scale features from the semantically enhanced features through a selective state space model; and step 5, constructing a point cloud segmentation network, predicting a category probability for each point, then optimizing segmentation network parameters through a cross entropy loss function, and realizing accurate distinguishing of the road surface and the obstacle area.

Description

TECHNICAL FIELD

[0001] The application relates to the field of point cloud segmentation, and in particular to a point cloud segmentation method based on a semantic perception parallel mask strategy. BACKGROUND

[0002] In the field of point cloud data processing, the prior art has the problems of single mask strategy and insufficient semantic utilization when processing sparse point clouds. The traditional mask autoencoder (MAE) adopts a random mask or block mask strategy, which can achieve a high mask ratio, but has the following defects: first, the single sequence mask mode limits the model's parallel learning ability for different semantic associated features, and cannot fully exploit the potential correlation between features; second, the mask selection lacks semantic guidance, which may cause important semantic features to be excessively covered, affecting the model's understanding of key structures; third, the data utilization rate is insufficient during training, as only part of the visible features is used for learning each time, resulting in waste of computing resources; finally, the existing method lacks a consistency constraint mechanism between different semantic groups, making it difficult to ensure the prediction stability of the model under different perspectives. These problems are particularly prominent in the automatic driving scenario, and the identification and segmentation accuracy of small obstacles (such as cones, roadblocks, etc.) is limited, affecting the safety and reliability of the system. SUMMARY

[0003] The technical problem to be solved by the application is to provide a point cloud segmentation method based on a semantic perception parallel mask strategy to overcome the shortcomings of the prior art.

[0004] To solve the above technical problems, the application discloses a point cloud segmentation method based on a semantic perception parallel mask strategy, which comprises the following steps:

[0005] Step 1: Calculate the geometric centroid of the point cloud, translate the point cloud coordinate system so that the centroid is at the origin (0, 0, 0), then calculate the distance of the farthest point from the origin after translation, and accordingly scale the entire point cloud to a unit sphere, thereby obtaining a standardized point cloud. This processing ensures that point clouds of different sizes have consistent proportions and reference frames in subsequent processing;

[0006] Step 2: Local feature extraction and encoding: representative center points are extracted by farthest point sampling on the standardized point cloud, a local point cloud group is constructed using the K-nearest neighbor algorithm, and initial feature representations are extracted by a shared convolutional network to obtain a tokens sequence of local point cloud group features with rich spatial information;

[0007] Step 3, semantic-aware parallel mask pre-training: based on cosine similarity calculation, the local point cloud group feature tokens sequence is divided into K semantic related groups, innovative parallel mask strategy is implemented for each group, loss weight is dynamically adjusted combined with inter-group semantic similarity, pre-training is carried out through a multi-path parallel mask autoencoder framework, and semantic-enhanced feature representation is obtained;

[0008] Step 4, multi-scale feature fusion: input the semantic-enhanced feature representation into the PointMamba architecture, extract multi-scale features through the selective state space model, and further enhance the spatial perception ability and feature expression ability of the point cloud;

[0009] Step 5, accurate point cloud segmentation: based on the multi-scale features output by step 4, a point cloud segmentation network is constructed, which can fully utilize the semantic-enhanced features for accurate segmentation, predict the class probability for each point, and then optimize the segmentation network parameters through the cross-entropy loss function to realize accurate differentiation between the road surface and the obstacle region.

[0010] Step 1 is specifically:

[0011] Step 1-1, given point cloud dataset , where each point , i represents the index of the point, which is used to uniquely identify each point in the point cloud, and the value range is 1 to N; N represents the total number of points in the point cloud, respectively represent the x coordinate, y coordinate and z coordinate value of the i-th point in the three-dimensional space, and the centroid of the point cloud is calculated:

[0012]

[0013] and all point cloud data is translated to a coordinate system with the centroid as the origin, obtaining the translated point cloud :

[0014]

[0015] Step 1-2, calculate the distance of the farthest point from the origin (the center point of the point cloud, i.e. coordinate (0, 0, 0)) in the translated point cloud:

[0016]

[0017] where is the distance from the farthest point to the origin, is the Euclidean norm of point ;

[0018] Step 1-3, scale the entire translated point cloud to the unit sphere to obtain the standardized point cloud :

[0019] .

[0020] The standardization processing ensures that the point cloud data will not be biased in subsequent calculations due to scale differences.

[0021] Step 2 is specifically: feature extraction combining FPS sampling, KNN clustering, and zigzag sorting:

[0022] Step 2-1, given the standardized point cloud set , use the FPS sampling algorithm to sample G representative center points from it :

[0023] Randomly select a point from the dataset as the first center point ;

[0024]

[0025] Select each subsequent center point through FPS sampling ;

[0026]

[0027] Wherein, represents the selected center point set, represents any point in the point cloud dataset, represents the point farthest from the existing center point set, represents that for each point , calculate the distance between it and all selected center points , and select the farthest point as the next center point ;

[0028] Then for each center point , find its nearest neighbor points in the original point cloud to form a local point cloud group , which is a three-dimensional tensor:

[0029] ,

[0030] Wherein, B represents the number of point cloud samples processed simultaneously in one training, G represents the number of representative center points sampled from each point cloud through FPS sampling (Farthest Point Sampling), and M represents the number of neighbor points found for each center point through KNN (K-Nearest Neighbor) algorithm;

[0031] Step 2-2, for the local point cloud group Perform spatial serialization:

[0032]

[0033] ZigzagSort() is a spatial sorting function that achieves ordered arrangement of point clouds by constructing continuous paths on different planes. This indicates that the point cloud after KNN is normalized.

[0034] Steps 2-3, Convolutional Network Feature Encoding:

[0035] Serialized local point cloud groups The input is shared by a convolutional neural network to encode features and form an initial feature sequence with spatial sequence information. :

[0036]

[0037]

[0038] in, The d-dimensional feature vector obtained after encoding the i-th local point cloud group by a convolutional network contains the geometric structure information and spatial texture features of the local region.

[0039] Step 3 specifically involves:

[0040] Step 3-1, Semantic similarity calculation and feature normalization:

[0041] For feature sequences Perform L2 norm normalization to ensure the accuracy of subsequent similarity calculations:

[0042]

[0043] in, Unevaluated eigenvectors Using the L2 norm, calculate the cosine similarity matrix S between normalized eigenvectors:

[0044]

[0045] Elements in the similarity matrix S Indicates the first The eigenvector and the eigenvector The cosine similarity between the feature vectors, with values ​​ranging from 1 to 2. The closer the value is to 1, the more similar the two feature vectors are;

[0046] Step 3-2: Based on semantic similarity, an intelligent grouping strategy divides the G feature tokens into K semantically related groups. ;

[0047] Step 3-3, construct a multi-path parallel mask autoencoder architecture training framework, realize the simultaneous training of K parallel mask tasks, obtain semantic enhanced feature representation through pre-training, the framework implements parallel mask strategy on original semantic related groups, respectively constructs K parallel mask learning tasks for K semantic related groups, only one group is visible in each task, and the remaining K-1 groups are masked, which greatly improves the training efficiency and feature learning ability; The framework is an end-to-end training architecture specially designed for semantic perception parallel mask strategy, which greatly improves the training efficiency and feature learning ability;

[0048] The parallel mask strategy implemented in step 3-3 on the original semantic related groups obtains K mask configurations under different semantic perspectives, each configuration maintains a mask ratio of 75%, but focuses on different semantic feature combinations, laying the foundation for subsequent multi-path parallel training; The core idea of this strategy is to convert the traditional single sequence mask into multi-path parallel mask, realizing diversified learning of semantic perspectives.

[0049] Parallel mask mechanism principle:

[0050] The traditional mask autoencoder (MAE) can only learn feature reconstruction under one mask perspective each time, while the parallel mask strategy of the present invention generates K different mask perspectives simultaneously, each perspective maintains the same mask ratio, but focuses on different semantic combinations.

[0051] Uniform design of mask ratio:

[0052] The mask ratio is 75%, consistent with the traditional MAE, ensuring the uniformity of model learning difficulty.

[0053] For the kth semantic related group, define its mask strategy as:

[0054] The visible part of the kth group is only the group itself:

[0055] The mask part of the kth group is that other groups are masked:

[0056] Such design makes each semantic group as the visible part of an independent mask learning task, while other groups as the target to be reconstructed, realizing semantic guided mask learning.

[0057] Maximize data utilization:

[0058] Traditional MAE: only (1-mask_ratio) x G = 25% x G data is used each time, and the application: by processing K mask tasks in parallel, K x (G / K) = G data is used each time, achieving 100% data utilization:

[0059] The mask autoencoder architecture training framework of step 3-3 comprises: a shared Mamba encoder network, a position encoding network, a multi-path parallel decoder network, and a mask token processing module; with "shared encoder + parallel decoder + position encoding + mask token processing" as the core, efficient feature learning and reconstruction of multiple semantic perspectives are realized.

[0060] The mask token processing module performs feature filling work for the masked positions, and the core component of this module is a learnable mask token parameter, which is automatically optimized in the training process as a unified feature representation for all masked positions; when the mask strategy removes the original features of certain point cloud groups, this module uses the learnable mask token parameter to fill the missing positions, ensuring that the subsequent decoder can receive a complete input sequence.

[0061] Step 3-3 specifically includes:

[0062] Step 3-3-1, construct K parallel mask tasks: for each semantic group in k = 1, 2,..., K, input the visible tokens sequence of the group into the shared Mamba encoder network, and set the other K-1 groups to the mask state, and input the visible tokens sequence of each group into the shared Mamba encoder network, and learn the global feature through the selective state space mechanism :

[0063]

[0064] The core calculation process of the Mamba encoder state space model is:

[0065]

[0066]

[0067] Where A, B, C, and D are input-dependent selective parameters (this is the core feature of Mamba), is the hidden state, is the input token, is the output feature. Compared with the traditional Transformer architecture, Mamba has the advantage of linear computational complexity and can efficiently process long sequence data.

[0068] Position encoding integration:

[0069] Step 3-3-2, the representative center point coordinates extracted by farthest point sampling (FPS) in step 2-1 As the input of position encoding, in order to maintain the spatial geometric information of the point cloud, the corresponding center point coordinates are converted into position features by the position encoding network:

[0070]

[0071] is the center point coordinate The position encoding vector obtained after processing by the position encoding network has the same dimension (both d dimensions) as the local point cloud group feature tokens extracted in step 2-3 , which is used to carry the spatial geometric position information of the point cloud.

[0072] where the position encoding subset of the kth mask strategy is ;

[0073] The position encoding network adopts a two-layer fully connected structure:

[0074]

[0075] where GELU is the Gaussian error linear unit activation function, which ensures effective encoding of position information;

[0076] Multi-channel parallel decoder network:

[0077] Step 3-3-3, the decoder network adopts a multi-layer Mamba state space module for sequence modeling, and the decoder network receives K sets of global features output by the encoder and the position prior information of the corresponding masked part, and obtains the point cloud reconstruction results of K sets of parallel mask autoencoders through decoding processing :

[0078]

[0079] The decoding processing includes the following steps (corresponding to the core process of the decoder framework):

[0080] 1) Mask token embedding layer: for the information missing part of the mask part , the learnable mask token parameters perform feature filling for each mask position, and then fill the filled mask token with the corresponding position encoding The addition obtains a mask token embedding vector fused with spatial position information;

[0081] The mask token embedding vector fused with position information is output, and a basis input of a mask area is provided for a decoder;

[0082] Since the original features of the mask part are not input into the encoder, there is information loss, so for each mask position, i.e. the token index position in the mask token embedding vector, a learnable mask token is used for filling, and through parallel processing of K mask perspectives (each perspective corresponds to filling and reconstruction of a group of mask positions), the training efficiency is significantly improved:

[0083] - traditional MAE training steps: 1 forward propagation → 1 mask perspective

[0084] - parallel training of the application: 1 forward propagation → K mask perspectives

[0085] 2) Mamba decoding layer: the visible part global feature output by the encoder is spliced with the mask token embedding vector obtained in the previous step to form a complete token sequence, and a long-distance dependency relationship is captured through multiple Mamba modules.

[0086] 3) Coordinate prediction layer: the feature decoded by the Mamba decoding layer is mapped to a three-dimensional space coordinate value to obtain the reconstruction result of each group of mask parts, and finally the reconstructed point cloud coordinates are output.

[0087] The total loss function used in the multi-path parallel mask autoencoder architecture training in step 3-3 is:

[0088]

[0089] Among them, the multi-level loss function includes the Chamfer distance reconstruction loss , the weighted semantic consistency loss , and the consistency is the weight coefficient of the consistency loss, usually taking the value 1.0.

[0090] Multi-level loss function design and optimization:

[0091] The Chamfer distance reconstruction loss:

[0092] The bidirectional Chamfer distance loss is calculated for the reconstruction result of each group:

[0093]

[0094] Where q represents a point in the k group point cloud reconstruction result , and p represents the point cloud data set the points in the first input point cloud;

[0095] the weighted semantic consistency loss:

[0096] To highlight the group pairs with larger semantic differences in the consistency constraint, the weighted consistency loss between groups is calculated:

[0097]

[0098] wherein, is the normalized weight of the kth group relative to the first group, which is calculated by the following steps:

[0099] 1) Quantify the semantic association strength of each group with the first group

[0100]

[0101] 2) Assign weights based on semantic differences

[0102] (the greater the semantic difference, the higher the weight)

[0103] (normalization)

[0104] wherein, is the normalized weight, is a small constant to prevent division by zero;

[0105] The above strategy of quantifying the semantic association strength between groups and dynamically adjusting the consistency loss weight according to it realizes a dynamic weight adjustment mechanism based on semantic differences and semantic similarity quantitative analysis between groups, ensuring that the model pays more attention to feature learning with larger semantic span.

[0106] wherein MSE is the mean square error loss, and the calculation principle is as follows:

[0107]

[0108] wherein, denotes the first input point cloud denotes the second input point cloud.

[0109] Step 3-2 specifically includes:

[0110] Based on the similarity matrix S, the G feature tokens are divided into K semantic related groups, and the grouping process follows the semantic aggregation principle:

[0111] First, randomly select the first group as the semantic anchor point: randomly select G / K tokens from the G feature tokens as the first group:

[0112]

[0113] wherein, is a set of randomly selected indices;

[0114] Then, based on semantic similarity, subsequent grouping is performed to obtain semantic grouping: the semantic similarity of the remaining tokens with the first group is calculated, and a similarity ranking matrix is constructed:

[0115]

[0116] According to the ranking from high to low similarity, grouping is performed according to the following strategy:

[0117] Second group: select tokens with the first rank of similarity with each token in the first group

[0118] Third group: select tokens with the second rank of similarity with each token in the first group

[0119] Fourth group: select the remaining tokens

[0120] Finally, K semantic groups are obtained:

[0121] Step 4 is specifically:

[0122] Intelligently fuse the K-group parallel mask pre-training results obtained in step 3-3, ensure that the feature information of different semantic perspectives is fully integrated, and weight the fused semantic enhanced features output by the K-group encoder to input the PointMamba architecture for multi-scale feature extraction, which integrates semantic enhanced information and spatial geometric information, providing a rich feature basis for subsequent point cloud segmentation tasks, and capturing long-range dependencies through a selective state space model.

[0123] Advantages:

[0124] 1. Semantic-aware grouping mechanism: The semantic grouping strategy based on cosine similarity can effectively identify the semantic association between features, ensuring that the features within the same group have strong semantic consistency, laying a solid foundation for subsequent parallel processing.

[0125] 2. Parallel mask training innovation: The innovative parallel mask strategy achieves 100% data utilization while maintaining a high mask ratio of 75%, while enhancing the model's ability to learn different semantic perspective features.

[0126] 3. Dynamic weight adjustment mechanism: The dynamic weight adjustment strategy based on inter-group semantic similarity makes the model pay more attention to groups with larger semantic differences, strengthening the relevance and robustness of feature learning and improving the model's ability to understand complex semantic relationships.​

[0127] 4. Multi-path parallel pre-training: The multi-path parallel mask autoencoder architecture enables the model to learn feature representations from multiple semantic perspectives simultaneously, improving the understanding and modeling of complex scenes. BRIEF DESCRIPTION OF DRAWINGS

[0128] Figure 1 For the overall flowchart of the present application.

[0129] Figure 2 For the parallel mask effect diagram.

[0130] Figure 3 For the reconstruction effect comparison diagram. DETAILED DESCRIPTION

[0131] The scene of the present embodiment is automatic driving road obstacle detection, and the detected object is a flowerpot:

[0132] As shown in Figure 1 :

[0133] Step 1, obtain the obstacle point cloud by scanning, calculate the geometric centroid of the obstacle point cloud, translate the point cloud coordinate system to make the centroid at the origin (0, 0, 0), then calculate the distance of the farthest point from the origin after translation, and scale the entire point cloud to a unit sphere according to the distance, thereby obtaining a standardized point cloud; This processing ensures that point clouds of different sizes have consistent proportions and reference frames in subsequent processing;

[0134] Step 2, local feature extraction and encoding: extract representative center points from the standardized point cloud by farthest point sampling, construct local point cloud groups using the K-nearest neighbor algorithm, and extract initial feature representations using a shared convolutional network to obtain a tokens sequence of local point cloud group features with rich spatial information;

[0135] Step 3, semantic-aware parallel mask pre-training: divide the local point cloud group feature tokens sequence into K semantic-related groups based on cosine similarity calculation, implement an innovative parallel mask strategy for each group, dynamically adjust the loss weight based on the semantic similarity between groups, and pre-train through a multi-path parallel mask autoencoder framework to obtain semantic-enhanced feature representations;

[0136] Step 4, multi-scale feature fusion: input the semantic-enhanced feature representations into the PointMamba architecture, extract multi-scale features through a selective state space model, and further enhance the spatial perception and feature expression capabilities of the point cloud;

[0137] Step 5, precise point cloud segmentation: based on the multi-scale features output in step 4, a point cloud segmentation network is constructed, which can fully utilize the semantic enhanced features for precise segmentation, predict the class probability for each point, and then optimize the segmentation network parameters through the cross-entropy loss function to realize the precise distinction between the road surface and the obstacle region.

[0138] Step 1 is specifically:

[0139] Step 1-1, given point cloud dataset , wherein each point , i represents an index representing a point, used to uniquely identify each point in the point cloud, taking a value range of 1 to N; N represents the total number of points in the point cloud, respectively represent the x coordinate, y coordinate and z coordinate value of the i-th point in the three-dimensional space, and the centroid of the point cloud is calculated as:

[0140]

[0141] and all point cloud data is translated to a coordinate system with the centroid as the origin, obtaining the translated point cloud :

[0142]

[0143] Step 1-2, calculate the distance of the farthest point from the origin in the translated point cloud:

[0144]

[0145] wherein, is the distance from the farthest point to the origin, is the Euclidean norm of point ;

[0146] Step 1-3, scale the entire translated point cloud to a unit sphere to obtain the standardized point cloud :

[0147] .

[0148] Step 2 is specifically: feature extraction combining FPS sampling, KNN clustering and zigzag sorting:

[0149] Step 2-1, given the standardized point cloud set , FPS sampling algorithm is used to sample G representative center points :

[0150] A point is randomly selected from the dataset as the first center point ;

[0151]

[0152] Select each subsequent center point by FPS sampling ;

[0153]

[0154] wherein, denotes the set of selected center points, denotes any point in the point cloud dataset, denotes selecting the point farthest from the existing set of center points, denotes for each point , calculating the distance between it and all selected center points , selecting the farthest point as the next center point ;

[0155] Then for each center point , finding its nearest neighbor points in the original point cloud, forming a local point cloud group , which is a three-dimensional tensor:

[0156] ,

[0157] wherein B denotes the number of point cloud samples processed simultaneously in one training, G denotes the number of representative center points sampled from each point cloud by FPS sampling, and M denotes the number of neighbor points found for each center point by KNN (K-Nearest Neighbor) algorithm;

[0158] Step 2-2, spatially serializing the local point cloud group :

[0159]

[0160] wherein ZigzagSort() is a spatial sorting function that realizes the ordered arrangement of point clouds by constructing a continuous path in different planes, denotes the normalization processing of the point cloud after KNN;

[0161] Step 2-3, inputting the serialized local point cloud group into a shared convolutional neural network for feature encoding to form an initial feature sequence with spatial sequence information :

[0162]

[0163]

[0164] wherein, is the d-dimensional feature vector of the i-th local point cloud group after being encoded by the convolutional network, which contains the geometric structure information and spatial texture features of the local region.

[0165] Step 3 is specifically:

[0166] Step 3-1, the feature sequence is L2 norm normalized:

[0167]

[0168] wherein, the L2 norm of the unfeatured vector , the cosine similarity matrix S between the normalized feature vectors is calculated:

[0169]

[0170] The element in the similarity matrix S represents the cosine similarity between the i-th feature vector and the j-th feature vector, and the value range is , the closer the value is to 1, the more similar the two feature vectors are; Step 3-2, the intelligent grouping strategy based on semantic similarity divides the G feature tokens into K semantic related groups ;

[0171] Step 3-3, a multi-path parallel mask autoencoder architecture training framework is constructed to realize the simultaneous training of K parallel mask tasks, and a semantic enhanced feature representation is obtained through pre-training. This framework implements a parallel mask strategy on the original semantic related groups, and constructs a parallel mask learning task for each of the K semantic related groups. In each task, only one group is visible, and the remaining K-1 groups are masked.

[0172] The parallel mask strategy implemented on the original semantic related groups in step 3-3:

[0173] For the k-th semantic related group, the mask strategy is defined as:

[0174] The visible part of the k-th group is only the self-group:

[0175] The mask part of the k-th group is that all other groups are masked:

[0176] The mask part of the k-th group is that all other groups are masked: ;

[0177] The mask autoencoder architecture training framework described in step 3-3 consists of: a shared Mamba encoder network, a position encoding network, a multi-path parallel decoder network, and a mask token processing module.​

[0178] The mask token processing module performs feature filling work on the masked positions, and the core component of the module is a learnable mask token parameter, which is automatically optimized in the training process and used as a unified feature representation for all masked positions. When the mask strategy removes the original features of certain point cloud groups, the module uses the learnable mask token parameter to fill the missing positions with features, ensuring that the subsequent decoder can receive a complete input sequence.

[0179] Step 3-3 specifically includes:

[0180] Step 3-3-1, construct K parallel mask tasks: for each semantic group in k=1, 2,..., K, input the visible tokens sequence of the group as input, while setting the other K-1 groups to mask state, K parallel tasks are performed simultaneously, and the respective visible tokens sequences are input into the shared Mamba encoder network to obtain global features through selective state space mechanism learning long program sequence dependencies :

[0181]

[0182] Step 3-3-2, the representative center point coordinates extracted by farthest point sampling (FPS) in step 2-1 are used as position encoding inputs. To maintain the spatial geometric information of the point cloud, the corresponding center point coordinates are converted into position features through a position encoding network:

[0183]

[0184] is the center point coordinate , and the position encoding vector obtained after processing by the position encoding network has the same dimension d as the local point cloud group feature tokens ( extracted in step 2-3, both of which are d-dimensional;

[0185] where the position encoding subset of the kth mask strategy is ;

[0186] The position encoding network uses a two-layer fully connected structure:

[0187]

[0188] where GELU is the Gaussian Error Linear Unit activation function, which ensures effective encoding of position information;

[0189] Step 3-3-3, the decoder network adopts a multi-layer Mamba state space module for sequence modeling, and the decoder network receives K sets of global features output by the encoder respectively Prior information of the position corresponding to the masked part , through decoding processing to obtain the point cloud reconstruction results of K sets of parallel mask autoencoders :

[0190] .

[0191] The decoding process includes the following steps:

[0192] 1) Mask token embedding layer: for the information missing part of the mask part , the learnable mask token parameters fill in the features of each mask position, and then add the filled mask token to the corresponding position code to obtain a mask token embedding vector that integrates spatial position information;

[0193] Output the mask token embedding vector that integrates position information, which provides the basic input of the mask area for the decoder;

[0194] Because the original features of the mask part are not input into the encoder, there is information missing, so for each mask position, i.e. the token index position, use a learnable mask token to fill it in, and through parallel processing of K mask perspectives, each perspective corresponds to a set of mask position filling and reconstruction;

[0195] 2) Mamba decoding layer: the visible part global features output by the encoder are spliced with the mask token embedding vector obtained in the previous step to form a complete token sequence, and a multi-layer Mamba module is used to capture long-distance dependencies;

[0196] 3) Coordinate prediction layer: map the features decoded by the Mamba decoding layer to three-dimensional space coordinate values to obtain the reconstruction results of each mask part, and finally output the reconstructed point cloud coordinates.

[0197] Step 3-2 specifically includes:

[0198] Based on the similarity matrix S, the G feature tokens are divided into K semantically related groups, and the grouping process follows the semantic aggregation principle:

[0199] First, randomly select the first group as the semantic anchor: randomly select G / K tokens from the G feature tokens as the first group:

[0200]

[0201] wherein, is a set of randomly selected indexes;

[0202] Then, based on semantic similarity, subsequent grouping is performed to obtain semantic grouping: the semantic similarity of the remaining tokens and the first group is calculated, and a similarity ranking matrix is constructed:

[0203]

[0204] According to the ranking from high to low similarity, the tokens are grouped according to the following strategy:

[0205] The second group: select the tokens with the first rank of similarity with each token in the first group

[0206] The third group: select the tokens with the second rank of similarity with each token in the first group

[0207] The fourth group: select the remaining tokens

[0208] Finally, K semantic groups are obtained: In the embodiment, K is 4;

[0209] As shown in Figure 2 , Branch_1, Branch_2, Branch_3, Branch_4 represent four groups, Branch_1 when the first group is a visible point, Branch_2, Branch_3, Branch_4 are mask groups, and the figure shows that each group masks 75% of the point cloud and retains 25% of the point cloud, which intuitively shows the mask coverage of each group.

[0210] The total loss function used in the multi-path parallel mask autoencoder architecture described in step 3-3 is:

[0211]

[0212] wherein, the multi-level loss function includes Chamfer distance reconstruction loss , weighted semantic consistency loss , and consistency is the weight coefficient of the consistency loss;

[0213] The Chamfer distance reconstruction loss is:

[0214] The bidirectional Chamfer distance loss is calculated for the reconstruction result of each group:

[0215]

[0216] wherein q represents a point belonging to the k-th group of point cloud reconstruction results wherein p represents a point in the point cloud dataset ;

[0217] The weighted semantic consistency loss is:

[0218] In order to highlight the group pairs with larger semantic differences in the consistency constraint, the weighted consistency loss between groups is calculated:

[0219]

[0220] wherein, is the normalized weight of the k-th group relative to the first group, which is calculated by the following steps:

[0221] 1) Quantify the semantic association strength of each group with the first group

[0222]

[0223] 2) Assign weights based on semantic differences

[0224] The greater the semantic difference, the higher the weight

[0225] Normalization processing

[0226] wherein, is the normalized weight, is a small constant to prevent division by zero;

[0227] The above strategy of quantifying the semantic association strength between groups and dynamically adjusting the consistency loss weight according to it realizes a dynamic weight adjustment mechanism based on semantic differences and semantic similarity quantification analysis between groups, ensuring that the model pays more attention to feature learning with larger semantic span;

[0228] wherein MSE is the mean square error loss, and the calculation principle is as follows:

[0229]

[0230] wherein, represents the first input point cloud represents the second input point cloud.

[0231] Step 4 is specifically:

[0232] Intelligently fuse the K groups of parallel mask pre-training results obtained in step 3-3 to ensure that the feature information of different semantic perspectives is fully integrated, and the semantic enhanced features output by the K group encoders are weighted and fused to input the PointMamba architecture for multi-scale feature extraction.

[0233] As Figure 3 The reconstruction effect contrast chart shown in the figure can clearly show that the reconstruction of the small obstacle is more complete, and the obstacle can be better distinguished.

[0234] The experimental data show that the classification accuracy on the ModelNet40 dataset is improved by 0.3%, the accuracy on the PB-T50-RS dataset is improved by 0.5%, and the mIoU of the segmentation task on the ShapeNetPart dataset is improved by 0.8%.

[0235] The method is particularly suitable for an automatic driving scene, can accurately identify and segment small obstacles, effectively reduces missed detection and false detection, improves the safety and reliability of the system, and has important engineering application value and broad market prospect.

[0236] The application provides a point cloud segmentation method based on a semantic perception parallel mask strategy, and there are many methods and ways to realize the technical scheme, and the above description is only the preferred embodiment of the application, and it should be pointed out that, for ordinary technical personnel in the technical field, some improvements and refinements can be made without departing from the principle of the application, and these improvements and refinements should be regarded as the protection scope of the application. The components not explicitly described in the embodiment can be realized by using the prior art.

Claims

1. A point cloud segmentation method based on semantic-aware parallel mask strategy, characterized in that, The steps comprise the following: Step 1, calculate the geometric center of the obstacle point cloud, translate the point cloud coordinate system to make the center at the origin (0, 0, 0), then calculate the distance of the farthest point from the origin in the translated point cloud, and scale the entire point cloud to the unit sphere accordingly, to obtain a standardized point cloud; Step 2, perform farthest point sampling on the standardized point cloud to extract representative center points, use the K-nearest neighbor algorithm to construct local point cloud groups, and extract initial feature representations through a shared convolutional network to obtain a tokens sequence of local point cloud group features with rich spatial information; Step 3, divide the local point cloud group feature tokens sequence into K semantically related groups based on cosine similarity calculation, implement a parallel mask strategy for each group, dynamically adjust the loss weight based on inter-group semantic similarity, and pre-train through a multi-path parallel mask autoencoder framework to obtain semantically enhanced feature representations; Step 4, input the semantically enhanced feature representations into the PointMamba architecture, and extract multi-scale features through a selective state space model; Step 5, based on the multi-scale features output in step 4, construct a point cloud segmentation network to predict the class probability for each point, then optimize the segmentation network parameters through a cross-entropy loss function to achieve accurate classification of obstacle point clouds.

2. The point cloud segmentation method based on semantic perception and parallel mask strategy according to claim 1, characterized in that, Step 1 is specifically: Step 1-1, given point cloud dataset where each point , i represents an index representing a point, used to uniquely identify each point in the point cloud, with a value range of 1 to N; N represents the total number of points in the point cloud, respectively represent the x-coordinate, y-coordinate and z-coordinate values of the i-th point in the three-dimensional space, and the centroid of the point cloud is calculated: ； Translate all the point cloud data to the coordinate system with the centroid as the origin, and obtain the translated point cloud : ； Step 1-2, calculate the distance of the farthest point from the origin in the translated point cloud: ； wherein is the distance from the farthest point to the origin, is the Euclidean norm of the point . Step 1-3, scale the whole translated point cloud into a unit sphere to get the normalized point cloud : 。 3. The point cloud segmentation method based on semantic perception and parallel mask strategy according to claim 1, characterized in that, Step 2 is specifically: feature extraction combined with FPS sampling, KNN clustering, and zigzag sorting: Step 2-1, given the set of normalized point clouds , G representative center points are sampled from the set using the FPS sampling algorithm : Select a point at random from the dataset as the first center point ;​ ； Selecting each subsequent center point by FPS sampling ; ； wherein, represents the set of selected center points, represents an arbitrary point in the point cloud dataset, represents selecting the point farthest from the existing set of center points, represents for each point , calculating the distance between it and all selected center points , selecting the point farthest as the next center point ; Then for each center point , find its nearest , neighbors in the original point cloud, constituting a local point cloud group , which is a three-dimensional tensor: , ； Where B represents the number of point cloud samples processed simultaneously in one training, G represents the number of representative center points sampled from each point cloud through FPS sampling, and M represents the number of neighbor points found for each center point through the KNN algorithm; Step 2-2, for local point cloud groups Spatial serialization processing: ； Wherein, ZigzagSort() is a spatial sorting function, which realizes the ordered arrangement of point cloud by constructing continuous path in different planes, represents the normalization processing of the point cloud after KNN. Step 2-3, the serialized local point cloud group input a shared convolutional neural network for feature encoding to form an initial feature sequence with spatial sequence information : ； ； wherein, is the d-dimensional feature vector of the i-th local point cloud group after convolutional network coding, which contains the geometric structure information and spatial texture features of the local region.

4. The point cloud segmentation method based on semantic perception and parallel mask strategy according to claim 1, characterized in that, Step 3 is specifically: Step 3-1, for the feature sequence L2 norm normalization: ； wherein, uncharacteristic vector L2-norm of the uncharacteristic vector, a cosine similarity matrix S between the normalized characteristic vectors is calculated: ； The element in the similarity matrix S represents the cosine similarity between the i-th feature vector and the j-th feature vector, and the value range is . The cosine similarity between the i-th feature vector and the j-th feature vector is represented, and the value range is . The value closer to 1 indicates that the two feature vectors are more similar. Step 3-2, the intelligent grouping strategy based on semantic similarity divides the G feature tokens into K semantically related groups ; Step 3-3, construct a multi-path parallel mask autoencoder architecture training framework to simultaneously train K parallel mask tasks and obtain semantically enhanced feature representations through pre-training. This framework implements a parallel mask strategy on the original semantically related groups and constructs K parallel mask learning tasks for the K semantically related groups. In each task, only one group is visible, and the remaining K-1 groups are masked.

5. The point cloud segmentation method based on semantic perception and parallel mask strategy according to claim 4, characterized in that, The parallel mask strategy implemented on the original semantically related groups in step 3-3 is: For the kth semantically related group, define its mask strategy as: The visible part of the kth group, i.e. only the own group is visible: ; The mask part of the kth group, i.e. the other groups, are masked: ; The mask autoencoder architecture training framework in step 3-3 consists of a shared Mamba encoder network, a position encoding network, a multi-path parallel decoder network, and a mask token processing module; The mask token processing module performs feature filling for masked positions. The core component of this module is a learnable mask token parameter that is automatically optimized during training to serve as a unified feature representation for all masked positions. When the mask strategy removes the original features of certain point cloud groups, this module uses the learnable mask token parameter to fill in the missing positions, ensuring that the subsequent decoder can receive a complete input sequence.

6. The point cloud segmentation method based on semantic perception and parallel mask strategy according to claim 5, characterized in that, Step 3-3 specifically includes: Step 3-3-1, construct K parallel mask tasks: for each semantic group in k = 1, 2,..., K, set the visible tokens sequence of the group as input, while setting the other K-1 groups as mask state, K parallel tasks are performed simultaneously, and the respective visible tokens sequence is input into the shared Mamba encoder network to obtain global features through selective state space mechanism learning long program sequence dependencies : as input, while setting the other K-1 groups as mask state, K parallel tasks are performed simultaneously, and the respective visible tokens sequence is input into the shared Mamba encoder network to obtain global features through selective state space mechanism learning long program sequence dependencies : ； Step 3-3-2, coordinates of representative center points extracted by farthest point sampling in step 2-1 As the input of position encoding, to keep the spatial geometric information of the point cloud, the corresponding center point coordinates are converted into position features through the position encoding network: ； is the center point coordinate is the position encoding vector obtained after the position encoding network processing, which has the same dimension d as the local point cloud group feature tokens extracted in step 2-3 is the position encoding vector obtained after the position encoding network processing, which has the same dimension d as the local point cloud group feature tokens extracted in step 2-3 wherein the position encoding subset of the kth group of mask strategies is ; The position encoding network uses a two-layer fully connected structure: ； Wherein, GELU is a Gaussian error linear unit activation function, which ensures the effective encoding of position information; Step 3-3-3, the decoder network adopts a multi-layer Mamba state space module for sequence modeling, and the decoder network respectively receives the global features of the K groups of encoder outputs the position prior information of the corresponding masked part , by decoding processing to obtain the point cloud reconstruction results of the K groups of parallel mask autoencoders : 。 7. The point cloud segmentation method based on semantic perception and parallel mask strategy according to claim 6, characterized in that, The The decoding process comprises the following steps: 1) Mask token embedding layer: for the information missing part of the mask part The learnable mask token parameters fill in the features for each mask position, and then add the filled mask token to the corresponding position encoding to obtain a mask token embedding vector that integrates spatial position information; The mask token embedding vector outputted by fusing the position information is used as the basic input of the decoder for the mask region; Since the original features of the mask part are not input into the encoder, there is information loss, so for each mask position, i.e. the token index position in the token, a learnable mask token is used for filling, and by processing K mask perspectives in parallel, each perspective corresponds to the filling and reconstruction of a group of mask positions. 2) Mamba decoding layer: the visible part global feature outputted by the encoder is spliced with the mask token embedding vector obtained in the previous step to form a complete token sequence, and a multi-layer Mamba module is used to capture long-distance dependency; 3) Coordinate prediction layer: the feature decoded by the Mamba decoding layer is mapped to a three-dimensional space coordinate value, and the reconstruction result of each mask part is obtained, and finally the reconstructed point cloud coordinates are outputted.

8. The point cloud segmentation method based on semantic perception and parallel mask strategy according to claim 4, characterized in that, Step 3-2 specifically comprises: Based on the similarity matrix S, the G feature tokens are divided into K semantically related groups, and the grouping process follows the semantic aggregation principle: First, randomly select the first group as the semantic anchor: randomly select G / K tokens from the G feature tokens as the first group: ； wherein is a randomly selected set of indices; Then, the subsequent grouping is performed based on the semantic similarity to obtain the semantic grouping: the semantic similarity between the remaining tokens and the first group is calculated, and a similarity ranking matrix is constructed: ； According to the similarity from high to low, the following strategy is used for grouping: Second group: select the tokens with the first similarity ranking of each token in the first group Third group: select the tokens with the second similarity ranking of each token in the first group Fourth group: select the remaining tokens K semantic groups are finally obtained: .

9. The point cloud segmentation method based on semantic perception and parallel mask strategy according to claim 8, characterized in that, The total loss function used in the training of the multi-path parallel mask autoencoder architecture in step 3-3 is: ； wherein the multi-level loss function comprises a Chamfer distance reconstruction loss , a weighted semantic consistency loss , and consistency is a weight coefficient of the consistency loss. The Chamfer distance reconstruction loss: The bidirectional Chamfer distance loss is calculated for the reconstruction result of each group: ； wherein q denotes a point belonging to the k-th group of point cloud reconstruction results wherein p denotes a point in the point cloud dataset ​ The weighted semantic consistency loss: In order to highlight the groups with larger semantic differences in the consistency constraint, the weighted consistency loss between groups is calculated: ； wherein, is the normalized weight of the kth group with respect to the first group, which is calculated by the following steps: 1) Quantify the semantic association strength of each group with the first group ； 2) Assign weights based on semantic differences The greater the semantic difference, the higher the weight Normalization wherein, is the normalized weight, is a small constant to prevent division by zero; The above strategy of quantifying the semantic association strength between groups and dynamically adjusting the consistency loss weight based on it realizes the dynamic weight adjustment mechanism based on semantic differences and the semantic similarity quantitative analysis between groups, ensuring that the model pays more attention to feature learning with larger semantic span; Wherein, MSE is the mean square error loss, and the calculation principle is as follows: ； wherein, represents the first input point cloud represents the second input point cloud.

10. The point cloud segmentation method based on semantic perception and parallel mask strategy according to claim 1, characterized in that, Step 4 specifically comprises: Intelligently fuse the K group parallel mask pre-training results obtained in step 3-3, ensure that the feature information of different semantic perspectives is fully integrated, and input the weighted fused semantic enhanced features output by the K group encoders into the PointMamba architecture for multi-scale feature extraction.

Images