Weak supervision point cloud semantic segmentation method based on correlation enhancement

Abstract

The invention discloses a weak supervision point cloud semantic segmentation method based on correlation enhancement, and the method comprises the steps: carrying out the feature extraction of down-sampled data through a Stratified encoder, and capturing the local and global features of a point cloud through a layered Transform structure through the Stratified encoder. Then the obtained features are input into a category incidence matrix module, and a correlation matrix is obtained through calculation. According to the weak supervision point cloud semantic segmentation method based on correlation enhancement, MLP operation is utilized to enrich representation of a correlation matrix, correlation enhancement operation is stacked for multiple times, and finally final feature representation of each point is obtained, so that a model is effectively propagated between labeled data and unlabeled data through enhanced correlation information. In a weak supervision environment, the method can effectively learn and generalize unseen data from limited annotation data, so that the performance of point cloud semantic segmentation is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a weakly supervised point cloud semantic segmentation method based on correlation enhancement. Background Art

[0002] With the development of autonomous driving and robotics, understanding and analyzing three-dimensional spatial environments has become increasingly important. Point clouds, a data structure that accurately describes the shape and position of objects in three-dimensional space, consist of a series of points in a three-dimensional coordinate system, each containing coordinate information. This representation provides rich geometric detail and spatial information and is widely used in fields such as computer vision, robot navigation, terrain mapping, and augmented reality. Point cloud semantic segmentation plays a crucial role in these applications. Semantic segmentation involves assigning each point in a point cloud a specific class label, such as person, vehicle, or building, enabling machines to understand the semantic information of objects and scenes in the point cloud data. This is crucial for environmental perception, navigation, and path planning. However, high-quality annotated data is expensive to obtain and difficult to obtain on a large scale, which limits the training and performance of point cloud semantic segmentation models. Therefore, weakly supervised point cloud semantic segmentation has emerged as a viable solution. It can effectively reduce the difficulty of annotation and the reliance on large amounts of accurately annotated data, thereby alleviating the cost and workload of annotation.

[0003] Weakly supervised point cloud semantic segmentation is rapidly developing, with researchers proposing a series of innovative methods and approaches, driving rapid progress in this field. Initially, most methods in this field relied on 2D images, training models by mapping annotations from these images to point cloud data. While this approach reduced annotation costs, it was limited by its inability to accurately capture the unique information of 3D point clouds, resulting in poor performance. Subsequently, morphological analysis methods emerged, using morphological properties of point clouds, such as curvature and normal vectors, to perform segmentation. While this approach eliminates the need for labeled data, its limitation lies in its ability to capture only local features of the point cloud and struggles to fully reveal its semantic content. Currently, the mainstream trend in weakly supervised point cloud semantic segmentation is shifting towards deep learning techniques based on point cloud features, leveraging a variety of network architectures and loss functions to optimize models. While several established methods have achieved success in certain application scenarios, the field still faces numerous challenges and unresolved issues. A key issue is insufficient learning, primarily due to the often incomplete annotation information that weakly supervised methods rely on. This incomplete information makes it difficult for the model to fully grasp the global characteristics of the data, especially when processing unlabeled point cloud data. The model often cannot fully utilize the feature information of these points, thus affecting the final performance of the model.

[0004] At the same time, in the field of point cloud semantic segmentation, the uneven distribution of samples across categories is a common problem. This imbalance can cause bias in the model training process, thereby affecting the overall performance of the model. In weakly supervised learning environments where labels are scarce, the problem of data imbalance can be even more serious. The loss function is a key tool for measuring the difference between the accuracy of the model's predictions and the actual labels. Together with the backpropagation algorithm, it constitutes the core mechanism of deep learning training. The choice of loss function has a direct impact on the model's training results and predictive ability, and therefore its importance cannot be ignored. When samples are unevenly distributed, the traditional cross-entropy loss may cause the model to favor predictions for categories with a larger number of samples, while the classification effect for categories with fewer samples may be unsatisfactory. Summary of the Invention

[0005] (1) Technical problems solved In view of the above-mentioned deficiencies in the prior art, the present invention provides a weakly supervised point cloud semantic segmentation method based on correlation enhancement to overcome the above-mentioned deficiencies in the prior art.

[0006] (2) Technical solution To achieve the above objectives, the present invention is implemented through the following technical solutions: a weakly supervised point cloud semantic segmentation method based on correlation enhancement, specifically comprising the following steps: S1, collect the point cloud to be trained as the training set and perform downsampling processing; S2. The downsampled data is subjected to feature extraction using a Stratified encoder. The Stratified encoder captures local and global features of the point cloud through a hierarchical Transformer structure. The obtained features are then input into a category association matrix module to obtain a correlation matrix through calculation. S3, the correlation matrix and the output features of the encoder are input into the correlation enhancement module, which consists of linear attention and MLP to obtain a new feature representation; S4. Apply the convolution layer to process the extracted features to generate the final semantic segmentation results.

[0007] Preferably, in step S4, a back propagation strategy is used to optimize the parameters of the network, and the network parameters are updated according to the value of the loss function, so that the loss function continues to decrease until it converges to a set value, thereby completing the training of the network segmentation model.

[0008] Preferably, in step S2, a hierarchical sampling strategy is applied to the original point cloud data. For each query point (i.e., a labeled point), nearby points are densely sampled, while distant points are sparsely sampled, so that the model can more effectively learn the distribution and characteristics of the data under limited label information, and a Stratified encoder is used to extract features from the sampled points. The Stratified encoder can efficiently extract long-distance context features, thereby providing strong support for subsequent segmentation tasks.

[0009] Preferably, when the Stratified encoder performs feature extraction in step S2, it first aggregates the local neighborhood features of each point through the first layer of point embedding module, and then the data is sent to multiple hierarchical Transformer blocks. In each block, for each query point, nearby points are densely sampled within its window as dense keys, and distant points are sparsely sampled in a larger range as sparse keys to form the final key set. Then, a multi-head self-attention mechanism is executed to enable the query point to pay attention to dense nearby features and sparse distant features at the same time, thereby significantly expanding the effective receptive field and establishing direct long-distance dependencies. In this way, the Stratified encoder can efficiently extract long-distance contextual features, providing strong support for subsequent segmentation tasks.

[0010] Preferably, after obtaining the features output by the encoder in step S2, for each category, key points are selected by farthest point sampling, and then clustering is performed based on the distance between the feature vectors of these points and the feature vectors of all points. Finally, the average feature vector of the point in each cluster is calculated to form the prototype feature of the category, that is, the representative feature of the category. For each point, the cosine similarity between its feature and the prototype feature of each category is calculated to obtain a correlation matrix.

[0011] Preferably, in step S3, the correlation matrix and the complete features output by the encoder are input into the correlation enhancement module, so that the model can simultaneously utilize fine-grained information and significant features, thereby more comprehensively understanding the point cloud data.

[0012] Preferably, linear attention is applied in step S3 to obtain a correlation weighted matrix, and the similarity is adjusted through linear transformation using the information in the correlation matrix to enhance the model's attention to key features, and the data is further processed through MLP to enhance feature representation.

[0013] Preferably, in step S4, the correlation enhancement operation is stacked multiple times to obtain the final feature representation of each point, obtain all relevant point features, and perform convolution decoding operations to obtain the final segmentation result.

[0014] The present invention also provides a weakly supervised point cloud semantic segmentation system based on correlation enhancement, which adopts a weakly supervised point cloud semantic segmentation method based on correlation enhancement. The system includes a Stratified encoder, a category association matrix module and a correlation enhancement module; The Stratified encoder is used for feature extraction, which can efficiently extract long-distance context features, thereby providing strong support for subsequent segmentation tasks; The category association matrix module is used to calculate the correlation between each point and the average features of all categories, thereby generating a rich feature representation for each point, which includes the degree of association between the point and each category.

[0015] Preferably, the correlation enhancement module is used to further enhance the correlation between points and categories through linear attention, highlighting the role of key features, and at the same time using MLP operations to enrich the representation of the correlation matrix, stacking the correlation enhancement operations multiple times, and finally obtaining the final feature representation of each point, so that the model can effectively propagate information between labeled data and unlabeled data through enhanced correlation information, thereby better realizing point cloud semantic segmentation in a weakly supervised environment.

[0016] (3) Beneficial effects This paper provides a weakly supervised point cloud semantic segmentation method based on correlation enhancement. Compared with the existing technology, it has the following advantages: (1) This weakly supervised point cloud semantic segmentation method based on correlation enhancement uses a Stratified encoder to extract features from the downsampled data. The Stratified encoder captures the local and global features of the point cloud through a layered Transformer structure. Afterwards, the obtained features are input into a category correlation matrix module, and the correlation matrix is ​​obtained by calculation. Afterwards, the correlation matrix and the output features of the encoder are input into the correlation enhancement module, which consists of linear attention and MLP stacking to obtain a new feature representation. In a weakly supervised environment, this method can effectively learn from limited annotated data and generalize to unseen data, thereby improving the performance of point cloud semantic segmentation.

[0017] (2) This weakly supervised point cloud semantic segmentation method based on correlation enhancement promotes the backpropagation of gradients, strengthens model convergence, solves the problem of data set imbalance, and further improves the model training effect by jointly using Dice Loss and Focal Loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flowchart for implementing the weakly supervised point cloud semantic segmentation method in an embodiment of the present invention.

[0019] Figure 2 4 is a network structure diagram of the weakly supervised point cloud semantic segmentation model in an embodiment of the present invention.

[0020] Figure 3 1 is a diagram of a Stratified encoder module of a weakly supervised point cloud semantic segmentation model in an embodiment of the present invention.

[0021] Figure 4 4 is a correlation enhancement module diagram of the weakly supervised point cloud semantic segmentation model in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0023] See also Figure 1-4 , an embodiment of the present invention provides a technical solution: a weakly supervised point cloud semantic segmentation method based on correlation enhancement, comprising the following steps: S1, data preprocessing: collect the point cloud to be trained as the training set and perform downsampling processing; In S2, the downsampled data is subjected to feature extraction using a Stratified Encoder. The Stratified Encoder uses a layered Transformer structure to capture local and global features of the point cloud. The obtained features are then fed into a Class Correlation Matrix module, which calculates the correlation matrix.

[0024] S3, the correlation matrix and the output features of the encoder are input into the correlation enhancement module, which consists of linear attention and MLP to obtain a new feature representation; S4 applies a convolutional layer to process the extracted features to generate the final semantic segmentation results. During training, a backpropagation strategy is used to optimize the network parameters. A loss function is used to assist in training. Based on the value of the loss function, the network parameters are updated so that the loss function continues to decrease until it converges to the set value. At this point, training ends and the trained network segmentation model is saved.

[0025] Specifically, this application uses a public dataset as a dataset and divides the dataset into a training set and a test set.

[0026] A hierarchical sampling strategy is applied to the original point cloud data. For each query point (i.e., a labeled point), nearby points are densely sampled, while distant points are sparsely sampled, so that the model can more effectively learn the distribution and characteristics of the data with limited label information.

[0027] The Stratified encoder is used to extract features from the sampled points. The Stratified encoder can efficiently extract long-distance context features, thereby providing strong support for subsequent segmentation tasks.

[0028] When the Stratified encoder performs feature extraction, it first aggregates the local neighborhood features of each point through the first layer of point embedding module. Assume that point p with dot p local area N(p) ,Point embedding can be expressed as: ; in, is a function that converts a point p and its neighboring points n feature combination.

[0029] The data is then fed into multiple hierarchical Transformer blocks, where for each query point q , densely sample nearby points within its window as dense keys K dense （q） , while sparsely sampling distant points in a larger range as sparse keys K sparsc （q） , thus forming the final key set K(q) Next, a multi-head self-attention mechanism is implemented, which enables the query point to simultaneously focus on dense nearby features and sparse long-distance features, thereby significantly expanding the effective receptive field and establishing direct long-distance dependencies.

[0030] After getting the features of the encoder output, for each category c , perform FPS sampling among the labeled points, and sample separately k points. Subsequently, using the sampling point as the initial center, the distance from each point to these seed points is calculated, that is, the Euclidean distance between the feature of each point in the point cloud and the feature of each seed point is calculated. Based on these distances, each point is assigned to the nearest seed point to form a cluster. For each cluster, the average feature value of its member points is calculated to form a prototype feature. That is, assuming P c For category c The prototype feature set, F(Pc ) Point Cloud P c The feature vector of each point in . Obtained by FPS sampling k The eigenvector of a point is: For each point The eigenvector of , calculate the Euclidean distance between it and each seed point feature vector, which is expressed by the following formula: ; in i Indicates the sequence number of the point in the point cloud, j Indicates the sequence number of the seed point. According to these distances, each point Assign to the nearest seed point ,form k clusters S 1 , S 2 ,…, S k .

[0031] For each cluster S j , calculate the feature average of its member points to form the prototype feature: ; in, Represents clustering The number of midpoints. Finally, the k cluster prototype features of each category c are .

[0032] For each point, calculate the cosine similarity between its features and the prototype features of each category to obtain the correlation matrix. C The formula is defined as: ; After performing an MLP on the correlation matrix, it is input into the correlation enhancement module along with the complete features output by the encoder. This allows the model to simultaneously utilize fine-grained information and salient features, thereby achieving a more comprehensive understanding of the point cloud data. Specifically, the correlation matrix obtained in the category correlation matrix module represents the correlation between each query point and all class prototypes. To enhance these correlations, the correlation enhancement module is introduced to exploit two potential relationships. First, query points are correlated and interdependent, and their correlations with all prototypes are also connected in the similarity matrix, resulting in point relationships. Second, in a multi-category setting, the classification of a single point into a specific category depends on the relative correlation between the point and each category prototype, thus forming a relationship between the point and the category. The correlation enhancement module refines these relationships by leveraging the relative correlations between points and categories to enhance classification accuracy.

[0033] In the relevance enhancement module, linear attention is first used to obtain a relevance weighted matrix, and the information in the relevance matrix is ​​used to adjust the similarity through linear transformation to enhance the model's attention to key features. In other words, given the relevance matrix C ,right C Perform linear transformation to obtain Q,K,V , assuming , , Denote the i-th tag vector from Q, K, and V respectively, then the linear attention can be obtained as follows: ; in, is the i-th output vector. is the i-th query vector The result of a linear transformation. are all key vectors The result after linear transformation and the value vector The sum of the dot products of . are all key vectors The sum of the results after linear transformation is used for normalization. , The calculation method of linear attention is different from the standard attention mechanism, which significantly reduces the computational complexity.

[0034] Dependency modules are designed to be stacked L Layer, the input of the module is recorded as , and use the effective transformation between points and categories to enhance the correlation between them. For each point, first calculate its correlation with all other points, and then Rearrange them according to the category dimension as their first dimension, and then calculate the linear attention between the points: ; in, means transposing the first two dimensions, and represents a linear attention layer, represents the dimension of this tensor, where is the number of categories, is the number of points, This calculation strengthens the relationship between points, which can enhance the feature representation of each point and capture richer spatial relationships and structural information, thereby improving the accuracy of classification tasks and enhancing the model's ability to generalize to new data.

[0035] After the attention layer, a multi-layer perceptron is applied to each point to learn complex and nonlinear feature mappings in the features to further enhance the relationship between them. The formula is as follows: ; in, is a multilayer perceptron function that performs a nonlinear transformation on the input. Next, the dimensions are rearranged so that becomes , and linear attention is applied, which is expressed as follows: ; Next, we apply a multi-layer perceptron to Convert to Through the correlation enhancement module, the category association matrix can interact not only in the spatial dimension but also in the category dimension, which enhances the contextual dependency between global features and significantly improves the segmentation performance.

[0036] at last, The decoder decodes the image into the final segmentation result.

[0037] Use training data for training. During the training process, use the back propagation strategy to optimize the network parameters and use the loss function to assist in training. The loss function used is the DF loss function, which is a weighted combination of Dice Loss and Focal Loss to assist in training. Back propagation optimizes the network parameters and solves the problem of data set imbalance.

[0038] The specific loss function can be expressed by the following formula: ; in, is a hyperparameter used to balance the two loss functions. Focal Loss is used to solve the problem of class imbalance by reducing the weight of easy-to-classify samples and increasing the weight of difficult-to-classify samples. In multi-classification tasks, the formula of Focal Loss can be expressed as: ; in, is the model’s predicted probability for the correct category, is the class weight, It is a parameter that adjusts the weight of easy-to-classify samples.

[0039] Dice Loss is used to measure the similarity between the predicted result and the true label, especially in the case of class imbalance. For multi-class segmentation problems, the calculation formula of Dice Loss is: ; in, represents the probability value predicted by the model, Represents the binary value of the true label (0 or 1), and Respectively represent i The predicted and true values ​​of a point. It is a small constant used to prevent the denominator from being zero and ensure the stability of the calculation.

[0040] Dice Loss optimizes classification accuracy by measuring the similarity between the predicted results and the true labels, especially performing well in the case of data imbalance. Focal Loss alleviates the problem of category imbalance by reducing the weight of easy-to-classify samples and increasing the weight of difficult-to-classify samples. The combination of the two not only improves the model's ability to recognize minority categories, but also enhances feature learning, thereby improving the overall performance and stability of the segmentation model.

[0041] According to the value of the loss function, the network parameters are updated so that the loss function continues to decrease until it converges to a smaller value. At this time, the training ends and the trained network model is saved; the saved trained model is used to form a weakly supervised point cloud semantic segmentation network model.

[0042] A weakly supervised point cloud semantic segmentation system based on correlation enhancement includes a Stratified encoder, a category association matrix module and a correlation enhancement module.

[0043] The Stratified encoder is used for feature extraction, which can efficiently extract long-distance context features, thereby providing strong support for subsequent segmentation tasks.

[0044] The category association matrix module is used to calculate the correlation between each point and all category prototype features, thereby generating a rich feature representation for each point, which includes the degree of association between the point and each category.

[0045] The correlation enhancement module is used to further enhance the correlation between points and categories through linear attention, highlighting the role of key features. At the same time, it uses MLP operations to enrich the representation of the correlation matrix and stacks the correlation enhancement operations multiple times to obtain the final feature representation of each point. The model can effectively propagate information between labeled data and unlabeled data through enhanced correlation information, thereby better realizing point cloud semantic segmentation in a weakly supervised environment.

[0046] The present invention is based on a weakly supervised point cloud semantic segmentation method with correlation enhancement. The downsampled data is subjected to feature extraction using a Stratified encoder. The Stratified encoder captures the local and global features of the point cloud through a layered Transformer structure. The obtained features are then input into a category correlation matrix module, and the correlation matrix is ​​obtained by calculation. The correlation matrix and the output features of the encoder are then input into a correlation enhancement module, which consists of linear attention and MLP stacking, to obtain a new feature representation. In a weakly supervised environment, this method can effectively learn from limited annotated data and generalize to unseen data, thereby improving the performance of point cloud semantic segmentation.

[0047] By jointly using Dice Loss and Focal Loss, we can promote the backpropagation of gradients, enhance model convergence, solve the problem of data set imbalance, and further improve the model training effect.

[0048] This application achieved competitive mACC and mIoU results on a set of public point cloud datasets, outperforming several popular weakly supervised point cloud semantic segmentation methods.

[0049] Embodiment: A correlation-enhanced weakly supervised point cloud semantic segmentation method comprises the following steps: S1. Perform necessary data preprocessing on the 3D point cloud data to make it suitable for model training and divide it into a training set and a test set.

[0050] The specific workflow is as follows: (1.1) Two sets of public datasets are used as datasets; (1.2) For the dataset described in step (1.1), use the pointops.furthestsampling function to select the farthest point from the point cloud as the sample center point. This function receives the coordinates xyz of the point cloud, the offset offset, and the new number of samples new_offset as input, and outputs the index idx of the sampling point.

[0051] (1.3) For the sampling index described in step (1.2), extract the sampling points from the original point cloud data, use the pointops.queryandgroup function to aggregate the neighborhood of each sampling point, transform the information of the aggregated points through the linear layer self.linear, and then normalize it through the normalization layer self.norm.

[0052] (1.4) For the sampling points described in step (1.2), the RGB color information is divided by 255 for normalization. The input of the final model is the normalized point information and the category label of the corresponding point.

[0053] S2. Based on the pre-processed data, the sampled data is input into the Stratified encoder for feature extraction. After obtaining the extracted features, the features are input into a category association matrix module to obtain the correlation matrix through calculation. The specific workflow is as follows: (2.1) For the data obtained in step (1.4), a Stratified encoder is introduced for feature extraction. First, the local neighborhood features of each point are aggregated through the first layer of point embedding module. Subsequently, the data is fed into multiple hierarchical Transformer blocks. In each block, for each query point q, the nearby points in its window are densely sampled as dense keys. Kdense(q) , while sparsely sampling distant points in a larger range as sparse keys Ksparsc (q) , thus forming the final key set K （q） Next, a multi-head self-attention mechanism is executed to obtain the feature vector of each point; (2.2) For the feature vector described in step (2.1), for each category c , perform fps sampling among the labeled points, and sample separately k points, and assign them to the nearest seed point based on the Euclidean distance, forming k clusters S 1 , S 2 ,…, S k Each cluster S j Prototype features Aj is the average value of the feature vectors of its member points, thereby obtaining k cluster prototype features of category c For each point, calculate the cosine similarity between its features and the prototype features of each category to obtain the correlation matrix C .

[0054] S3. Design a correlation enhancement module, use linear attention to obtain a correlation weighted matrix, use the information in the correlation matrix to adjust the similarity through linear transformation to enhance the model's attention to key features. After the attention layer, a multi-layer perceptron is applied to each point to learn the complex and nonlinear feature mapping in the features to further enhance the relationship between them. The correlation enhancement module stacks L Layer, the input of the module is recorded as , and at the same time use effective transformations between points and categories to enhance the correlation between them. The specific workflow is as follows: (3.1), for each point, first calculate its correlation with all other points, Rearrange them according to the category dimension as their first dimension, and then calculate the linear attention between the points: ; (3.2) After step (3.1), a multi-layer perceptron is applied to each point to learn the complex and nonlinear feature mapping in the features. ; (3.3), rearrange the dimensions of step (3.3) so that becomes , and applying linear attention, we get , then apply the multi-layer perceptron again to Convert to .

[0055] S4. Finally, The decoder decodes the image into the final segmentation result.

[0056] S5. For the network model described in step S4, during the training process, Dice Loss and Focal Loss are jointly used to promote gradient backpropagation, enhance model convergence, solve the problem of data set imbalance, and further improve the model training effect; S6. For the trained 3D weakly supervised point cloud semantic segmentation model, use the test point cloud as input to obtain the segmentation result. The specific workflow is as follows: (6.1) For the 3D weakly supervised point cloud semantic segmentation model described in step S4, the test set is used as input to obtain the model segmentation result; (6.2) Comparing the category labels obtained by automatic segmentation of the three-dimensional weakly supervised point cloud semantic segmentation model described in step (6.1) with the true value labels, it was found that the three-dimensional weakly supervised point cloud semantic segmentation model described in step (6.1) achieved excellent segmentation results.

[0057] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0058] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0059] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A weakly supervised point cloud semantic segmentation method based on correlation enhancement, characterized by: The specific steps include: S1, collect the point cloud to be trained as a training set and perform downsampling processing; S2. The downsampled data is extracted using a Stratified encoder. The Stratified encoder captures the local and global features of the point cloud through a hierarchical Transformer structure. The obtained features are then input into a category association matrix module to obtain the correlation matrix through calculation. S3, input the relevance matrix and the output features of the encoder into the relevance enhancement module, which consists of linear attention and MLP, to obtain a new feature representation; S4. Apply the convolutional layer to process the extracted features to generate the final semantic segmentation result.

2. The method for weakly supervised point cloud semantic segmentation based on correlation enhancement according to claim 1, characterized in that: In step S4, the back propagation strategy is used to optimize the parameters of the network, and the network parameters are updated according to the value of the loss function, so that the loss function continues to decrease until it converges to the set value, thereby completing the training of the network segmentation model.

3. The method for weakly supervised point cloud semantic segmentation based on correlation enhancement according to claim 1, characterized in that: In step S2, a stratified sampling strategy is applied to the original point cloud data. For each query point, nearby points are densely sampled, while distant points are sparsely sampled, so that the model can more effectively learn the distribution and characteristics of the data under limited label information, and the Stratified encoder is used to extract features from the sampled points. The Stratified encoder can efficiently extract long-distance context features, thereby providing strong support for subsequent segmentation tasks.

4. The method for weakly supervised point cloud semantic segmentation based on correlation enhancement according to claim 1, characterized in that: When the Stratified encoder performs feature extraction in step S2, the local neighborhood features of each point are first aggregated through the first layer point embedding module, and then the data is sent to multiple hierarchical Transformer blocks. In each block, for each query point, nearby points are densely sampled in its window as dense keys, and distant points are sparsely sampled in a larger range as sparse keys to form a final key set, and then a multi-head self-attention mechanism is executed to enable the query point to pay attention to dense nearby features and sparse distant features at the same time.

5. The method for weakly supervised point cloud semantic segmentation based on correlation enhancement according to claim 1, characterized in that: After obtaining the features output by the encoder in step S2, for each category, key points are selected by sampling the farthest points, and then clustering is performed based on the distance between the feature vectors of these points and the feature vectors of all points. Finally, the average feature vector of the points in each cluster is calculated to form the prototype features of the category, that is, the representative features of the category. For each point, the cosine similarity between its features and the prototype features of each category is calculated to obtain a correlation matrix.

6. The method for weakly supervised point cloud semantic segmentation based on correlation enhancement according to claim 1, characterized in that: In step S3, the correlation matrix and the complete features output by the encoder are input into the correlation enhancement module, so that the model can simultaneously utilize fine-grained information and significant features, thereby more comprehensively understanding the point cloud data.

7. The method for weakly supervised point cloud semantic segmentation based on correlation enhancement according to claim 1, characterized in that: In step S3, linear attention is applied to obtain a relevance weighted matrix, and the similarity is adjusted through linear transformation using the information in the relevance matrix to enhance the model's attention to key features, and the data is further processed through MLP to enhance feature representation.

8. The method for weakly supervised point cloud semantic segmentation based on correlation enhancement according to claim 1, characterized in that: In step S4, the correlation enhancement operation is stacked multiple times to obtain the final feature representation of each point, obtain all relevant point features, and obtain the final segmentation result through convolution decoding operation.

9. A weakly supervised point cloud semantic segmentation system based on correlation enhancement, adopting a weakly supervised point cloud semantic segmentation method based on correlation enhancement according to any one of claims 1 to 8, characterized in that: The system includes a Stratified encoder, a category association matrix module, and a relevance enhancement module; The Stratified encoder is used for feature extraction, which can extract long-distance context features, thereby providing strong support for subsequent segmentation tasks; The category association matrix module is used to calculate the correlation between each point and the average features of all categories, so as to generate a rich feature representation for each point, which includes the degree of association between the point and each category.

10. The weakly supervised point cloud semantic segmentation system based on correlation enhancement according to claim 9, characterized in that: The correlation enhancement module is used to enhance the correlation between points and categories through linear attention, highlighting the role of key features, and at the same time using MLP operations to enrich the representation of the correlation matrix, stacking the correlation enhancement operations multiple times, and finally obtaining the final feature representation of each point, so that the model can effectively propagate information between labeled data and unlabeled data through enhanced correlation information, thereby better realizing point cloud semantic segmentation in a weakly supervised environment.

Images