A weakly supervised point cloud semantic segmentation method based on consistent projection constraint

Abstract

The application discloses a weakly supervised point cloud semantic segmentation method based on consistency projection constraint, comprising: data preprocessing and weak ground truth labeling with different labeling proportions; projecting the point cloud subjected to two different data enhancement methods to a two-dimensional plane, and establishing a consistency projection constraint by calculating the mean square error; random sampling; extracting deeper feature vectors in the local neighborhood of each encoding layer by using a global feature aggregation module; completing sub-scene boundary contrast optimization by minimizing a sub-scene boundary contrast loss function on four encoding layers; querying spatially adjacent point features; interpolating the query point features; inputting the feature vectors obtained by splicing the interpolated point features into three MLPs to complete the weakly supervised point cloud semantic segmentation task. The application solves the problems of overfitting, underfitting, information loss and boundary blur caused by sparse point cloud ground truth labeling in weakly supervised point cloud semantic segmentation, thereby improving the segmentation accuracy.

Description

Technical Field

[0001] This invention relates to the field of point cloud processing technology, and in particular to a weakly supervised point cloud semantic segmentation method based on consistent projection constraints. Background Technology

[0002] In recent years, the application of deep learning-based methods for processing point clouds has yielded increasing results both domestically and internationally. Point cloud semantic segmentation, in particular, refers to the task of assigning each point in a point cloud dataset to its corresponding semantic category. In a point cloud, each point possesses location and attribute information and can represent objects or scenes in space. The goal of point cloud semantic segmentation is to classify or label each point in the point cloud based on its geometric and semantic features, thereby achieving semantic understanding and segmentation of objects, scenes, or environments.

[0003] Currently, deep learning-based point cloud semantic segmentation methods are categorized into fully supervised and weakly supervised methods based on whether they rely on fully labeled point cloud datasets for training. Unlike fully supervised methods, which require accurate labels for every point, weakly supervised point cloud semantic segmentation uses partially or incompletely labeled point cloud data for semantic segmentation, leveraging less labeling information or other auxiliary data. Existing weakly supervised methods employ various techniques to improve the segmentation ability of network models, broadly categorized into three types: consistency regularization, contrastive pre-training, and pseudo-label generation. While these methods have been proposed and achieved some success in specific scenarios, weakly supervised point cloud semantic segmentation typically only obtains partial point cloud labeling information, leading to noise in the training data and impacting the model's learning and generalization abilities.

[0004] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a weakly supervised point cloud semantic segmentation method based on consistent projection constraints, which aims to solve the problem that the weakly supervised point cloud semantic segmentation in the prior art can usually only obtain partial point cloud annotation information, which leads to noise in its training data, thereby affecting the model's learning and generalization ability.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: Firstly, a weakly supervised point cloud semantic segmentation method based on consistent projection constraints is presented, with the following specific steps: Step 1: Data preprocessing: Perform preprocessing operations on the input point cloud, and on the basis of random sparse annotation, perform weak ground truth annotation on the preprocessed point cloud with different annotation ratios. Step 2, Consistent Projection Constraints: Perform two different data augmentations on the preprocessed point cloud: perform random mirror or rotation transformation and perform both mirror and rotation transformations simultaneously. Project the two augmented point cloud data onto a two-dimensional plane and establish consistent projection constraints by calculating the mean of their squared differences. Step 3, Random Sampling: The point cloud input to the network model is downsampled using random sampling to reduce the density of the point cloud; Step 4, Global Feature Aggregation: The global feature aggregation module, as the encoding layer, extracts deeper feature vectors in the local neighborhood of each encoding layer by stacking two sets of local feature extraction modules, thereby obtaining aggregated features on the four encoding layers. Step 5, Sub-scene boundary comparison optimization: Minimize the sub-scene boundary comparison loss function on the four encoding layers to maximize the similarity of the feature vector representations of two boundary points; Step 6: Query spatial neighbor features: Given a query point p, search for the K nearest neighbor points to the query point in each of the four coding layers based on the Euclidean distance between the points; Step 7: Interpolate query point features: Use the trilinear interpolation method to calculate the feature vector of query point p, and then concatenate the four hierarchical feature vectors together. Step 8: Infer the semantic category of the point cloud: Input the interpolated and concatenated feature vector into three MLPs to obtain the predicted category of the point cloud, thus completing the weakly supervised point cloud semantic segmentation task.

[0007] Furthermore, in step one, the preprocessing operations for the point cloud include loading point cloud data, resampling the point cloud, dividing the point cloud, and extracting features, and performing weak ground truth annotations on the preprocessed point cloud with different annotation ratios.

[0008] Furthermore, in step two, the mirror transformation is to invert the X-axis, that is, to take the negative value of the Y-coordinate of the point cloud; the rotation transformation is to rotate the point cloud around the Y-axis by an angle. ,in It is in (0, Random angles uniformly distributed within the data; point cloud augmentation. And point cloud The projections are respectively projected onto the XoY plane, and consistent projection constraints are applied on the X and Y axes. That is, the mean squared error loss is calculated on both the X and Y axes simultaneously. The consistent projection loss function is shown in formula (1): (1) In the formula, The number of 3D point clouds, , Point clouds Projection on the X and Y axes , Point clouds Projection onto the X and Y axes.

[0009] Furthermore, in step four, the local feature extraction module first uses the K-nearest neighbor algorithm, i.e., based on point-by-point Euclidean distance, to construct the feature information of neighboring points to obtain the center point feature information. Then, the features of neighboring points are... Enhanced point semantic features are obtained through channel attention mechanism, while the coordinates of neighboring points are also included. The spatial features of the augmented point are obtained by encoding the local location, and then concatenated with the semantic features of the augmented point to obtain the spatial semantic features of the augmented point. Finally, attention pooling is used to aggregate the features of each neighboring point in the local neighborhood to the center point.

[0010] Furthermore, in step four, the channel attention mechanism weights the features of each point within each channel, making the points related to that channel more prominent, resulting in enhanced point semantic features. As shown in formula (2): (2) In the formula, through and The attention score is obtained by subtracting the two values ​​to measure similarity, and then normalized. Obtain the weight coefficient matrix between single-point channels and The values ​​are multiplied by a matrix to calculate their own attention level, and finally combined with the point features. Perform the summation operation by adding the elements together.

[0011] Furthermore, in step five, the boundary contrast loss function is as shown in formula (3): (3) Point cloud The i-th point is denoted as The corresponding truth label is Its local neighborhood is The set of truth labels for boundary points is , To compare the temperature coefficients in the learning process; For point eigenvectors, for Positive feature vectors obtained through data augmentation and feature extraction For negative pairs of eigenvectors, that is... Midpoint Feature vector representations with different semantics The cosine similarity between the two vectors was measured.

[0012] The truth labels of the boundary points are obtained in each downsampling process in step three, as shown in formula (4): (4) In the formula, Indicates average pooling. Indicates the previous stage The local neighborhood; Finally, using formula (3), the sub-scene boundary contrast loss with boundary point ground truth labels is calculated from the boundary point features obtained in the global feature aggregation module. This is to complete the sub-scene boundary comparison optimization.

[0013] Furthermore, in step eight, the class probabilities output by the third MLP are used to calculate the weighted cross-entropy loss. By introducing weight parameters, the network can achieve better segmentation results even when the number of samples is extremely imbalanced. The loss function is shown in formula (5): (5) In the formula, Represents the set of sample labels. This represents the semantic label of the sample represented using one-hot encoding. Indicates that the sample belongs to the first The probability of a class These are weighting parameters, and their calculation formula is: , Indicates the total number of samples. Indicates that the semantic tag belongs to The number of samples in the class; The total loss function used during training As shown in formula (6): (6) in For weighted cross-entropy loss, For consistent projection loss, For sub-scene boundary contrast loss, use hyperparameters. and Control the weight ratio among the three loss functions.

[0014] The technical solution adopted in this invention has the following beneficial effects: 1. In this application, the method learns the consistency of point cloud data after augmentation, providing additional constraints to compensate for the deficiencies of incomplete annotation, promoting the model to learn more accurate and consistent semantic representations, and improving the model's generalization ability and robustness. Furthermore, since the consistency projection constraint is calculated in a two-dimensional plane, it is more efficient than high-dimensional space computation, which can accelerate training speed and improve computational efficiency, making it more suitable for weakly supervised point cloud semantic segmentation networks.

[0015] 2. This invention employs a channel attention mechanism in the feature extraction section to help the model focus more on channels with important semantic information, thereby enhancing the model's representation and perception capabilities in segmentation tasks. This invention helps the model better understand boundary information in point clouds and learn feature representations to distinguish boundary points, thus addressing the more pronounced edge blurring problem in weakly supervised point cloud semantic segmentation and improving the segmentation accuracy of boundary regions. Attached Figure Description

[0016] Figure 1 A flowchart illustrating the implementation of a weakly supervised point cloud semantic segmentation method based on consistent projection constraints provided by this invention. Figure 2 The network structure diagram of a weakly supervised point cloud semantic segmentation method based on consistent projection constraints provided by the present invention is shown below. Figure 3 A schematic diagram of the global feature aggregation module provided by this invention; Figure 4 A schematic diagram of the local feature extraction module provided by the present invention; Figure 5 A schematic diagram of the channel attention mechanism provided by the present invention; Figure 6 The image shows the prediction results of a weakly supervised point cloud semantic segmentation method based on consistent projection constraints provided by this invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0018] It should be noted that when a component is referred to as being "fixed to" or "set on" another component, it can be directly on or indirectly on that other component. When a component is referred to as being "connected to" another component, it can be directly connected to or indirectly connected to that other component.

[0019] It should also be noted that the same or similar reference numerals in the accompanying drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the accompanying drawings are only for illustrative purposes and should not be construed as limiting the present patent. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0020] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0021] like Figure 1 As shown, a weakly supervised point cloud semantic segmentation method based on consistent projection constraints specifically includes the following steps: Step 1: Data Preprocessing: Preprocessing operations for point clouds include loading point cloud data, point cloud resampling, point cloud partitioning, and feature extraction. First, the input point cloud dataset is loaded by reading the point cloud file and parsing the coordinates and attribute information of the points, loading the point cloud into memory for subsequent processing. Next, a regular voxel grid-based method is used for point cloud resampling. The aim is to uniformly sample the dense points of the original point cloud into a sparser point set, reducing computational complexity and improving efficiency. Second, point cloud partitioning is performed, dividing the resampled point cloud into local neighborhoods or blocks so that each region can be computed independently in subsequent processing. Finally, a convolutional neural network is used to extract features from the point cloud. Based on the point cloud partitioning, features need to be extracted from each local neighborhood or block to obtain a semantically meaningful representation for efficient and accurate segmentation.

[0022] Secondly, random sparse annotations are performed on the preprocessed point cloud. The goal of weakly supervised point cloud semantic segmentation is to obtain accurate prediction results with the lowest possible annotation cost. Therefore, different weak ground truth annotations are performed on the point cloud and then trained to study the performance of the network model under weakly supervised learning.

[0023] Step 2: Consistent Projection Constraints: such as Figure 2 As shown, firstly, the point cloud... Two different data augmentations were performed: one was a random mirror or rotation transformation, which resulted in an augmented point cloud. The enhanced point cloud is obtained by simultaneously performing mirroring and rotation transformations. The mirror transformation performed is to invert the X-axis, that is, to take the negative value of the Y-coordinate of the point cloud. The mirror transformation is shown in formula (1): (1) Rotation transformation is the point cloud rotating around the Y-axis by an angle. ,in It is in (0, The random angles uniformly distributed within the range are rotated as shown in formula (2): (2) The obtained 3D point cloud The expression formula is or 3D point cloud The expression formula is .

[0024] Subsequently, the enhanced point cloud And point cloud Projected onto the XoY plane, as shown in formula (3): (3) in For point clouds Projection on the X-axis For point clouds Projection onto the Y-axis; For point clouds Projection on the X-axis For point clouds Projection onto the Y-axis.

[0025] Finally, the point cloud was analyzed on the X and Y axes respectively. And point cloud The projection of the object is subject to a consistent projection constraint, that is, the mean squared error loss is calculated on both the X-axis and Y-axis. The consistent projection loss function is shown in formula (4): (4) In the formula, The number of 3D point clouds, , Point clouds Projection on the X and Y axes , Point clouds Projection onto the X and Y axes.

[0026] Step 3: Random sampling: such as Figure 2As shown, the point feature extractor retains 25% of the points in the point cloud after four random samplings, meaning the number of 3D point clouds at each level is respectively ( Furthermore, the feature dimension of the point cloud is increased to (32, 128, 256, 512) one by one. At the same time, in order to facilitate subsequent point feature query, the network always retains the corresponding point position coordinate information for each layer feature vector.

[0027] Step 4: Global Feature Aggregation: such as Figure 3 As shown, the global feature aggregation module stacks two sets of local feature extraction modules. First, it aggregates the input point features... The dimensionality is reduced to half of its original value using an MLP (Multi-Level Processing) module, and then input along with the coordinates of neighboring points in the local neighborhood into the first local feature extraction module. The resulting local features are then... The coordinates of local neighboring points are used as input to the second local feature extraction module, and the resulting features are processed by MLP to obtain the local enhanced features. Then, the input point features The dimension of the feature is doubled by MLP and added to the local enhancement features. Finally, the feature is activated by the ReLU function to obtain the aggregated feature. .

[0028] like Figure 4 As shown, the local feature extraction module in the global feature aggregation module first uses the K-nearest neighbor algorithm, which is based on point-by-point Euclidean distance, to construct the feature information of neighboring points to obtain the central point feature information. Then, the features of neighboring points are... Enhanced point semantic features are obtained through channel attention mechanism, while the coordinates of neighboring points are also included. The spatial features of the augmented point are obtained by encoding the local location, and then concatenated with the semantic features of the augmented point to obtain the spatial semantic features of the augmented point. Finally, attention pooling is used to aggregate the features of each neighboring point in the local neighborhood to the center point.

[0029] like Figure 5 As shown, the channel attention mechanism makes points related to that channel more prominent by weighting the features of each point within each channel. This adaptively adjusts the features of each point in a single channel, enhancing the discriminative power of local features and resulting in enhanced point semantic features. As shown in formula (5): (5) In the formula, through and The attention score is obtained by subtracting the two values ​​to measure similarity, and then normalized. Obtain the weight coefficient matrix between single-point channels and The values ​​are multiplied by a matrix to calculate their own attention level, and finally combined with the point features. Perform the summation operation by adding the elements together.

[0030] like Figure 4 As shown, to compensate for the loss of geometric information in the extraction of local neighborhood information, local location encoding is used to enhance the spatial features of points. A random sampling algorithm is used to select a set of points in the point cloud as the center points, denoted as . Then, the K-nearest neighbor algorithm is used to find the K nearest neighbors to the center point and their positional relationships are encoded, resulting in enhanced point spatial features. As shown in formula (6): (6) In the formula, The coordinates of the center point, The coordinates of the nearest points to the center point This means subtracting the coordinates of points to obtain the relative coordinates of the center point and its neighboring points. It is expressed as the Euclidean distance between the center point and its neighboring points. This indicates a splicing operation.

[0031] Finally, attention pooling is used to aggregate the point feature set. By learning to focus on important parts of local features, more accurate local neighborhood information is obtained. First, the enhanced point spatial semantic features are processed by a shared MLP, then a normalization function is used to calculate the attention score of each neighboring point, and finally, a weighted sum is obtained to obtain the feature. As shown in formula (7): (7) In the formula, This refers to the enhanced point spatial semantic features obtained through channel attention mechanism and local position encoding. For weight hyperparameters, The attention score for each point within its local neighborhood.

[0032] Step 5: Sub-scene boundary contrast optimization: For a boundary point in the point cloud, we encourage its learned representation to be more similar to its neighbors from the same category and more distinct from its neighbors from different categories. That is, we use the boundary contrast loss function to supervise the learning of the boundary of the point cloud. The loss function is shown in Equation (8): (8) Point cloud The i-th point is denoted as The corresponding truth label is Its local neighborhood is The set of truth labels for boundary points is , To compare the temperature coefficient in the learning process. For point eigenvectors, for Positive feature vectors obtained through data augmentation and feature extraction For negative pairs of eigenvectors, that is... Midpoint Feature vector representations with different semantics. The cosine similarity between the two vectors was measured and divided by... This is to restrict its positive and negative pairs to the local neighborhood of the point boundary.

[0033] The truth labels of the boundary points are obtained in each downsampling process in step three, as shown in formula (9): (9) In the formula, Indicates average pooling. Indicates the previous stage The local neighborhood.

[0034] Finally, using formula (8), the sub-scene boundary contrast loss with boundary point ground truth labels is calculated from the boundary point features obtained in the global feature aggregation module. This is to complete the sub-scene boundary comparison optimization.

[0035] Step Six: Query the spatial neighbor features: such as Figure 2 As shown, given a query point p containing location coordinates, the K nearest neighbor algorithm is used to search for the K nearest neighbors of the query point in four coding layers based on the Euclidean distance between the points.

[0036] Step 7: Interpolation query point characteristics: such as Figure 2 As shown, for each coding layer feature, trilinear interpolation is applied in three dimensions based on the feature vectors of the query point and its K nearest neighbors, as well as the distance weights between the points, to obtain the feature vector of point p. The interpolation distance weights for each neighboring point are equal. The interpolated features obtained from the four layers are then concatenated to represent the features of all relevant points in the entire point cloud.

[0037] Step 8: Infer the semantic category of the point cloud: such as Figure 2As shown, the feature vector of the query point p, after interpolation and concatenation, is input into three MLPs to infer the semantic category of the point cloud. First, it is input into the first MLP for processing, and the resulting point feature dimension is 128. Next, the output features of the first MLP are further processed, and the resulting point feature dimension is 64. Finally, the third MLP is used to predict the category of the output of the second MLP. Its output layer uses the softmax activation function, which can map the obtained feature vector to each possible semantic category and output the probability distribution of each category, thereby selecting the category with the highest probability to determine the semantic category of the point. Among them, the network uses the category probability obtained from the third MLP to calculate the weighted cross-entropy loss, and uses the backpropagation strategy to optimize the network parameters. The weighted cross-entropy loss function is shown in formula (10): (10) In the formula, Represents the set of sample labels. This represents the semantic label of the sample represented using one-hot encoding. Indicates that the sample belongs to the first The probability of a class These are weighting parameters, and their calculation formula is: , Indicates the total number of samples. Indicates that the semantic tag belongs to The number of samples in a class is used as a weight parameter to enable the network to achieve better segmentation results when the number of samples is extremely imbalanced.

[0038] The total loss function used during training As shown in formula (11): (11) in For weighted cross-entropy loss, For consistent projection loss, For sub-scene boundary contrast loss, use hyperparameters. and Control the weight ratio among the three loss functions.

[0039] This invention performs weakly supervised point cloud semantic segmentation tests on an Area-5 input to a weakly supervised point cloud semantic segmentation network based on consistent projection constraints, using ground truth annotations of 0.1% and 1% on the S3DIS dataset. The prediction results for the conference room scene are as follows: Figure 6As shown, from left to right, the images display the original point cloud, ground truth results, prediction results of the training set with 0.1% ground truth annotation, and prediction results of the training set with 1% ground truth annotation. Observation reveals that the network in this paper can effectively segment categories such as "door," "table," and "chair" under both different weak ground truth annotations. It can also effectively segment edge regions, such as "ceiling" and "floor" categories in the image. Furthermore, as seen in the segmentation of the picture frame on the wall in the two prediction result images, the segmentation prediction result with 1% ground truth annotation is superior to that with 0.1% ground truth annotation.

[0040] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the solutions disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

Claims

1. A weakly supervised point cloud semantic segmentation method based on consistent projection constraints, characterized in that, The specific steps are as follows: Step 1: Data preprocessing: Perform preprocessing operations on the input point cloud, and on the basis of random sparse annotation, perform weak ground truth annotation on the preprocessed point cloud with different annotation ratios. Step 2, Consistent Projection Constraints: Perform two different data augmentations on the preprocessed point cloud: perform random mirror or rotation transformation and perform both mirror and rotation transformations simultaneously. Project the two augmented point cloud data onto a two-dimensional plane and establish consistent projection constraints by calculating the mean of their squared differences. Step 3, Random Sampling: The point cloud input to the network model is downsampled using random sampling to reduce the density of the point cloud; Step 4, Global Feature Aggregation: The global feature aggregation module, as the encoding layer, extracts deeper feature vectors in the local neighborhood of each encoding layer by stacking two sets of local feature extraction modules, thereby obtaining aggregated features on the four encoding layers. Step 5, Sub-scene boundary comparison optimization: Minimize the sub-scene boundary comparison loss function on the four encoding layers to maximize the similarity of the feature vector representations of two boundary points; Step 6: Query spatial neighbor features: Given a query point p, search for the K nearest neighbor points to the query point in each of the four coding layers based on the Euclidean distance between the points; Step 7: Interpolate query point features: Use the trilinear interpolation method to calculate the feature vector of query point p, and then concatenate the four hierarchical feature vectors together. Step 8: Infer the semantic category of the point cloud: Input the interpolated and concatenated feature vector into three MLPs to obtain the predicted category of the point cloud, thus completing the weakly supervised point cloud semantic segmentation task.

2. The weakly supervised point cloud semantic segmentation method based on consistent projection constraints according to claim 1, characterized in that, In step one, the preprocessing operations for the point cloud include loading point cloud data, resampling the point cloud, dividing the point cloud, and extracting features, and then performing weak ground truth annotations on the preprocessed point cloud with different annotation ratios.

3. The weakly supervised point cloud semantic segmentation method based on consistent projection constraints according to claim 1, characterized in that, In step two, the mirror transformation is to invert the X-axis, that is, to take the negative value of the Y-coordinate of the point cloud; the rotation transformation is to rotate the point cloud around the Y-axis by an angle. ,in It is in (0, Random angles uniformly distributed within the data; point cloud augmentation. And point cloud The projections are respectively projected onto the XoY plane, and consistent projection constraints are applied on the X and Y axes. That is, the mean squared error loss is calculated on both the X and Y axes simultaneously. The consistent projection loss function is shown in formula (1): （1） In the formula, The number of 3D point clouds, , Point clouds Projection on the X and Y axes , Point clouds Projection onto the X and Y axes.

4. The weakly supervised point cloud semantic segmentation method based on consistent projection constraints according to claim 1, characterized in that, In step four, the local feature extraction module first uses the K-nearest neighbor algorithm, which is based on point-by-point Euclidean distance, to construct the feature information of neighboring points for the center point feature information. Then, the features of neighboring points are... Enhanced point semantic features are obtained through channel attention mechanism, while the coordinates of neighboring points are also included. The spatial features of the augmented points are obtained by encoding the local location, and then concatenated with the semantic features of the augmented points to obtain the spatial semantic features of the augmented points. Finally, attention pooling is used to aggregate the features of each neighboring point in the local neighborhood to the center point.

5. A weakly supervised point cloud semantic segmentation method based on consistent projection constraints according to claim 4, characterized in that, In step four, the channel attention mechanism weights the features of each point within each channel, making the points related to that channel more prominent, resulting in enhanced point semantic features. As shown in formula (2): （2） In the formula, through and The attention score is obtained by subtracting the two values ​​to measure similarity, and then normalized. Obtain the weight coefficient matrix between single-point channels and The values ​​are multiplied by a matrix to calculate their own attention level, and finally combined with the point features. Perform the summation operation by adding the elements together.

6. The weakly supervised point cloud semantic segmentation method based on consistent projection constraints according to claim 3, characterized in that, In step five, the boundary contrast loss function is shown in formula (3): （3） Point cloud The i-th point is denoted as The corresponding truth label is Its local neighborhood is The set of truth labels for boundary points is , To compare the temperature coefficients in the learning process; For point eigenvectors, for Positive feature vectors obtained through data augmentation and feature extraction For negative pairs of eigenvectors, that is... Midpoint Feature vector representations with different semantics The cosine similarity between the two vectors was measured. The truth labels of the boundary points are obtained in each downsampling process in step three, as shown in formula (4): （4） In the formula, Indicates average pooling. Indicates the previous stage The local neighborhood; Finally, using formula (3), the sub-scene boundary contrast loss with boundary point ground truth labels is calculated from the boundary point features obtained in the global feature aggregation module. This is to complete the sub-scene boundary comparison optimization.

7. The weakly supervised point cloud semantic segmentation method based on consistent projection constraints according to claim 6, characterized in that, In step eight, the class probabilities output by the third MLP are used to calculate the weighted cross-entropy loss. By introducing weight parameters, the network can achieve better segmentation results even when the number of samples is extremely imbalanced. The loss function is shown in formula (5): （5） In the formula, Represents the set of sample labels. This represents the semantic label of the sample represented using one-hot encoding. Indicates that the sample belongs to the first The probability of a class These are weighting parameters, and their calculation formula is: , Indicates the total number of samples. Indicates that the semantic tag belongs to The number of samples in the class; The total loss function used during training As shown in formula (6): （6） in For weighted cross-entropy loss, For consistent projection loss, For sub-scene boundary contrast loss, use hyperparameters. and Control the weight ratio among the three loss functions.

Images