Sparse labeling complex product point cloud semantic segmentation method based on semi-supervised learning

Abstract

The invention discloses a sparse labeling complex product point cloud semantic segmentation method based on semi-supervised learning. Performing preliminary sampling on the complex product point cloud based on a random sampling method to obtain a sparse labeling complex product point cloud, then inputting the sparse labeling complex product point cloud into a semi-supervised semantic segmentation model for training, and outputting a semantic feature and semantic type prediction result of each point in the complex product point cloud; inputting the point-by-point semantic features and the prediction result into an active learning module, and determining unlabeled points in a specified proportion for labeling; and if the training rounds do not exceed five rounds, updating the sparse labeling complex product point cloud to perform semi-supervised training again. And repeating the above process until five rounds of training are completed, and finally outputting a point cloud segmentation result of the complex product. According to the method, the requirement of large-scale point-by-point point cloud marking in point cloud semantic segmentation can be greatly reduced, manual operation is greatly reduced, the marking time is saved, and the requirement of modern industrial production is met.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of point cloud segmentation, and particularly relates to a sparse labeling complex product point cloud semantic segmentation method based on semi-supervised learning. BACKGROUND

[0002] Point cloud semantic segmentation is a key task in three-dimensional content understanding, which assigns a semantic label to each point in the three-dimensional point cloud. This technology has been widely applied in various industries, such as spraying complex engine surfaces into multiple spraying areas through semantic segmentation, and designing different spraying trajectories for these areas to achieve the required spraying quality. In autonomous driving, point cloud semantic segmentation has been widely used to perceive the external environment.

[0003] In the past decade, with the development of deep learning, point cloud segmentation methods based on deep learning have shown excellent performance. They label the semantic labels of each point in the point cloud, and then use a fully supervised deep learning model to train and predict the corresponding semantic label of each point in the point cloud.

[0004] However, in industrial applications, large-scale semantic labels are usually difficult to obtain. The number of points on the surface of a complex product often exceeds 100,000, and the number of points in the entire dataset can easily reach the order of magnitude of millions. Obtaining such a large number of semantic labels is a time-consuming and labor-intensive task, and therefore the demand for dense labels severely limits the practical industrial application of point cloud semantic segmentation.

[0005] Therefore, how to reduce the demand for dense point-by-point semantic labels of the point cloud semantic segmentation model, and use sparse labeled point clouds for training and prediction to reduce the consumption of manpower and time, is a very important and meaningful challenge. SUMMARY

[0006] To solve the above problems, the application provides a sparse labeling complex product point cloud semantic segmentation method based on semi-supervised learning, which reduces the demand for dense point-by-point semantic labels and saves manpower and time.

[0007] The technical scheme adopted by the application is as follows:

[0008] A sparse labeling complex product point cloud semantic segmentation method based on semi-supervised learning, the method comprising the following steps:

[0009] 1) Based on a random sampling method, a sparse labeling complex product point cloud is obtained by preliminary sampling of the complex product point cloud;

[0010] 2) The sparse labeling complex product point cloud is input into a semi-supervised semantic segmentation model for training, and the semantic features and semantic category prediction results of each point in the complex product point cloud are output;

[0011] 3) input the point-wise semantic features and the predicted probability distribution into the active learning module to determine a specified proportion of unlabelled points for labelling;

[0012] 4) repeat steps 2) and 3) until five rounds of training are completed, and finally output the complex product point cloud segmentation result.

[0013] In the step 1), the complex product is an engine, and the specific process of sparse labelling of the complex product point cloud is as follows:

[0014] Given an initial labelling ratio ratio, the number of points to be labelled is determined as N l =N×ratio according to the number of points N in the point cloud; then, N l points are randomly selected from the complex product point cloud, and the semantic label is determined through manual labelling, while the remaining majority of points in the complex product point cloud are not marked with a semantic label; finally, a preliminarily labelled sparse labelled complex product point cloud is obtained.

[0015] The point cloud is manually labelled according to the spraying operation requirement of the engine surface.

[0016] In the step 2), the specific steps of training the semi-supervised semantic segmentation model are as follows:

[0017] 2.1) combine the sparse labelled complex product point cloud to generate a disturbed sparse labelled complex product point cloud, specifically, the combined disturbance includes overall disturbance, point-wise disturbance, and attribute disturbance;

[0018] The step 2.1) is specifically as follows:

[0019] The input sparse labelled complex product point cloud is denoted as The point cloud includes 3D coordinates p i (x i , y i , z i ) and color c i (r i , g i , b i ), wherein i∈{1, 2, …, N};

[0020] The overall disturbance is applied to the 3D coordinates P of the sparse labelled complex product point cloud, including sequentially performing random scaling T S ∈R 3 and random rotation T r ∈R 3 ; the random scaling T S is randomly selected in [0.9, 1.1], and the scaled sparse labelled complex product point cloud is denoted as P s =P·T s, P represents the 3D coordinates of the point cloud; the rotation angle around the z-axis in the random rotation is randomly selected from [0, 2π], and the rotation angles around the x-axis and the y-axis are randomly selected from [-π / 64, π / 64], and the rotated sparse labeled complex product point cloud is represented as P r = P · T r ;

[0021] The 3D coordinates P of the complex product point cloud after applying the overall interference are subjected to point-by-point interference, and the sparse labeled complex product point cloud subjected to point-by-point interference is marked as P d = P + T d · m, where T d ∈ R N×3 is a random displacement field obtained by convolving Gaussian noise on the 3D coordinates of the point cloud, and m is the point-by-point disturbance amplitude;

[0022] The color C of the complex product point cloud after applying the point-by-point interference is subjected to point-by-point interference, including color translation T ct ∈ R 3 , color flipping T cc ∈ R N×3 and color jitter T cj ∈ R N×3 ; wherein the color translation T ct is randomly selected from [-0.1 × 255, 0.1 × 255], and the translated color attribute is represented as C ct = C + T ct ; the color attribute subjected to color flipping is represented as C cc = (1 - ε) × C + ε × T cc , wherein ε = 0.5 is a control factor, and T cc is obtained by scaling the color attribute C to [0, 255]; color jitter is obtained by applying Gaussian noise with amplitude A = 0.05 × 255 and variance δ = 1.0, and the color attribute subjected to color jitter is expressed as C cj = C + T cj ;

[0023] By applying the above three kinds of interference, the disturbed sparse labeled complex product point cloud is finally obtained.

[0024] 2.2) Use the sparse labeled complex product point cloud and the disturbed sparse labeled complex product point cloud to train the dual teacher network and the student network respectively, obtain the combined training loss, and the two teacher models in the dual teacher network and the student model all use the same network structure but assign different network model parameters, and the dual teacher network teaches the student model through the prediction results of the network.

[0025] The step 2.2) is specifically:

[0026] The dual-teacher network model first outputs semantic features S(χ), then reduces the number of channels of the semantic features to the number of semantic categories through a multi-layer perceptron, marked as F(χ) = MLP(S(χ)), and finally converts the model output into a probability distribution of semantic prediction through a SoftMax operation, i.e. P(χ) = SoftMax(F(χ)). In order to make the prediction results of the dual-teacher network more credible, the supervision loss of the two teacher models on the labeled points is used as an auxiliary loss to fuse the prediction results of the two teacher models. Among them, the two supervision auxiliary losses L aux (χ L ,θ t1 ) and L aux (χ L ,θ t2 ) is calculated by the following two formulas:

[0027]

[0028] where χ L is a set of labeled points, θ t1 and θ t2 are the model parameters of the two teacher models, L CE is the cross entropy loss function, p t1 (·)=σ(f t1 (·)) and p t2 (·)=σ(f t2 (·)) is the predicted probability distribution of the two teacher models, f t1 (·) and f t2 (·) is the output of the two teacher models, σ(·) is the SoftMax operation, and y i is the marked point x i The true label of the teacher model; the confidence weights c1 and c2 are calculated by the following two formulas:

[0029]

[0030] The teacher model with smaller supervised auxiliary loss is assigned a larger confidence weight, while the teacher model with larger supervised auxiliary loss is assigned a smaller confidence weight, so the confidence-weighted prediction probability distribution of the dual-teacher network is calculated as follows:

[0031] p cw (χ)=σ(c1×f t1 (χ)+c2×f t2 (χ))

[0032] Based on the confidence-weighted predicted probability distribution, the predicted semantic category of each unlabeled point is determined to be the semantic category corresponding to the maximum value in the probability distribution, which is defined as the pseudo label of the unlabeled point; then, for all unlabeled points x i ∈χU The unlabeled points whose predicted semantic categories correspond to probabilities greater than a threshold are used to teach the student model, thus obtaining a partial unlabeled point set where χ U is the unlabeled point set;

[0033] The combined training loss function of the student model based on the pseudo-labels predicted by the dual-teacher network for the unlabeled points, the true labels of the labeled points, and the confidence weighted prediction probability distribution of the dual-teacher network is calculated by the following formula:

[0034] L(χ L , χ U , θ s ) = L sup (χ L , θ s ) + L sup (χ U′ , θ s ) + L cons (χ, θ s )

[0035] where L sup (χ L , θ s ) is the supervised loss of the student model on the labeled point set, which is calculated by the following formula:

[0036]

[0037] where θ s is the model parameter of the student model, and p s (·) = σ(f s (·)) is the probability distribution output by the student model; L sup (χ U′ , θ s ) is the supervised loss of the student model on the unlabeled point set, which is calculated by the following formula:

[0038]

[0039] where y' is the pseudo-label of the unlabeled point output by the dual-teacher model; L cons (χ, θ s ) is the loss that measures the consistency between the probability distributions output by the dual-teacher network and the student network, which is calculated by the following formula:

[0040]

[0041] where L KLKullback-Leibler loss function is used to measure the similarity between two probability distributions; based on this combined loss function, the student model calculates the gradient of the loss by the SGD optimizer and updates the network model parameters according to back propagation; then, the model parameters of the teacher model with lower confidence weight in the dual-teacher network are updated by exponential moving average, expressed as:

[0042] θ' t = 0.95 x θ t + 0.05 x θ s

[0043] where θ t is the model parameter of the teacher model with lower confidence weight in the dual-teacher network, θ' t is the updated model parameter; the model parameter of the teacher model with higher confidence weight in the dual-teacher network remains fixed; after several rounds of training, the final output of the student model S S (x) and the probability distribution p s (x) are obtained.

[0044] In step 3), the specific steps of active learning to select the marked points are as follows:

[0045] 3.1) Select valuable candidate marked points from all unmarked points: for each unmarked point in the unmarked point set, first measure the certainty of the model's prediction result by mixed certainty, the more uncertain the model's prediction, the more likely the point will greatly improve the model's prediction accuracy by marking, and arrange the candidate points in descending order of mixed certainty to get a batch of candidate points with low certainty; mixed certainty includes global certainty and local certainty;

[0046] The step 3.1) is specifically as follows:

[0047] The global stability is calculated by the following formula:

[0048] u global (x i ) = p1(x i ) - p2(x i )

[0049] where p1(x i ) and p2(x i ) are the maximum and second maximum values in p S (x i ), respectively;

[0050] The local certainty is calculated by the following process:

[0051] The complex product point cloud is first rasterized to generate a sparse point cloud x G , and for the sparse point cloud xG Each point in the set of candidate points, its probability distribution is calculated by the following formula:

[0052]

[0053] where χ j is the sparse point χ i adjacent to the original points, their probability distributions are aggregated by the probability distribution output in the previous step to obtain the probability distribution of the sparse point; then the local certainty of the sparse point χ i is calculated by the following formula:

[0054] U local (χ i ) = p1(χ i )-p2(χ i )

[0055] The local certainty of the original point is set to the local certainty of the nearest sparse point;

[0056] Therefore, for each unlabeled point, its mixed certainty is calculated by the following formula:

[0057] U(χ i ) = 1.0 x U global (χ i )+0.1 x U local (χ i ).

[0058] 3.2) For the candidate points obtained in step 3.1), the final set of unlabeled points that need to be labeled is obtained by redundancy point suppression, specifically:

[0059] For each point in the candidate points, first search for the labeled points near it, if the candidate point has no adjacent labeled points, it is directly labeled; if there are labeled points near the candidate point, then for each labeled point, the feature similarity between it and the candidate point is calculated by the following formula:

[0060]

[0061] where x c and x j are the candidate point and the labeled point near it, s c and s j are the semantic features of the candidate point and the labeled point near it; if there is no labeled point near the candidate point with a feature similarity greater than 0.8, the candidate point can be labeled, otherwise it is not labeled;

[0062] Therefore, after a sufficient number of unlabeled points are labeled, the sparse labeled complex product point cloud is updated and used for the next round of semi-supervised training.

[0063] In step 4), the specific steps of sparse annotation complex product point cloud semantic prediction are: training and annotation are performed five times by repeating steps 2) and 3), and finally the semantic prediction result is output by the student model.

[0064] The effective effect of the present application is that:

[0065] (1) The present application trains and predicts the sparse annotation complex product point cloud by the semi-supervised semantic segmentation model, outputs the semantic features and semantic category prediction results of each point in the complex product point cloud, can effectively determine the classification boundary in the semantic space, and thus improves the accuracy of point cloud semantic segmentation.

[0066] (2) According to the semantic features and prediction results of the unlabeled points in the complex product point cloud, the present application determines the valuable candidate points in the unlabeled points by active learning, and removes unnecessary annotation points in the candidate points by redundant point suppression. The unlabeled points that can most improve the performance of the point cloud semantic segmentation model are selected under the smallest annotation ratio, which greatly reduces the annotation demand.

[0067] (3) The present application can complete the semantic segmentation task by using the sparse annotation complex product point cloud, greatly reduces the demand for dense point-by-point semantic label, and saves manpower and time. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 is the point cloud semantic segmentation flowchart in the embodiment;

[0069] Figure 2 is a complex engine point cloud example in the embodiment;

[0070] Figure 3 is a point cloud semantic segmentation prediction result graph in the embodiment. DETAILED DESCRIPTION

[0071] In order to better understand the technical scheme of the present application, the present application will be further described in detail below in combination with the drawings and specific examples, Figure 1 is the planning flowchart of the present example, specifically

[0072] (1) The complex product point cloud is preliminarily sampled based on the random sampling method to obtain the sparse annotation complex product point cloud;

[0073] (2) The sparse annotation complex product point cloud is input into the semi-supervised semantic segmentation model for training, and the semantic features and semantic category prediction results of each point in the complex product point cloud are output;

[0074] (3) The point-by-point semantic features and prediction probability distribution are input into the active learning module to determine the specified proportion of unlabeled points for annotation;

[0075] (4) Repeat the above process until five rounds of training are completed, and finally output the complex product point cloud segmentation result.

[0076] In this embodiment, 181 front engine point clouds and 231 back engine point clouds are obtained by shooting using an RGB-D three-dimensional camera, Figure 2 Some point clouds and their semantic annotations are shown.

[0077] In order to train the semi-supervised point cloud semantic segmentation model, the batch_size is set to 16, the learning rate is set to 0.1, the SDG optimizer is used to optimize the model parameters, and the cosine annealing method is used as the learning rate reduction strategy. In the initial labeling, a total of 0.02% of the points are labeled by random sampling to obtain the initial labeled sparse annotated complex product point cloud. In the semi-supervised learning step, the threshold for generating pseudo labels is set to 0.75, and the semi-supervised point cloud semantic segmentation model is trained for 120 rounds each time. In the active learning step, the search radius of the candidate points near the labeled points is set to 0.2m. In each round of active learning process, a total of 0.01% of the unlabeled points are labeled.

[0078] After five rounds of training and unlabeled point labeling, the model accuracy reaches 84.47%. According to the semantic label prediction result of each point output by the student model, the engine point cloud is finally segmented, and some segmentation results are shown in Figure 3 , wherein, Figure 3 (a) and Figure 3 (d) are the original input engine original point cloud, Figure 3 (b) and Figure 3 (e) are the corresponding prediction results, and Figure 3 (c) and Figure 3 (f) are the corresponding true labels. After five rounds of labeling, only 0.06% of the engine point cloud needs to be labeled to achieve very accurate point cloud semantic segmentation results.

[0079] The above detailed the basic principles and main features of the present application in combination with the accompanying drawings. Using the above application, sparse annotated complex product point clouds can be used to complete high-precision semantic segmentation tasks, greatly reducing the need for dense point-by-point semantic labels, saving manpower and time. However, these descriptions cannot be understood as limiting the scope of the present application, the protection scope of the present application is defined by the appended claims, and any modification made on the basis of the claims of the present application is within the protection scope of the present application.

Claims

1. A method for semantic segmentation of sparsely labeled complex product point clouds based on semi-supervised learning, characterized by: The method comprises the following steps: 1) Based on the random sampling method, preliminary sampling of complex product point clouds is performed to obtain sparsely labeled complex product point clouds; 2) Input the sparsely labeled complex product point cloud into the semi-supervised semantic segmentation model for training, and output the semantic features and semantic category prediction results of each point in the complex product point cloud; 3) Input the point-by-point semantic features and predicted probability distribution into the active learning module to determine the specified proportion of unlabeled points for labeling; 4) Repeat steps 2) and 3) until five rounds of training are completed, and finally output the complex product point cloud segmentation results.

2. The method for semantic segmentation of sparsely labeled complex product point clouds based on semi-supervised learning according to claim 1, characterized in that: In step 1), the complex product is an engine, and the specific process of sparsely labeling the point cloud of the complex product is as follows: Given the initial annotation ratio, the number of points to be annotated is determined to be N according to the number of points in the point cloud. l =N×ratio; Then, randomly select N points in the complex product point cloud l points and determine the semantic labels through manual annotation, while the remaining points in the complex product point cloud are not marked with semantic labels; finally, a sparsely labeled complex product point cloud with preliminary labels is obtained.

3. The method for semantic segmentation of sparsely labeled complex product point clouds based on semi-supervised learning according to claim 1, characterized in that: In step 2), the specific steps of training the semi-supervised semantic segmentation model are: 2.1) Performing combined interference on the sparsely labeled complex product point cloud to generate a disturbed sparsely labeled complex product point cloud. Specifically, the combined interference includes overall interference, point-by-point interference, and attribute interference. 2.2) Use sparsely labeled complex product point clouds and perturbed sparsely labeled complex product point clouds to train the dual-teacher network and the student network respectively to obtain the combined training loss. The two teacher models and the student model in the dual-teacher network use the same network structure but are assigned different network model parameters. The dual-teacher network teaches the student model through the network's prediction results.

4. The method for semantic segmentation of sparsely labeled complex product point clouds based on semi-supervised learning according to claim 3, characterized in that: The step 2.1) is specifically as follows: The input sparsely labeled complex product point cloud is recorded as The point cloud includes 3D coordinates p i (x i ,y i , z i ) and color c i (r i , g i , b i ), where i∈{1,2,...,N}; The overall interference is applied to the 3D coordinates P of the sparsely labeled complex product point cloud, including random scaling T in sequence. s ∈R 3 and random rotation T'∈R 3 ; Random scaling T s The scaled sparsely labeled complex product point cloud is randomly selected in [0.9, 1.1] and represented as P s =P·T s ,; The rotation angle around the z axis in the random rotation is randomly selected from [0, 2π], and the rotation angle around the x axis and y axis is randomly selected from [-π / 6, π / 64]. The rotated sparsely labeled complex product point cloud is represented as P r =P·T r ; The 3D coordinates P of the complex product point cloud after the overall interference is applied are interfered point by point, and the sparsely labeled complex product point cloud interfered point by point is marked as P d =P+T d m, where T d ∈R N×3 is the random displacement field obtained by convolving Gaussian noise on the 3D coordinates of the point cloud, and m is the point-by-point disturbance amplitude; Perform point-by-point interference on the color C of the complex product point cloud after point-by-point interference, including sequential color shift T ct ∈R 3 , color flip T cc ∈R N×3 and color dithering T cj ∈R N×3 ; The color shift T ct Randomly select from [-0.1×255, 0.1×255], and the shifted color attribute is represented by C ct =C+T d ; The color attribute that is color-reversed is represented by C cc =(1-ε)×C+ε×T cc , where ε = 0.5 is the control factor, and T cc It is obtained by scaling the color attribute C to [0, 255]. Color dithering is obtained by applying Gaussian noise with amplitude A = 0.05 × 255 and variance δ = 1.

0. The color attribute to which color dithering is applied is expressed as C cj =C+T cj ; By applying the above three types of interference, we finally obtain the disturbed sparsely labeled complex product point cloud.

5. The method for semantic segmentation of sparsely labeled complex product point clouds based on semi-supervised learning according to claim 3, characterized in that: The step 2.2) is specifically as follows: The dual-teacher network model first outputs the semantic feature S(χ), then reduces the number of channels of the semantic feature to the number of semantic categories through a multi-layer perceptron, marked as F(χ) = MLP(S(χ)), and finally converts the model output into a probability distribution of semantic prediction through a SoftMax operation, i.e. P(X) = SoftMax(F(χ)). In order to make the prediction results of the dual-teacher network more credible, the supervision loss of the two teacher models on the labeled points is used as an auxiliary loss to fuse the prediction results of the two teacher models. Among them, the two supervision auxiliary losses L aux (χ L ,θ t1 ) and L aux (χ L ,θ t2 ) is calculated by the following two formulas: where χ L is a set of labeled points, θ t1 and θ t2 are the model parameters of the two teacher models, L CF is the cross entropy loss function, p t1 (·)=σ(f t1 (·)) and P t2 (·)=σ(f t2 (·)) is the predicted probability distribution of the two teacher models, f t1 (·) and f t2 (·) is the output of the two teacher models, σ(·) is the SoftMax operation, and y i is the marked point x i The true label of the teacher model; the confidence weights c1 and c2 are calculated by the following two formulas: The teacher model with smaller supervised auxiliary loss is assigned a larger confidence weight, while the teacher model with larger supervised auxiliary loss is assigned a smaller confidence weight, so the confidence-weighted prediction probability distribution of the dual-teacher network is calculated as follows: p cw (x)=σ(c1×f t1 (x)+c2×f t2 (x)) Based on the confidence-weighted predicted probability distribution, the predicted semantic category of each unlabeled point is determined to be the semantic category corresponding to the maximum value in the probability distribution, which is defined as the pseudo label of the unlabeled point; then, for all unlabeled points x i ∈χ U , the unlabeled points whose probability corresponding to the predicted semantic category is greater than the threshold are used to teach the student model, so a partial set of unlabeled points is obtained where χ U is a set of unlabeled points; Based on the pseudo labels predicted by the dual-teacher network for the unlabeled points, the true labels of the labeled points, and the confidence-weighted prediction probability distribution of the dual-teacher network, the combined training loss function of the student model is calculated as follows: L(x L ,x U ,i s )=L sup (x L ,i s )+L sup (x U ,i s )+L cons (x,θ s ) Among them, L sup (χ L ,θ s ) is the supervision loss of the student model on the set of labeled points, calculated as follows: Where θ s is the model parameter of the student model, p s (·)=σ(f s (·)) is the probability distribution of the student model output; L sup (χ U′ ,θ s ) is the supervision loss of the student model on the set of unlabeled points, calculated as follows: Where y′ is the pseudo label of the unlabeled point output by the dual-teacher model; L cons (χ,θ s ) is the loss that measures the consistency between the probability distributions output by the dual teacher network and the student network, and is calculated as follows: Among them L KL is the Kullback-Leibler loss function, which is used to measure the similarity between two probability distributions. Based on this combined loss function, the student model minimizes the loss by the SGD optimizer to calculate the gradient and update the network model parameters according to back propagation. Then, the model parameters of the teacher model with lower confidence weight in the dual-teacher network are updated by exponential sliding average, which is expressed as: θ′ t =0.95×θ t +0.05×θ s where θ t is the model parameter of the teacher model with lower confidence weight in the dual-teacher network, θ t ′ is the updated model parameter; the model parameters of the teacher model with higher confidence weight in the dual-teacher network remain fixed; after several rounds of training, the final output is the result S of the student model S (χ) and the probability distribution p s (χ).

6. The method for semantic segmentation of sparsely labeled complex product point clouds based on semi-supervised learning according to claim 1, characterized in that: In step 3), the specific steps of actively learning and selecting marker points are: 3.1) Selecting valuable candidate labeled points from all unlabeled points: For each unlabeled point in the set, first measure the model's certainty of its prediction using hybrid certainty. Then, sort the hybrid certainty in descending order to obtain a batch of candidate points with lower certainty. Hybrid certainty includes global certainty and local certainty. 3.2) For the candidate points obtained in step 3.1), the final set of unlabeled points that need to be labeled is obtained by suppressing redundant points, specifically: For each candidate point, first search for nearby marked points. If the candidate point has no nearby marked points, it is directly marked. If there are marked points near the candidate point, then for each marked point, the feature similarity between it and the candidate point is calculated using the following formula: Among them, x c and x j are the candidate points and the marked points near them, s c and s j are the semantic features of the candidate point and the semantic features of the nearby marked points respectively; if there is no marked point near the candidate point and the feature similarity between it and the candidate point is greater than 0.8, then the candidate point can be marked, otherwise it will not be marked; Therefore, after labeling a sufficient number of unlabeled points, the sparsely labeled complex product point cloud is updated and used for the next round of semi-supervised training.

7. The method for semantic segmentation of sparsely labeled complex product point clouds based on semi-supervised learning according to claim 6, characterized in that: The step 3.1) is specifically as follows: Global stability is calculated as follows: U global (x i )=p1(x i )-p2(x i ) where p1(χ i ) and p2(χ i ) are p s (χ i ) the maximum and second largest value; The local certainty is calculated by the following process: The complex product point cloud is first rasterized to generate a sparse point cloud χ G , for sparse point cloud χ G For each point in , its probability distribution is calculated by the following formula: where χ j is a sparse point χ i The adjacent original points; then the sparse point χ is calculated by the following formula i Local certainty of: U local (x i )=p1(x i )-p2(x i ) The local certainty of the original point is set to the local certainty of the nearest sparse point; Therefore, for each unlabeled point, its mixed certainty is calculated as follows: U(x i )=1.0×U global (x i )+0.1×U local (x i )。 8. The method for semantic segmentation of sparsely labeled complex product point clouds based on semi-supervised learning according to claim 1, characterized in that: In step 4), the specific steps of semantic prediction of sparsely labeled complex product point cloud are: training and labeling are repeated five times by repeating steps 2) and 3), and finally the student model outputs the final semantic prediction result.

Images