Point cloud semantic segmentation method based on dynamic uncertainty active learning strategy

Abstract

The application provides a point cloud semantic segmentation method and system based on a dynamic uncertainty active learning strategy, and belongs to the technical field of pavement monitoring. The method comprises the following steps: obtaining a plurality of labeled first super point samples and a plurality of unlabeled second super point samples based on three-dimensional point cloud data samples; training an initial model based on all the first super point samples to obtain a first training model; predicting each second super point sample using the first training model to obtain the actual uncertainty of each second super point sample; screening all the second super point samples based on the actual uncertainty to obtain a plurality of candidate super point samples; constructing a training data set based on all the candidate super point samples; and updating and training the first training model based on the training data set to obtain a point cloud semantic segmentation model. The application improves the class diversity and reduces the labeling cost by intelligently screening and optimizing the labeling sample distribution, thereby significantly improving the segmentation accuracy and generalization ability of the model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pavement monitoring, in particular to a point cloud semantic segmentation method based on a dynamic uncertainty active learning strategy. BACKGROUND

[0002] With the rapid development of automatic driving, urban planning and remote sensing modeling, three-dimensional LiDAR point cloud semantic segmentation technology has become particularly important. LiDAR can obtain high-precision three-dimensional point cloud data by emitting laser pulses and measuring their reflection time, which contains rich spatial and color information. However, the high dimensionality and sparsity of point cloud data make the semantic segmentation task complex and require high computational resources, usually relying on deep learning technology to achieve accurate classification. At present, although the method based on full-supervised learning can provide high-precision segmentation results, its performance is highly dependent on large-scale and high-quality labeled data sets. Due to the tedious and expensive labeling process of three-dimensional point cloud data, the labeling cost is high and the labeled data is limited, which has become the main bottleneck restricting the improvement of model performance.

[0003] In order to solve the problem of lack of labeled data and high cost, active learning method is introduced into the field of point cloud semantic segmentation, which selects the most informative samples for labeling to reduce the required labeling quantity. However, the existing active learning methods still have deficiencies when dealing with large-scale point cloud data. These methods often ignore the spatial distribution and class diversity of samples, resulting in the concentration of labeled samples in certain high-uncertainty regions, while ignoring other important regions, limiting the generalization ability and segmentation accuracy of the model. In addition, the existing methods lack efficient and accurate strategies in the merging and division process of super points, making it difficult to significantly reduce the labeling cost while ensuring the labeling accuracy.

[0004] Therefore, there is an urgent need for an innovative active learning strategy that can optimize the distribution of labeled samples, improve class diversity, and effectively reduce labeling costs, to promote the widespread application and development of three-dimensional point cloud semantic segmentation technology in various application fields. SUMMARY

[0005] The present application provides a point cloud semantic segmentation method based on a dynamic uncertainty active learning strategy, a system, an electronic device and a storage medium, to solve the defects in the prior art and achieve.

[0006] The present application provides a point cloud semantic segmentation method based on a dynamic uncertainty active learning strategy, comprising the following steps:

[0007] Input the three-dimensional point cloud data into the point cloud semantic segmentation model to obtain the semantic class of each point in the three-dimensional point cloud data; wherein the point cloud semantic segmentation model is constructed based on the following way:

[0008] Based on the three-dimensional point cloud data samples, a plurality of first super point samples and a plurality of second super point samples are obtained, the first super point samples are labeled super point samples, and the second super point samples are unlabeled super point samples;

[0009] Based on all the first super point samples, an initial model is trained to obtain a first training model;

[0010] Each of the second super point samples is predicted using the first training model to obtain an actual uncertainty of each of the second super point samples;

[0011] Based on the actual uncertainty, all the second super point samples are screened to obtain a plurality of candidate super point samples;

[0012] Based on all the candidate super point samples, a training data set is constructed;

[0013] Based on the training data set, the first training model is updated and trained to obtain the point cloud semantic segmentation model.

[0014] According to the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy provided by the application, the second super point sample includes a plurality of three-dimensional points; the actual uncertainty of each of the second super point samples is obtained by predicting each of the second super point samples using the first training model, and specifically includes:

[0015] All three-dimensional points included in each of the second super point samples are divided to obtain a first point set and a second point set, and the first point set is a set composed of three-dimensional points with the highest category proportion in the second super point sample;

[0016] The information entropy of each of the three-dimensional points is calculated using the first training model to obtain the first uncertainty of each three-dimensional point in the first point set and the second uncertainty of each three-dimensional point in the second point set;

[0017] Based on all the first uncertainty and all the second uncertainty, the initial uncertainty of each of the second super point samples is calculated;

[0018] Each of the initial uncertainty is multiplied by the category balance weight coefficient of each of the second super point samples to obtain the actual uncertainty of each of the second super point samples, and the category balance weight coefficient of each of the second super point samples is determined based on the proportion of the second super point sample of each category in all category super point samples.

[0019] According to the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy provided by the application, based on the actual uncertainty, all the second super point samples are screened to obtain a plurality of candidate super point samples, and specifically includes:

[0020] In each round of screening, the second super point sample with the highest actual uncertainty is selected as a candidate super point sample, and the actual uncertainty of the remaining second super point samples is dynamically attenuated based on the spatial distance and feature similarity of the candidate super point sample and the remaining second super point samples, until a preset number of candidate super point samples are screened.

[0021] According to the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy provided by the application, a training data set is constructed based on all the candidate super point samples, specifically including:

[0022] A multi-way tree for saving super point neighbor relationships is established;

[0023] Based on the multi-way tree, first neighbor super point samples corresponding to each of the candidate super point samples are selected from all super point samples;

[0024] It is judged whether each of the first neighbor super point samples is similar to the corresponding candidate super point sample;

[0025] Second neighbor super point samples are determined from all the first neighbor super point samples, and the purity value of each of the second neighbor super point samples is calculated, the second neighbor super point samples being similar to the corresponding candidate super point sample;

[0026] Third neighbor super point samples with purity values higher than a preset purity threshold and fourth neighbor super point samples with purity values not higher than the preset purity threshold are determined from all the second neighbor super point samples;

[0027] The fourth neighbor super point samples are divided into a plurality of first sub-super point samples, and second sub-super point samples with purity values higher than a preset purity threshold are determined from all the first sub-super point samples;

[0028] Based on all the candidate super point samples, all the third neighbor super point samples and all the second sub-super point samples, the training data set is constructed.

[0029] According to the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy provided by the application, it is judged whether each of the first neighbor super point samples is similar to the corresponding candidate super point sample, specifically including:

[0030] The first normal vector of the current first neighbor super point sample and the second normal vector of the corresponding candidate super point sample are solved;

[0031] The cosine value of the included angle of the first normal vector and the second normal vector is calculated;

[0032] It is judged whether the included angle cosine value is less than a preset normal vector similarity threshold;

[0033] calculate a color similarity between the current first neighboring super point sample and the corresponding candidate super point sample when the included angle cosine value is less than a preset normal vector similarity threshold value;

[0034] determine whether the color similarity is less than a preset color similarity threshold value;

[0035] determine that the current first neighboring super point sample is similar to the corresponding candidate super point sample when the color similarity is less than the preset color similarity threshold value.

[0036] According to the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy provided by the application, the first training model is updated and trained based on the training data set, and the point cloud semantic segmentation model is obtained, and specifically includes:

[0037] updating and training the first training model based on the training data set to obtain a second training model;

[0038] the second training model is used as the first training model, and the step of using the first training model to predict each second super point sample to obtain the actual uncertainty of each second super point sample is continued, all the second super point samples are screened based on the actual uncertainty, a plurality of candidate super point samples are obtained, a training data set is constructed based on all the candidate super point samples, and the first training model is updated and trained based on the training data set until a preset stop condition is reached, and the point cloud semantic segmentation model is obtained.

[0039] The application also provides a point cloud semantic segmentation system based on a dynamic uncertainty active learning strategy, comprising the following modules:

[0040] The first processing module is used for inputting three-dimensional point cloud data into a point cloud semantic segmentation model to obtain the semantic category of each point in the three-dimensional point cloud data.

[0041] The second processing module is used for obtaining a plurality of first super point samples and a plurality of second super point samples based on a three-dimensional point cloud data sample, wherein the first super point sample is a labeled super point sample, and the second super point sample is an unlabeled super point sample.

[0042] The training module is used for training an initial model based on all the first super point samples to obtain a first training model.

[0043] The second processing module is also used for predicting each second super point sample using the first training model to obtain the actual uncertainty of each second super point sample.

[0044] The second processing module is further configured to filter all the second super point samples based on the actual uncertainty to obtain a plurality of candidate super point samples.

[0045] The second processing module is further configured to construct a training data set based on all the candidate super point samples.

[0046] The training module is further configured to update and train the first training model based on the training data set to obtain the point cloud semantic segmentation model.

[0047] The application further provides an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy according to any one of the above when executing the program.

[0048] The application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy according to any one of the above.

[0049] The application further provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy according to any one of the above.

[0050] To sum up, the one or more technical solutions provided in the embodiments of the application have at least the following technical effects or advantages:

[0051] By dividing the three-dimensional point cloud data samples into a plurality of labeled first super point samples and unlabeled second super point samples, the data is classified and managed efficiently, ensuring that the labeled and unlabeled data are effectively distinguished. By training an initial model based on all labeled first super point samples, a preliminary model with basic semantic recognition capabilities is constructed, which can quickly learn the basic semantic features of the point cloud data. By using the first training model to predict each unlabeled second super point sample and calculating its actual uncertainty, the labeling value of each unlabeled super point is accurately evaluated, ensuring efficient use of labeling resources. By filtering all second super point samples based on actual uncertainty, a plurality of candidate super point samples are obtained, thereby optimizing the allocation of labeling resources and preferentially selecting samples with the most value for model performance improvement. By constructing a training dataset based on all candidate super point samples, high-value and diverse labeled samples are integrated, improving the quality and representativeness of the training data and enhancing the learning effect of the model. By updating and training the first training model based on the training dataset, a point cloud semantic segmentation model with better performance and stronger generalization ability is obtained, significantly improving the segmentation accuracy and robustness of the model in complex urban outdoor scenes. In summary, the present application optimizes the distribution of labeled samples, improves the diversity of classes, and effectively reduces the labeling cost, thereby promoting the widespread application and development of three-dimensional point cloud semantic segmentation technology in various application fields. The active learning strategy not only improves the segmentation accuracy and generalization ability of the model, but also significantly reduces the overall labeling investment through efficient use of labeling resources, ensuring the efficiency and economy of the system in practical applications. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0053] Figure 1 is one of the flowcharts of the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy provided by the present application.

[0054] Figure 2 is the second flowchart of the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy provided by the present application.

[0055] Figure 3 is the third flowchart of the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy provided by the present application.

[0056] Figure 4 is a schematic diagram of ablation experiment results of the SPDU and SPMD modules provided by the present application.

[0057] Figure 5 is a schematic diagram of performance comparison with other active learning methods on the Semantic3D dataset provided by the present application.

[0058] Figure 6 is a schematic diagram of comparison results of different active learning strategies provided by the present application.

[0059] Figure 7 is a schematic diagram of the structure of a point cloud semantic segmentation system based on a dynamic uncertainty active learning strategy provided by the present application.

[0060] Figure 8 is a schematic diagram of the structure of an electronic device provided by the present application. DETAILED DESCRIPTION

[0061] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0062] It should be noted that, in the description of the present application, the terms “comprise”, “contain” or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement “comprises a” does not exclude the presence of another identical element in the process, method, article or device comprising the element. The terms “upper”, “lower” and the like indicate the orientation or positional relationship shown in the drawings, and are only used to facilitate the description of the present application and simplify the description, and do not indicate or imply that the indicated system or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0063] The terms "first", "second", and the like in the present disclosure are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of a kind and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" means at least one of the connected objects, and the character " / ", generally means that the objects before and after are in an "or" relationship.

[0064] The following will be described in conjunction with Figures 1-8 The point cloud semantic segmentation method, system, electronic device and storage medium based on the dynamic uncertainty active learning strategy provided by the present disclosure are described.

[0065] Figure 1 is one of the flowcharts of the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy provided by the present disclosure, as Figure 1 shown, including but not limited to the following steps:

[0066] Step 1: input the three-dimensional point cloud data into the point cloud semantic segmentation model to obtain the semantic category of each point in the three-dimensional point cloud data.

[0067] In this embodiment, first, three-dimensional point cloud data of an urban outdoor scene is obtained, which is collected by a high-precision three-dimensional laser radar (LiDAR) system. In order to ensure the quality of the point cloud data, the original data is first preprocessed, including denoising, downsampling and coordinate normalization, etc. steps to eliminate measurement errors and redundant points, and improve the efficiency and accuracy of subsequent processing. The preprocessed point cloud data is input into the pre-trained point cloud semantic segmentation model, which has been trained on a large-scale and diversified data set, and has good generalization ability and high-precision semantic classification performance.

[0068] By inputting the preprocessed three-dimensional point cloud data into the point cloud semantic segmentation model, the model classifies each point in the point cloud semantically, identifies and labels its belonging semantic category, such as road, building, vehicle, pedestrian and vegetation, etc.

[0069] The specific training method of the point cloud semantic segmentation model will be described in detail in steps 2 to 7 of the subsequent embodiments.

[0070] Step 2: based on the three-dimensional point cloud data samples, a plurality of first super point samples and a plurality of second super point samples are obtained, the first super point samples are labeled super point samples, and the second super point samples are unlabeled super point samples.

[0071] Specifically, first, the preprocessed point cloud data is divided into superpoints using the GlobalEnergy algorithm. The GlobalEnergy algorithm assesses the spatial distribution and feature similarity of points in the point cloud, and clusters adjacent and feature-similar points into superpoints. After division, according to existing labeled data, some superpoint samples are labeled as first superpoint samples. These labeled first superpoint samples contain rich semantic information and can provide high-quality training data for the initial model training. At the same time, the unlabeled second superpoint samples are reserved for subsequent active learning processes, and by intelligently selecting and labeling these unlabeled samples, the performance of the model is gradually improved.

[0072] In this way, the three-dimensional point cloud data is systematically divided into labeled and unlabeled superpoint samples, not only realizing the ordered management and efficient use of data, but also laying a solid foundation for the implementation of the active learning strategy. The labeled first superpoint samples serve as the initial training set, ensuring that the initial model has basic semantic recognition ability; while the unlabeled second superpoint samples gradually introduce high-value labeled samples through uncertainty evaluation and screening in subsequent steps, optimizing the learning process of the model.

[0073] Step 3: Train the initial model based on all first superpoint samples to obtain the first training model.

[0074] Specifically, first, all labeled first superpoint samples are input as training data into a deep learning framework, such as using the RandLA-Net model architecture. To ensure the effectiveness of the training process and the generalization ability of the model, the training data is first preprocessed as necessary, including feature standardization and data augmentation, such as random rotation, scaling, and adding noise, to simulate different environmental conditions and enhance the robustness of the model.

[0075] Then, the initial model is trained using these labeled first superpoint samples. During training, the model continuously optimizes its weight parameters through the backpropagation algorithm to minimize the loss function between the predicted class and the actual class. Since the first superpoint samples contain rich semantic information and high-quality labeled data, the initial model can quickly learn the basic semantic features and spatial structure of the point cloud data, thereby possessing preliminary semantic segmentation ability.

[0076] Step 4: Use the first training model to predict each second superpoint sample to obtain the actual uncertainty of each second superpoint sample.

[0077] In the initial stage of active learning, the prediction accuracy of the model for some types or characteristics of point cloud data may be low due to the insufficient amount of labeled data. In particular, when the model has not been sufficiently exposed to data of a certain semantic category, its prediction accuracy is low. Generally, due to the semantic coherence of point cloud data in space, such inaccurate prediction can lead to generally high uncertainty in the connected region. In order to avoid continuous selection of hyperpoints for labeling in these high-uncertainty regions, the present application particularly introduces the consideration of spatial dispersion when measuring the uncertainty of hyperpoints. This measure aims to optimize the distribution of labeled samples and ensure that the selected hyperpoints have good representativeness and diversity in space, thereby improving the learning efficiency and generalization ability of the model.

[0078] In one possible implementation, with reference to Figure 2 , Figure 2 is a second flowchart of the point cloud semantic segmentation method based on the active learning strategy of dynamic uncertainty provided by the present application, and step 4 specifically includes steps 41-44:

[0079] Step 41: divide all three-dimensional points included in each second hyperpoint sample to obtain a first point set and a second point set, and the first point set is a set of three-dimensional points with the highest category proportion in the second hyperpoint sample.

[0080] In the specific operation process, first, all three-dimensional points inside each second hyperpoint sample are counted according to their predicted semantic categories to determine the category with the highest proportion in the hyperpoint. Then, all points belonging to the dominant category are classified into the first point set, and the points of the remaining categories are classified into the second point set. This division method ensures that the points of the dominant category can be focused on in the subsequent uncertainty calculation, thereby reducing the uncertainty deviation caused by the diversity of categories. In addition, by clearly dividing the dominant category and the secondary category, the hyperpoint with low purity can be more effectively identified, so that appropriate merging or division strategies can be adopted in the subsequent steps to further optimize the selection of labeled samples.

[0081] Step 42: calculate the information entropy of each three-dimensional point using the first training model to obtain the first uncertainty of each three-dimensional point in the first point set and the second uncertainty of each three-dimensional point in the second point set.

[0082] Specifically, first, the first training model trained in step 3 is used to predict the semantic category of each three-dimensional point in the second hyperpoint sample. The model outputs the probability distribution of each point belonging to each category where c represents the category and pv represents the point v. Based on these probability distributions, the information entropy Uv of each point is calculated as its uncertainty, and the formula is as follows:

[0083] ;

[0084] where C is the total number of classes. The higher the information entropy Uv, the greater the uncertainty of the model's prediction for the point. Subsequently, the information entropy of each point is calculated for the first point set and the second point set divided in step 41, obtaining the first uncertainty Uvi of each point in the first point set and the second uncertainty Uvj of each point in the second point set.

[0085] Step 43: Based on all the first uncertainties and all the second uncertainties, the initial uncertainty of each second superpoint sample is calculated.

[0086] Specifically, first, the uncertainties Uvi of all points in the first point set obtained in step 42 are accumulated to obtain the total uncertainty of the superpoint sample in the dominant class. At the same time, the uncertainties Uvj of all points in the second point set are accumulated to obtain the total uncertainty of the superpoint sample in the non-dominant class. Then, the two total uncertainties are combined to calculate the initial uncertainty of the superpoint sample by using the following formula :

[0087] ;

[0088] where ψ represents the majority point set inside the superpoint, and represents the other point set inside the superpoint. In this way, the initial uncertainty not only reflects the uncertainty of the dominant class points, but also reduces the probability of misselection of superpoint samples with low purity by subtracting the uncertainty of non-dominant class points, thereby reducing the negative impact of noise data on model training.

[0089] Step 44: Multiply each initial uncertainty by the class balance weight coefficient of each second superpoint sample to obtain the actual uncertainty of each second superpoint sample. The class balance weight coefficient of each second superpoint sample is determined based on the proportion of second superpoint samples of each class in all class superpoint samples.

[0090] Specifically, the class balance weight coefficient ω(c) is calculated by the following formula:

[0091] ;

[0092] where c is one of the preset classes, sp is a second superpoint sample, d(sp) is the class of the second superpoint sample, U is the set of second superpoint samples, and L is the set of first superpoint samples.

[0093] The actual uncertainty of the second superpoint sample is calculated by the following formula:

[0094] ;

[0095] In this way, the actual uncertainty Not only reflects the prediction uncertainty of the super point sample on the semantic category, but also considers the balance of the category distribution. Specifically, when the super point sample of a certain category accounts for a high proportion in the whole, the corresponding weight coefficient ω(c) will be lower, thereby reducing the actual uncertainty of the super point sample of the category, and avoiding excessive labeling of the sample of the category in the active learning process. On the contrary, for those categories that account for a low proportion in the whole, the weight coefficient ω(c) is higher, thereby increasing the actual uncertainty and increasing the probability of being selected for labeling.

[0096] Step 5: screening all second super point samples based on the actual uncertainty to obtain a plurality of candidate super point samples.

[0097] In one possible implementation, step 5 specifically includes the following steps:

[0098] In each round of screening, the second super point sample with the highest actual uncertainty is selected as the candidate super point sample, and the actual uncertainty of the remaining second super point samples is dynamically attenuated based on the spatial distance and feature similarity of the candidate super point sample and the remaining second super point samples, until a predetermined number of candidate super point samples are screened.

[0099] Specifically, after the actual uncertainty of all second super point samples is obtained in step 4, the second super point sample with the highest uncertainty is selected as the starting point, and it is determined as the first candidate super point sample to be labeled. Subsequently, to avoid redundant labeling in this local area, the uncertainty of other second super point samples near the second super point sample as the starting point is dynamically attenuated.

[0100] Based on the feature similarity and spatial dispersion of the remaining second super point samples and the candidate super point sample, the actual uncertainty of the remaining second super point samples is attenuated, thereby reducing the possibility of these remaining second super point samples as the next candidate super point sample to be labeled. This dynamic attenuation strategy ensures the efficiency and coverage of the labeling process. The method of dynamic attenuation is as follows:

[0101] ;

[0102] wherein, is the actual uncertainty of the second super point sample after attenuation, is the actual uncertainty of the second super point sample before attenuation, is the attenuation coefficient, and dist is the Euclidean distance between the second super point sample performing the attenuation operation and the center of the candidate super point sample.

[0103] According to the dynamic attenuation rule, the actual uncertainty of the remaining second superpoint samples is dynamically updated after each candidate superpoint sample is selected until the K labeled budget is exhausted, and finally K candidate superpoint samples are obtained.

[0104] Step 6: Based on all candidate superpoint samples, a training data set is constructed.

[0105] In this embodiment, after a plurality of candidate superpoint samples are selected based on the actual uncertainty in step 5, step 6 is performed, that is, based on all candidate superpoint samples, a training data set is constructed. The reason for this operation is to integrate high-value candidate superpoint samples and their adjacent superpoint samples to form a training data set with diversity and representativeness, thereby optimizing the training effect of the model and improving the accuracy and generalization ability of the point cloud semantic segmentation.

[0106] In one possible implementation, referring to Figure 3 , Figure 3 is a third flowchart of the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy provided by the application, and step 6 specifically includes steps 61-67:

[0107] Step 61: Establish a multi-way tree for saving superpoint neighbor relationships.

[0108] Specifically, first, the spatial position and feature information of all superpoint samples in the constructed training data set are input into the multi-way tree data structure. The multi-way tree is a hierarchical data structure that can effectively organize and store the neighbor relationships between a large number of superpoints. By layering and dividing the superpoints according to their spatial coordinates, the multi-way tree can maintain the order of data organization while significantly reducing query time and computational complexity.

[0109] In the actual construction process, the superpoint samples generated by the GlobalEnergy algorithm are used as nodes of the multi-way tree, and each node contains the spatial coordinates, color features and normal vector information of the superpoint. The system first inserts the superpoint into the corresponding node of the multi-way tree according to its spatial position, ensuring that adjacent or close superpoints are organized in similar levels of the tree. In this way, the neighbor superpoints of any superpoint can be quickly located and accessed by traversing the local area of the multi-way tree, greatly improving the efficiency of neighbor relationship query.

[0110] Step 62: Based on the multi-way tree, select the first neighbor superpoint sample corresponding to each candidate superpoint sample from all superpoint samples.

[0111] First, the multi-ary tree T constructed in step 61 is used to query the first neighboring super point sample of each candidate super point sample. For each candidate super point sample, a neighbor search operation is performed in the multi-ary tree to quickly locate the super point samples similar in spatial position and features to the candidate super point sample, forming a first neighboring super point sample set. In this way, the first neighboring super point sample corresponding to each candidate super point sample can be efficiently extracted from all super point samples, forming an accurate and relevant super point adjacency relationship network.

[0112] Step 63: Determine whether each first neighboring super point sample is similar to the corresponding candidate super point sample.

[0113] In this embodiment, after selecting the first neighboring super point sample corresponding to each candidate super point sample from all super point samples based on the multi-ary tree in step 62, step 63 is performed, i.e., determining whether each first neighboring super point sample is similar to the corresponding candidate super point sample in the normal vector. This determination is needed to accurately identify super point samples located on the same plane, thereby ensuring the effectiveness and accuracy of the merging operation and further improving the segmentation accuracy and generalization ability of the model.

[0114] In one possible implementation, step 63 specifically includes the following steps:

[0115] Solving the first normal vector of the current first neighboring super point sample and the second normal vector of the corresponding candidate super point sample;

[0116] Calculating the cosine value of the included angle between the first normal vector and the second normal vector;

[0117] Determining whether the cosine value of the included angle is less than a preset normal vector similarity threshold;

[0118] When the cosine value of the included angle is less than the preset normal vector similarity threshold, calculating the color similarity of the current first neighboring super point sample and the corresponding candidate super point sample;

[0119] Determining whether the color similarity is less than a preset color similarity threshold;

[0120] When the color similarity is less than the preset color similarity threshold, determining that the current first neighboring super point sample is similar to the corresponding candidate super point sample.

[0121] Specifically, the Singular Value Decomposition (SVD) method is first used to calculate the normal vectors Vj and Vi of the current first neighboring super point sample Spj and the corresponding candidate super point sample Spi, respectively. As an important geometric feature describing the surface orientation of a super point sample, the main direction information of the super point sample can be accurately extracted by the SVD method.

[0122] Subsequently, the cosine of the included angle between the two normal vectors, aj, is calculated, and the formula is as follows:

[0123]

[0124] wherein Vi and Vj are the normal vectors of the candidate superpoint sample and the first neighboring superpoint sample respectively, · represents the dot product of vectors, and ||·|| represents the length of the normal vector. The closer aj is to 1, the more similar the directions of the two normal vectors are, and the more consistent the planes where the superpoint samples are located are.

[0125] Then, the calculated cosine of the included angle aj is compared with a preset normal vector similarity threshold a. If aj is less than the threshold a, it is determined that the normal vectors of the two superpoint samples have low similarity, indicating that they are not located on the same plane, and the system skips the subsequent color similarity calculation and continues to process the next superpoint sample pair.

[0126] When aj is greater than or equal to the threshold a, the color similarity βj of the two superpoint samples is further calculated. The color similarity is calculated by converting the RGB color information into the LAB color space, and setting the weights of the luminance (L) and chrominance (A, B) components as 0.5, 1.0 and 1.0 respectively, so as to reduce the influence of the change of lighting conditions on the color similarity. Specifically, first, the RGB color values of the two superpoint samples are converted into LAB color values, then the color distance in the LAB space is calculated, and a comparison is made according to a preset color similarity threshold β. If βj is less than the threshold β, it is determined that the two superpoint samples are not similar in color features, and the system continues to skip the subsequent similarity confirmation and processes the next superpoint sample pair. When the color similarity βj is greater than or equal to the threshold β, it is determined that the current first neighboring superpoint sample Spj and the corresponding candidate superpoint sample Spi have similarity in features. This determination indicates that the two superpoint samples not only are located on the same plane in geometric structure, but also are highly consistent in color features, and have the potential of the same semantic category. Therefore, the two superpoint samples are marked as similar and are prepared for merging operation.

[0127] Step 64: Determine the second neighboring superpoint sample from all the first neighboring superpoint samples, and calculate the purity value of each second neighboring superpoint sample, which is similar to the corresponding candidate superpoint sample.

[0128] ​​Specifically, first, based on the first neighboring superpoint samples that have been confirmed to be similar in step 63, the neighboring superpoints of these first neighboring superpoint samples are extracted from the multi-ary tree structure to form a second neighboring superpoint sample set. This step aims to expand the coverage of the area around the candidate superpoint sample and ensure that the purity and semantic consistency of the superpoint sample are evaluated in a larger range. Next, the purity calculation is performed for each second neighboring superpoint sample. The purity value is defined as the proportion of the dominant class in the superpoint sample, and the specific calculation method is to count the number of points of each semantic class in the superpoint sample, determine the class with the highest proportion, and calculate the proportion it occupies. The formula is as follows: φj=class proportion highest point number / total point number in superpoint sample. Wherein, φj represents the purity value of the second neighboring superpoint sample Spj.

[0129] Step 65: Determine the third neighboring superpoint samples with purity values higher than the preset purity threshold and the fourth neighboring superpoint samples with purity values not higher than the preset purity threshold from all second neighboring superpoint samples.

[0130] Specifically, first, all second neighboring superpoint samples obtained in step 64 are traversed, and the purity value φj of each sample is evaluated. The purity value is defined as the proportion of the dominant class in the superpoint sample. The purity value of each second neighboring superpoint sample is compared with the preset purity threshold φ. If the purity value φj of a superpoint sample is greater than or equal to the threshold φ, the superpoint sample is classified as a third neighboring superpoint sample, indicating that it has high semantic consistency and is suitable for being directly included in the training data set; otherwise, if the purity value φj of a superpoint sample is less than the threshold φ, the superpoint sample is classified as a fourth neighboring superpoint sample, indicating that it contains multiple semantic classes and may introduce noise.

[0131] Step 66: Divide the fourth neighboring superpoint samples into a plurality of first sub-superpoint samples, and determine the second sub-superpoint samples with purity values higher than the preset purity threshold from all first sub-superpoint samples.

[0132] In the specific operation process, first, each superpoint sample classified as a fourth neighboring superpoint sample in step 65 is further subdivided. A density-based clustering algorithm such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is used to divide the three-dimensional points in each fourth neighboring superpoint sample to generate a plurality of first sub-superpoint samples. This clustering method can automatically identify and segment point groups with obvious spatial aggregation based on the spatial density and distribution characteristics of point cloud data, thereby effectively dividing complex and diverse low-purity superpoint samples into a plurality of sub-superpoint samples with higher semantic consistency.

[0133] After the division is completed, the purity value φk of each generated first sub-hyperpoint sample is calculated, and the formula is as follows: φk=the number of points with the highest category proportion / total number of points in the first sub-hyperpoint sample. Wherein, φk represents the purity value of the first sub-hyperpoint sample Spk. Through this calculation, the semantic consistency of each sub-hyperpoint sample can be evaluated, and only those second sub-hyperpoint samples Spk' with a purity value higher than a preset purity threshold φ are selected into the training data set, while sub-hyperpoint samples with low purity values are further screened out or processed, so as to avoid introducing low-purity samples containing multiple semantic categories into the training data.

[0134] Step 67: Based on all candidate hyperpoint samples, all third adjacent hyperpoint samples, and all second sub-hyperpoint samples, a training data set is constructed.

[0135] In the specific operation process, first, the multiple candidate hyperpoint samples screened out in step 5, all third adjacent hyperpoint samples determined in step 65, and all second sub-hyperpoint samples screened out in step 66 are combined together to form a comprehensive hyperpoint set. In order to ensure the diversity and representativeness of the training data set, further data enhancement and balance processing is performed on the comprehensive hyperpoint set. For example, to solve the problem of uneven category distribution, oversampling or undersampling methods can be used to ensure that each semantic category has sufficient sample quantity in the training data set. In addition, data enhancement techniques such as random rotation, scaling, and adding noise can be used to increase the diversity of the training data and improve the model's adaptability to different environments and conditions.

[0136] Next, the processed comprehensive hyperpoint set is organized into a training data set and input into a deep learning framework for model training. During training, the optimized training data set is used to continuously adjust the model parameters through the backpropagation algorithm to minimize the loss function between the predicted results and the actual labels. Since the training data set contains high-quality hyperpoint samples from different spatial regions and diverse semantic categories, the model can learn more comprehensive and detailed semantic features, improving its segmentation accuracy and robustness in complex urban outdoor scenes.

[0137] Step 7: Update training of the first training model based on the training data set to obtain a point cloud semantic segmentation model.

[0138] In this embodiment, after the training data set is constructed based on all candidate hyperpoint samples, third adjacent hyperpoint samples, and second sub-hyperpoint samples in step 6, step 7 is performed, that is, the first training model is updated and trained based on the training data set to obtain the final point cloud semantic segmentation model. The reason for this operation is to further improve the semantic recognition ability and generalization performance of the model using the optimized high-quality training data set, so as to achieve higher precision and stronger robustness of point cloud semantic segmentation.

[0139] In a possible implementation, step 7 specifically includes the following steps:

[0140] updating training of the first training model based on the training data set to obtain a second training model;

[0141] taking the second training model as the first training model, and continuing to perform the step of predicting each second super point sample using the first training model to obtain an actual uncertainty of each second super point sample, screening all second super point samples based on the actual uncertainty to obtain a plurality of candidate super point samples, and constructing a training data set based on all candidate super point samples, and updating training of the first training model based on the training data set until a preset stopping condition is reached to obtain the point cloud semantic segmentation model.

[0142] Specifically, first, the first training model is updated and trained using the optimized training data set to generate a second training model with better performance. During the training process, the weight parameters of the model are constantly adjusted through the back propagation algorithm to minimize the loss function between the prediction result and the actual label. Since the training data set has been strictly screened and optimized, it contains candidate super point samples with high uncertainty and high purity, third adjacent super point samples, and second sub-super point samples. The second training model can learn more comprehensive and detailed semantic features and spatial structures during the training process, significantly improving its segmentation accuracy and robustness in complex urban outdoor scenes.

[0143] Subsequently, the second training model is taken as a new first training model to enter the next round of active learning cycle. The updated first training model is used to continue predicting all unlabeled second super point samples, calculate their actual uncertainty, and select new candidate super point samples based on the uncertainty. Subsequently, the training data set is constructed again based on these new candidate super point samples, and the first training model is further updated and trained. Through this iterative training and optimization process, high-value labeled samples can be continuously introduced, gradually improving the performance and generalization ability of the model.

[0144] Further, in order to verify the actual effect of the point cloud semantic segmentation model of the present application, the present application carries out experiments on the three-dimensional laser radar point cloud data set Semantic3D of large city outdoor scene. The Semantic3D data set includes large-scale three-dimensional point cloud data from different regions in Europe, mainly used for studying the semantic segmentation technology of three-dimensional point cloud. The data set is obtained by ground laser scanning equipment, covering a wide range of urban outdoor scenes, including churches, streets, rails, squares, castles, etc., totaling more than 4 billion points, divided into 8 specific semantic categories. Selecting the Semantic3D data set as the experimental object aims to verify the effectiveness and robustness of the present application in processing large-scale complex city point cloud data, ensuring that the proposed active learning strategy can maintain excellent performance in a diversified and high-complexity data environment.

[0145] In order to efficiently process the Semantic3D dataset, the present application first uses a grid downsampling technique to reduce the original point cloud data to the order of ten million. This preprocessing step not only significantly reduces the consumption of computing resources, but also preserves the key geometric and semantic features of the point cloud data. Subsequently, the GlobalEnergyModel method is used to divide the downsampled points into superpoints, further organizing the point cloud data into multiple superpoint samples with spatial coherence and similar features. In addition, in order to efficiently manage the near-neighbor relationship between superpoints, a multi-way tree structure is constructed, which optimizes the storage and query efficiency of the neighbor relationship of superpoints. In order to improve the generalization performance of the model, the input point cloud data is processed by random rotation, scaling and adding random noise to simulate different environmental conditions and enhance the robustness of the model. In the framework of active learning, RandLA-Net is used as the segmenter, and the Adam optimizer is used, with β1 set to 0.9 and β2 set to 0.99. The initial learning rate of the learning process is set to 0.01, and the learning rate is reduced by 8% after each training round, with a total of 30 training rounds, and the batch size is set to 6. In each active learning cycle, the decay coefficient of the uncertainty dynamic measurement method is set to the maximum distance of the superpoints in the scene, and 3000 high-uncertainty candidate superpoints with spatial dispersion and class diversity are selected from the unlabeled set. For the merging operation of superpoints, the normal vector similarity threshold is set to 0.97, and the weight of the LAB component in the color feature is set to 0.5, 1.0 and 1.0 respectively when calculating the similarity, and the similarity threshold is set to 0.95. In addition, the identified noise superpoints are further divided and labeled according to the purity threshold of 0.9. Finally, these labeled superpoints will be added to the labeled set. All codes are implemented based on the Tensorflow framework, and experiments are conducted on a server equipped with an NVIDIA A100 GPU, with 80GB of device memory, an Ubuntu 22.04 operating system, and a CUDA version of 12.2. Through this series of implementation steps, the present application can efficiently process large-scale point cloud data, optimize the selection of labeled samples and the training process of the model, and ensure that the model has excellent segmentation performance in diverse and complex environments.

[0146] The present application evaluates the performance of the model trained using the active learning strategy of the present application on the validation set through two indicators: Mean Intersection over Union (MIoU) and Overall Accuracy (OA). MIoU measures the average overlap between the predicted class and the true class, reflecting the segmentation accuracy of the model on each class; OA measures the proportion of points predicted correctly to the total number of points, reflecting the overall classification accuracy of the model. The specific definitions are as follows:

[0147] ;

[0148] ;

[0149] ;

[0150] where Tp, Fp, Tn and Fn represent the number of true positive, false positive, true negative and false negative respectively, and C is the predicted label class. Through the two indicators, the segmentation performance of the model on different classes and the whole can be comprehensively evaluated, and the effectiveness and superiority of the proposed active learning strategy in practical application are ensured.

[0151] In order to verify the effectiveness of the uncertainty dynamic measurement method SPDU and the super point merging and dividing module SPMD, the present application carries out a series of ablation experiments on the Semantic3D dataset. Referring to Figure 4 , Figure 4 is the ablation experiment result schematic diagram of the SPDU and SPMD module provided by the present application. In order to verify the effectiveness of each module, experiments of different combinations are carried out, including using only SPDU, using only SPMD and using the combination of SPDU and SPMD at the same time. The benchmark experiment adopts random sampling super points for labeling and training, the labeling budget is set to 3000 super points per round of active learning period, a total of 3 rounds, and the mIoU of the model on the test set is used as the index to measure the segmentation effect.

[0152] It can be known from Figure 4 that when using the super point random sampling strategy, the mIoU of the model prediction is only 54.7%. When the SPDU module is applied, the model segmentation accuracy is significantly improved to 63.2%; when only the SPMD module is used, the model performance is slightly improved to 58.3%; and when the two modules are used together, the model performance can reach an accuracy of 67.8% under the condition of labeling a total of 9000 super points. This shows the effectiveness of the SPDU and SPMD modules, and verifies the significant advantage of the active learning strategy proposed by the present application in improving the model segmentation accuracy.

[0153] In order to further verify the superiority of the active learning framework proposed by the present application, it is compared with other 5 kinds of advanced active learning methods on the Semantic3D dataset. Referring to Figure 5 , Figure 5 is the performance comparison schematic diagram of the present application provided on the Semantic3D dataset with other active learning methods, as shown in Figure 5 , the horizontal axis represents the consumed labeling budget, that is, the number of labeled super points, which increases with the iteration of the active learning period; the vertical axis represents the performance of the model on the test set after each round of active learning period, which is measured by the mIoU index.Figure 5 This demonstrates that the active learning method of the present invention significantly outperforms other methods. Specifically, it exhibits higher segmentation performance with the same annotation budget, especially in earlier active learning cycles when the annotation budget is lower. The model trained using this method shows significantly higher segmentation accuracy compared to other methods. Furthermore, this method consumes less annotation budget for the same accuracy requirements. Specifically, refer to... Figure 6 , Figure 6 This is a schematic diagram showing the comparison results of different active learning strategies provided by this invention. Figure 6 The annotation budget required for different active learning methods to achieve 90% performance (mIoU) of full supervision on the Semantic3D test set is shown.

[0154] Depend on Figure 6 As can be seen, compared with the strategy of randomly selecting superpoint samples, the active learning framework proposed in this paper reduces the annotation budget by up to 56.1% while achieving the same performance; even compared with the current state-of-the-art active learning method SSDR, the annotation budget required by this invention is reduced by 21.8%. This result demonstrates that the proposed method has a significant advantage in reducing the annotation budget, further verifying its efficiency and economy in practical applications.

[0155] In summary, experiments and analysis on the Semantic3D dataset validated the effectiveness and superiority of the proposed active learning strategy in point cloud semantic segmentation. Experimental results show that using mesh downsampling and the GlobalEnergyModel method for hyperpoint partitioning, combined with the active learning framework of SPDU and SPMD modules, can significantly improve the model's segmentation accuracy and generalization ability while substantially reducing the annotation budget. Ablation experiments and comparative tests further demonstrate the effectiveness of each module and the superiority of the overall method, ensuring the efficiency, economy, and reliability of this invention in practical applications, and providing solid technical support and guarantee for the development of 3D point cloud semantic segmentation technology.

[0156] Reference Figure 7 , Figure 7 This is a schematic diagram of the point cloud semantic segmentation system based on a dynamic uncertainty active learning strategy provided by the present invention. The system includes:

[0157] The first processing module is used to input the 3D point cloud data into the point cloud semantic segmentation model to obtain the semantic category of each point in the 3D point cloud data.

[0158] The second processing module is used to obtain multiple first superpoint samples and multiple second superpoint samples based on 3D point cloud data samples. The first superpoint samples are labeled superpoint samples, and the second superpoint samples are unlabeled superpoint samples.

[0159] The training module is configured to train the initial model based on the first super point samples to obtain a first training model;

[0160] The second processing module is further configured to predict each second super point sample using the first training model to obtain an actual uncertainty of each second super point sample;

[0161] The second processing module is further configured to filter all the second super point samples based on the actual uncertainty to obtain a plurality of candidate super point samples;

[0162] The second processing module is further configured to construct a training data set based on all the candidate super point samples;

[0163] The training module is further configured to update and train the first training model based on the training data set to obtain a point cloud semantic segmentation model.

[0164] In a possible implementation, the second processing module is further configured to:

[0165] divide all the three-dimensional points included in each second super point sample to obtain a first point set and a second point set, the first point set being a set of three-dimensional points with the highest category proportion in the second super point sample;

[0166] calculate the information entropy of each three-dimensional point using the first training model to obtain a first uncertainty of each three-dimensional point in the first point set and a second uncertainty of each three-dimensional point in the second point set;

[0167] calculate an initial uncertainty of each second super point sample based on all the first uncertainties and all the second uncertainties;

[0168] multiply each initial uncertainty by a category balance weight coefficient of each second super point sample to obtain an actual uncertainty of each second super point sample, the category balance weight coefficient of each second super point sample being determined based on the proportion of the second super point sample of each category in all the super point samples of all categories.

[0169] In a possible implementation, the second processing module is further configured to, in each round of filtering, select a second super point sample with the highest actual uncertainty as a candidate super point sample, and dynamically attenuate the actual uncertainty of the remaining second super point samples based on the spatial distance and feature similarity between the candidate super point sample and the remaining second super point samples until a preset number of candidate super point samples are filtered.

[0170] In a possible implementation, the second processing module is further configured to:

[0171] establish a multi-way tree for saving super point neighbor relationships;

[0172] selecting, based on the multi-ary tree, first neighboring super point samples corresponding to each candidate super point sample from all super point samples;

[0173] determining whether each first neighboring super point sample is similar to the corresponding candidate super point sample;

[0174] determining second neighboring super point samples from all first neighboring super point samples, and calculating a purity value of each second neighboring super point sample, the second neighboring super point sample being similar to the corresponding candidate super point sample;

[0175] determining third neighboring super point samples with purity values higher than a preset purity threshold and fourth neighboring super point samples with purity values not higher than the preset purity threshold from all second neighboring super point samples;

[0176] dividing the fourth neighboring super point samples into first sub-super point samples, and determining second sub-super point samples with purity values higher than the preset purity threshold from all first sub-super point samples;

[0177] constructing a training data set based on all candidate super point samples, all third neighboring super point samples, and all second sub-super point samples.

[0178] In a possible implementation, the second processing module is further configured to:

[0179] solving a first normal vector of the current first neighboring super point sample and a second normal vector of the corresponding candidate super point sample;

[0180] calculating a cosine value of an included angle between the first normal vector and the second normal vector;

[0181] determining whether the cosine value is less than a preset normal vector similarity threshold;

[0182] when the cosine value is less than the preset normal vector similarity threshold, calculating a color similarity between the current first neighboring super point sample and the corresponding candidate super point sample;

[0183] determining whether the color similarity is less than a preset color similarity threshold;

[0184] when the color similarity is less than the preset color similarity threshold, determining that the current first neighboring super point sample is similar to the corresponding candidate super point sample.

[0185] In a possible implementation, the training module is further configured to:

[0186] updating and training the first training model based on the training data set to obtain a second training model;

[0187] The second training model is taken as the first training model, and the step of using the first training model to predict each second super point sample to obtain an actual uncertainty of each second super point sample, screening all second super point samples based on the actual uncertainty to obtain a plurality of candidate super point samples, constructing a training data set based on all candidate super point samples, and updating and training the first training model based on the training data set is continuously performed until a preset stop condition is reached, and the point cloud semantic segmentation model is obtained.

[0188] It should be noted that the point cloud semantic segmentation system based on the dynamic uncertainty active learning strategy provided by the present application can execute the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy of any one of the above embodiments during specific operation, and the present embodiment will not be described here.

[0189] Figure 8 is a structural schematic diagram of an electronic device provided by the present application, as Figure 8 shown, the electronic device can include a processor 810 (processor), a communication interface 820 (Communications Interface), a memory 830 (memory) and a communication bus 840, wherein the processor 810, the communication interface 820, the memory 830 complete mutual communication through the communication bus 840. The processor 810 can call the logic instructions in the memory 830 to execute the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy, which comprises: inputting three-dimensional point cloud data into a point cloud semantic segmentation model to obtain the semantic category of each point in the three-dimensional point cloud data; wherein the point cloud semantic segmentation model is constructed based on the following manner: based on the three-dimensional point cloud data sample, a plurality of first super point samples and a plurality of second super point samples are obtained, the first super point sample is a labeled super point sample, and the second super point sample is an unlabeled super point sample; training an initial model based on all first super point samples to obtain a first training model; using the first training model to predict each second super point sample to obtain an actual uncertainty of each second super point sample; screening all second super point samples based on the actual uncertainty to obtain a plurality of candidate super point samples; based on all candidate super point samples, a training data set is constructed; based on the training data set, the first training model is updated and trained to obtain the point cloud semantic segmentation model.

[0190] Moreover, the logic instructions in the memory 830 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or parts of the present application that essentially contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory 830 (ROM, Read-Only Memory), a random access memory 830 (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0191] In another aspect, the present application also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform the method of point cloud semantic segmentation based on a dynamic uncertainty active learning strategy provided by any of the embodiments described above, the method comprising: inputting three-dimensional point cloud data into a point cloud semantic segmentation model to obtain a semantic class of each point in the three-dimensional point cloud data; wherein the point cloud semantic segmentation model is constructed based on the following manner: based on three-dimensional point cloud data samples, a plurality of first superpoint samples and a plurality of second superpoint samples are obtained, the first superpoint samples are labeled superpoint samples, and the second superpoint samples are unlabeled superpoint samples; training an initial model based on all first superpoint samples to obtain a first training model; using the first training model to predict each second superpoint sample to obtain an actual uncertainty of each second superpoint sample; filtering all second superpoint samples based on the actual uncertainty to obtain a plurality of candidate superpoint samples; constructing a training data set based on all candidate superpoint samples; updating and training the first training model based on the training data set to obtain the point cloud semantic segmentation model.

[0192] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by the processor 810, implements the method for point cloud semantic segmentation based on a dynamic uncertainty active learning strategy provided by the above embodiments, and the method comprises: inputting three-dimensional point cloud data into a point cloud semantic segmentation model to obtain a semantic category of each point in the three-dimensional point cloud data; wherein the point cloud semantic segmentation model is obtained based on the following manner: based on a three-dimensional point cloud data sample, a plurality of first superpoint samples and a plurality of second superpoint samples are obtained, the first superpoint sample is a labeled superpoint sample, and the second superpoint sample is an unlabeled superpoint sample; an initial model is trained based on all the first superpoint samples to obtain a first training model; each second superpoint sample is predicted using the first training model to obtain an actual uncertainty of each second superpoint sample; all the second superpoint samples are screened based on the actual uncertainty to obtain a plurality of candidate superpoint samples; and based on all the candidate superpoint samples, a training data set is constructed; and the first training model is updated and trained based on the training data set to obtain the point cloud semantic segmentation model.

[0193] The system embodiments described above are merely illustrative, wherein the units shown as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0194] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary general hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method of each embodiment or some parts of the embodiment.

[0195] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A point cloud semantic segmentation method based on a dynamic uncertainty active learning strategy, characterized in that, The application relates to a method for constructing a point cloud semantic segmentation model. The method comprises the following steps: inputting three-dimensional point cloud data into a point cloud semantic segmentation model to obtain the semantic category of each point in the three-dimensional point cloud data; wherein the point cloud semantic segmentation model is constructed based on the following method: based on three-dimensional point cloud data samples, a plurality of first super point samples and a plurality of second super point samples are obtained, the first super point sample is a labeled super point sample, and the second super point sample is an unlabeled super point sample; training an initial model based on all the first super point samples to obtain a first training model; using the first training model to predict each second super point sample to obtain the actual uncertainty of each second super point sample; the second super point sample comprises a plurality of three-dimensional points; the actual uncertainty of each second super point sample is obtained by using the first training model to predict each second super point sample, and specifically comprises the following steps: dividing all three-dimensional points included in each second super point sample to obtain a first point set and a second point set, wherein the first point set is a set composed of three-dimensional points with the highest category proportion in the second super point sample; using the first training model to calculate the information entropy of each three-dimensional point to obtain the first uncertainty of each three-dimensional point in the first point set and the second uncertainty of each three-dimensional point in the second point set; based on all the first uncertainty and all the second uncertainty, the initial uncertainty of each second super point sample is calculated; the initial uncertainty is obtained according to a first total uncertainty and a second total uncertainty, the first total uncertainty is obtained by accumulating the uncertainty of all points in the first point set, and the second total uncertainty is obtained by accumulating the uncertainty of all points in the second point set; multiplying each initial uncertainty by the category balance weight coefficient of each second super point sample to obtain the actual uncertainty of each second super point sample; the category balance weight coefficient of each second super point sample is determined based on the proportion of the second super point sample of each category in all category super point samples; based on the actual uncertainty, all the second super point samples are screened to obtain a plurality of candidate super point samples; the actual uncertainty is used to screen all the second super point samples to obtain a plurality of candidate super point samples, which specifically comprises the following steps: in each round of screening, the second super point sample with the highest actual uncertainty is selected as a candidate super point sample, and the actual uncertainty of the remaining second super point samples is dynamically attenuated based on the spatial distance and feature similarity of the candidate super point sample and the remaining second super point samples until a preset number of candidate super point samples are screened; the dynamic attenuation specifically comprises: the actual uncertainty of the second super point sample is attenuated according to an attenuation coefficient and the Euclidean distance between the second super point sample and the center of the candidate super point sample to obtain the actual uncertainty of the attenuated second super point sample; based on all the candidate super point samples, a training data set is constructed; the first training model is updated and trained based on the training data set to obtain the point cloud semantic segmentation model.

2. The method of claim 1, wherein the dynamic uncertainty-based active learning strategy for point cloud semantic segmentation is characterized by, The training data set is constructed based on all the candidate super point samples, and specifically includes: A multi-way tree for saving super point neighbor relationships is established; Based on the multi-way tree, first neighbor super point samples corresponding to each of the candidate super point samples are selected from all super point samples; It is judged whether each of the first neighbor super point samples is similar to the corresponding candidate super point sample; Second neighbor super point samples are determined from all the first neighbor super point samples, and a purity value of each of the second neighbor super point samples is calculated, the second neighbor super point samples being similar to the corresponding candidate super point sample; Third neighbor super point samples with a purity value higher than a preset purity threshold and fourth neighbor super point samples with a purity value not higher than the preset purity threshold are determined from all the second neighbor super point samples; The fourth neighbor super point samples are divided into a plurality of first sub-super point samples, and second sub-super point samples with a purity value higher than the preset purity threshold are determined from all the first sub-super point samples; The training data set is constructed based on all the candidate super point samples, all the third neighbor super point samples, and all the second sub-super point samples.

3. The point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy of claim 2, characterized in that, It is judged whether each of the first neighbor super point samples is similar to the corresponding candidate super point sample, and specifically includes: A first normal vector of the current first neighbor super point sample and a second normal vector of the corresponding candidate super point sample are solved; An included angle cosine value of the first normal vector and the second normal vector is calculated; It is judged whether the included angle cosine value is less than a preset normal vector similarity threshold; When the included angle cosine value is less than the preset normal vector similarity threshold, a color similarity of the current first neighbor super point sample and the corresponding candidate super point sample is calculated; It is judged whether the color similarity is less than a preset color similarity threshold; When the color similarity is less than the preset color similarity threshold, it is determined that the current first neighbor super point sample is similar to the corresponding candidate super point sample.

4. The method of claim 1, wherein the dynamic uncertainty-based active learning strategy for point cloud semantic segmentation is characterized by, The first training model is updated and trained based on the training data set to obtain the point cloud semantic segmentation model, and specifically includes: The first training model is updated and trained based on the training data set to obtain a second training model; The second training model is used as the first training model, and the steps of using the first training model to predict each of the second super point samples to obtain an actual uncertainty of each of the second super point samples, screening all the second super point samples based on the actual uncertainty to obtain a plurality of candidate super point samples, constructing a training data set based on all the candidate super point samples, and updating and training the first training model based on the training data set are continued to be performed until a preset stop condition is reached, and the point cloud semantic segmentation model is obtained. 5.A point cloud semantic segmentation system based on a dynamic uncertainty active learning strategy, characterized in that, It includes: A first processing module is configured to input three-dimensional point cloud data into a point cloud semantic segmentation model to obtain a semantic category of each point in the three-dimensional point cloud data; A second processing module is configured to obtain a plurality of first super point samples and a plurality of second super point samples based on a three-dimensional point cloud data sample, the first super point samples being labeled super point samples, and the second super point samples being unlabeled super point samples. The training module is configured to train an initial model based on all the first super point samples to obtain a first training model; The second processing module is further configured to use the first training model to predict each of the second super point samples to obtain an actual uncertainty of each of the second super point samples; each of the second super point samples comprises a plurality of three-dimensional points; the use of the first training model to predict each of the second super point samples to obtain an actual uncertainty of each of the second super point samples specifically comprises: dividing all the three-dimensional points included in each of the second super point samples to obtain a first point set and a second point set, the first point set being a set of three-dimensional points with the highest category proportion in the second super point sample; using the first training model to calculate the information entropy of each of the three-dimensional points to obtain a first uncertainty of each of the three-dimensional points in the first point set and a second uncertainty of each of the three-dimensional points in the second point set; based on all the first uncertainties and all the second uncertainties, calculating an initial uncertainty of each of the second super point samples; the initial uncertainty is obtained according to a first total uncertainty and a second total uncertainty, the first total uncertainty being obtained by accumulating the uncertainties of all points in the first point set, and the second total uncertainty being obtained by accumulating the uncertainties of all points in the second point set; multiplying each of the initial uncertainties by a category balance weight coefficient of each of the second super point samples to obtain an actual uncertainty of each of the second super point samples, the category balance weight coefficient of each of the second super point samples being determined based on the proportion of the second super point samples of each category in all category super point samples; The second processing module is further configured to filter all the second super point samples based on the actual uncertainties to obtain a plurality of candidate super point samples; the filtering of all the second super point samples based on the actual uncertainties to obtain a plurality of candidate super point samples specifically comprises: in each round of filtering, selecting a second super point sample with the highest actual uncertainty as a candidate super point sample, and dynamically attenuating the actual uncertainties of the remaining second super point samples based on the spatial distance and the feature similarity between the candidate super point sample and the remaining second super point samples until a preset number of candidate super point samples are filtered; the dynamic attenuation specifically comprises: attenuating the actual uncertainty of the second super point sample according to an attenuation coefficient and the Euclidean distance between the second super point sample and the center of the candidate super point sample to obtain the actual uncertainty of the attenuated second super point sample; The second processing module is further configured to construct a training data set based on all the candidate super point samples; The training module is further configured to update train the first training model based on the training data set to obtain the point cloud semantic segmentation model.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy according to any one of claims 1-4.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy according to any one of claims 1-4.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by a processor to implement the point cloud semantic segmentation method based on the dynamic uncertainty active learning strategy according to any one of claims 1-4.

Images