A point cloud semantic segmentation method based on active domain adaptation

Abstract

The application provides a point cloud semantic segmentation method based on active domain adaptation, and relates to the technical field of three-dimensional point cloud processing. A semantic segmentation network model is trained by using a source domain dataset with labels and a target domain dataset without labels. For the target domain dataset without labels, voxel-level preliminary selection and point-level fine screening are sequentially performed, a target domain sample point subset with the highest information value is selected, and the selected sample point subset is labeled; the labeled target domain sample point subset is added to the labeled target domain set. Based on the source domain dataset with labels and the labeled target domain dataset, the preliminarily trained model is fine-tuned for domain adaptation, and a trained model is obtained; the screening, labeling and fine-tuning processes are iteratively performed until the labeling budget is exhausted, and a trained semantic segmentation network model is obtained for semantic segmentation of the target domain dataset. The application effectively reduces the feature difference between domains under limited labeling budget, and realizes efficient and accurate cross-domain segmentation.

Description

Technical Field

[0001] This invention belongs to the technical field of 3D point cloud processing, and more specifically relates to a point cloud semantic segmentation method based on active domain adaptation. Background Technology

[0002] Point cloud semantic segmentation, a core task in the field of 3D vision, aims to assign precise semantic category labels to each point in discrete 3D space, thereby achieving digital and structured analysis of the physical environment. Point cloud semantic segmentation technology constitutes a crucial link from 3D perception to higher-level cognition, providing important technical support for several cutting-edge fields such as autonomous driving, robot autonomous navigation, augmented reality, smart city management, and industrial visual inspection. Specifically, in autonomous driving systems, point cloud semantic segmentation can effectively identify vehicles, pedestrians, and obstacles in road scenes, providing perceptual basis for path planning and driving decisions. In smart city applications, semantic parsing of point clouds in large-scale urban scenes can automate tasks such as building change monitoring, green coverage statistics, and public facility management. In the industrial manufacturing field, it can be used for spatial semantic guidance in parts quality inspection and robot grasping operations.

[0003] Deep learning-based point cloud semantic segmentation methods have made significant progress on benchmark datasets such as the indoor scanning dataset S3DIS and the outdoor autonomous driving dataset SemanticKITTI. However, their performance typically relies on the ideal assumption that the training and test data are independently and identically distributed. In real-world open-scene deployments, the models often experience significant performance degradation due to distributional differences between the source and target domains (i.e., domain shift). Given the high cost and long lead times of acquiring large-scale, accurately labeled data in new environments, developing methods that can achieve efficient domain adaptation under limited labeling conditions has become crucial. Therefore, improving the cross-domain generalization ability of point cloud semantic segmentation models, enabling them to quickly adapt to target domains with different distributions, is a core challenge and important research direction for driving this technology towards large-scale practical applications.

[0004] To improve the cross-domain generalization ability of point cloud semantic segmentation models, existing techniques can be divided into unsupervised domain adaptation, semi-supervised domain adaptation, and active domain adaptation, based on the way and extent to which target domain annotation information is utilized.

[0005] The core of unsupervised domain adaptation lies in learning domain-invariant feature representations by aligning the feature distributions of labeled source and unlabeled target domains. Representative techniques include adversarial training-based and self-training-based methods. Adversarial training-based methods introduce a domain discriminator and a gradient inversion layer to construct an adversarial training paradigm, prompting the feature extractor to learn domain-invariant features that can obfuscate the discriminator. Self-training-based methods rely on the model's predictions in the target domain to generate pseudo-labels, and use these pseudo-labels to iteratively optimize the model, gradually adapting it to the data distribution of the target domain. For example, a two-branch symmetric network architecture, where each branch operates within its corresponding domain, achieves adaptation by fusing selected data fragments from another domain and utilizing the semantic information provided by the source domain labels and target domain pseudo-labels. However, due to the lack of true labels in the target domain, the performance ceiling of unsupervised domain adaptation still lags significantly behind that of fully supervised methods.

[0006] Semi-supervised domain adaptation methods aim to achieve optimal domain adaptation performance under limited annotation cost constraints by collaboratively utilizing source domain data, a small amount of labeled target domain data, and a large amount of unlabeled target domain data. For semi-supervised point cloud semantic segmentation tasks, an SSPC-Net network architecture is proposed. This architecture divides the overall point cloud into superpoint structures, then uses labeled superpoints to infer the semantic labels of unlabeled superpoints, and finally combines the real labels of labeled superpoints with the pseudo-labels generated from unlabeled superpoints to jointly train the semantic segmentation model. For point cloud semantic segmentation scenarios across different weather conditions, a semi-supervised domain adaptation framework is designed. For each abnormal weather type, a small number of point cloud samples are randomly selected and integrated into the normal weather point cloud data of the source domain for joint training. However, the performance of semi-supervised domain adaptation methods is highly dependent on the quality and representativeness of the initial small number of labeled samples. If the distribution of the initial labeled samples is biased or fails to effectively cover the key features of the target domain, the model's adaptation process may deviate from the expected optimization direction, leading to limited performance improvement and even a decline in generalization ability.

[0007] Active domain adaptation combines the selective labeling mechanism of active learning with the distribution alignment goal of domain adaptation. Its core process involves iteratively selecting a small number of samples with the highest information content or the most representative features from the target domain for labeling, and then incorporating these newly labeled samples into the training set to optimize the model. Compared to unsupervised domain adaptation, active domain adaptation, by introducing a small number of precisely labeled samples, can more directly guide the model to adapt to the data distribution of the target domain. Compared to semi-supervised methods, its advantage lies in the active selection of labeled data, rather than passive utilization. For the active domain adaptation task, existing research has proposed a voxel-based active domain adaptation method, aiming to measure the class diversity within each voxel grid through voxel confusion, thereby selecting voxels with high diversity as information-rich regions and labeling all points within them for model training. However, this method does not consider cross-domain information during sample selection, which may lead to the selection of samples that are domain-irrelevant, thus hindering feature alignment between the source and target domains. Furthermore, voxels may contain insensitive or redundant points, resulting in limited labeling efficiency and wasted query budget.

[0008] In summary, existing point cloud semantic segmentation methods mainly rely on large-scale densely labeled data for supervised training. In practical applications such as cross-scene, cross-sensor, or cross-weather scenarios, they face serious domain offset problems, resulting in a significant decrease in the model's generalization ability. Although unsupervised, semi-supervised, and active domain adaptation techniques have alleviated the contradiction between labeling costs and domain adaptation to some extent, they are still limited by key bottlenecks such as feature alignment efficiency, accumulation of pseudo-label noise, and sample selection bias, making it difficult to achieve high-precision cross-domain segmentation with limited labeling budgets. Summary of the Invention

[0009] To address the issue of insufficient segmentation accuracy of existing point cloud semantic segmentation methods in cross-domain application scenarios, this invention proposes a point cloud semantic segmentation method based on active domain adaptation, which effectively reduces the feature differences between the source and target domains and improves the point cloud semantic segmentation accuracy in cross-domain scenarios.

[0010] To achieve the above-mentioned technical effects, the technical solution of the present invention is as follows:

[0011] S1: Obtain the point cloud semantic segmentation dataset and divide the point cloud semantic segmentation dataset into a labeled source domain dataset and an unlabeled target domain dataset; S2: Train the pre-defined semantic segmentation network model based on the labeled source domain dataset and the unlabeled target domain dataset to obtain the initially trained semantic segmentation network model. S3: For target domain datasets without labels, perform voxel-level initial selection and point-level fine screening in sequence to select the target domain sample point subset with the most informational value. S4: Label the subset of target domain sample points with the most informational value, add the labeled subset of target domain sample points with the most informational value to the labeled target domain set, and remove it from the target domain dataset without labels. S5: Based on the labeled source domain dataset and the labeled target domain dataset, perform domain adaptation fine-tuning training on the initially trained semantic segmentation network model to obtain a well-trained semantic segmentation network model. S6: Determine if the annotation budget is exhausted. If so, proceed to step S7; otherwise, use the trained semantic segmentation network model as the initial semantic segmentation network model and repeat steps S3-S5. S7: Based on the trained semantic segmentation network model, perform semantic segmentation on the target domain dataset without labels to obtain the semantic segmentation results of the target domain dataset.

[0012] Further, step S2 is as follows: set the cross-entropy loss function, and use the cross-entropy loss function to train the preset semantic segmentation network model during the training process. When the cross-entropy loss function is minimized, the initially trained semantic segmentation network model is obtained. The expression for the cross-entropy loss function is:

[0013] In the formula, Represents the cross-entropy loss function. Represents a single point cloud. This indicates the number of points contained in the point cloud. express X The first in One point, Indicates the total number of categories. express The unique hot tags of the point, Represents the predicted points of the semantic segmentation network model Category The probability of.

[0014] Furthermore, the voxel-level initial selection process is as follows: The point cloud in the target domain dataset is voxelized to construct a voxel set; Based on the domain uncertainty of voxel internal points and the category distribution weights of voxels, a comprehensive information score is calculated to evaluate the value of voxel information. Based on the comprehensive information score of each voxel, all voxels are sorted in descending order, and the top n voxels are selected as candidate voxels to obtain the initial voxel-level selection results.

[0015] Furthermore, the process of voxelizing the point cloud in the target domain dataset and constructing a voxel set is as follows: Set a predefined voxel size, calculate the voxel coordinates of each point inside each point cloud in the target domain dataset, and aggregate points with the same voxel coordinates into a voxel; construct a three-dimensional voxel mesh set from all non-empty voxels to obtain the voxel set. The expression for calculating the voxel coordinates is:

[0016] In the formula, Indicates the predefined voxel size. This indicates the floor function. Representing a voxel grid x-axis coordinate value, Representing a voxel grid The y-axis coordinate value, Representing a voxel grid z-axis coordinate value, This represents the x-axis coordinate of a point in the coordinate system. This represents the y-axis coordinate of a point in the coordinate system. This represents the z-axis coordinate of a point in the coordinate system.

[0017] Furthermore, the process of calculating the comprehensive information score used to evaluate the value of voxel information is as follows: The voxel domain uncertainty is calculated based on the domain score of each voxel's interior points. The calculation expression is as follows:

[0018] In the formula, This indicates uncertainty in the voxel domain. Voxel representation midpoint field fractions, Voxel representation The mean of all the points in the region. Indicates the first j Individual factors, Voxel representation The number of midpoints; Normalizing the voxel-domain uncertainty yields the normalized voxel-domain uncertainty, expressed as:

[0019] In the formula, This indicates the uncertainty of the voxel field after normalization. This represents the minimum value of the uncertainty in the voxel domain within a single point cloud. This represents the maximum value of the uncertainty in the voxel domain within a single point cloud; Calculate the voxel class weights of points within each voxel using the following expression:

[0020] In the formula, Indicates the class weight of voxels. Voxel representation Belongs to the category The number of points, This indicates the categories in the labeled target domain dataset. The number of points, Voxel representation Total number of points in K Indicates the number of categories; Based on the normalized voxel domain uncertainty and voxel category weights, the comprehensive information score for each voxel is calculated using the following expression:

[0021] In the formula, A comprehensive information score representing voxels.

[0022] Furthermore, the calculation process for the domain fraction of points inside a voxel is as follows: Construct a domain classification network model for calculating domain scores; Construct a cross-entropy loss function; train the domain classification network model based on the source domain dataset; when the cross-entropy loss function is minimized, the pre-trained domain classification network model is obtained. Based on a pre-trained domain classification network model, predict the domain score of points inside voxels.

[0023] Furthermore, the domain classification network model includes: a first convolutional layer module, a second convolutional layer module, a third convolutional layer module, a fourth convolutional layer module, a fifth convolutional layer module, a sixth convolutional layer module, a seventh convolutional layer module, an eighth convolutional layer module, a ninth convolutional layer module, a first linear layer, and a first domain score output module, all connected in sequence.

[0024] Furthermore, the point-level fine screening process includes: calculating the information entropy of each point within each candidate voxel in the voxel-level preliminary selection results, screening for high-uncertainty points, and obtaining a high-uncertainty point candidate set; this process is as follows: Each point within a candidate voxel is input into a pre-trained semantic segmentation network model to determine the semantic category to which that point belongs. k The predicted probability; Based on the fact that this point belongs to various semantic categories k The predicted probability is calculated, and the information entropy of that point within each candidate voxel is calculated using the following expression:

[0025] In the formula, Point Belongs to semantic category The probability, Indicates the first k A semantic category label, Indicates the first voxel inside the candidate One point, K Indicates the total number of semantic categories; Based on the information entropy of each point, the points within each candidate voxel are sorted in descending order to obtain the sorted candidate set S of uncertain points.

[0026] Furthermore, the point-level fine screening process also includes: based on feature diversity, deleting points with similar features from the candidate point set to obtain the most informative subset of target domain sample points; this process is as follows: Step 1: For each point in the candidate point set, extract the high-dimensional feature vector of the point using the pre-trained semantic segmentation network model; Step 2: Sort the candidate points in descending order based on the information entropy of each point; Step 3: Calculate the cosine similarity between the high-dimensional feature vector of the current point and the high-dimensional feature vectors of all points in the candidate point set that are ranked before the current point. The expression is:

[0027] In the formula, This represents the high-dimensional feature vector of the current point. This represents the high-dimensional feature vector of all points in the candidate point set that are ranked before the current point. The cosine similarity between the high-dimensional feature vector of the current point and the high-dimensional feature vectors of all points in the candidate point set that are ranked before the current point. Step 4: Determine if the cosine similarity is lower than a preset threshold. If it is, retain the current point in the point candidate set; otherwise, delete the current point from the point candidate set. Step 5: Repeat steps 3 and 4 to process each point in the candidate point set in turn; after each point in the candidate point set has been processed, the points that are finally retained in the candidate point set are taken as the subset of target domain sample points with the most information value.

[0028] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the active domain adaptation-based point cloud semantic segmentation program is executed by the processor, it implements the steps of the active domain adaptation-based point cloud semantic segmentation method.

[0029] Compared with existing technologies, the beneficial effects of this method are: This invention provides a point cloud semantic segmentation method based on active domain adaptation. First, the point cloud semantic segmentation dataset is divided into a labeled source domain dataset and an unlabeled target domain dataset, and the semantic segmentation network is initially trained. For the unlabeled target domain dataset, voxel-level initial selection and point-level fine-tuning are performed sequentially to select the subset of target domain sample points with the highest information value. This subset of target domain sample points is then labeled and added to the labeled target domain dataset, while simultaneously being removed from the unlabeled target domain dataset. Based on the labeled source domain dataset and the labeled target domain dataset, the initially trained semantic segmentation network model is fine-tuned through domain adaptation to obtain a well-trained semantic segmentation network model. The filtering, labeling, and fine-tuning process is iteratively executed until the labeling budget is exhausted, resulting in a fully trained semantic segmentation network model. Finally, the trained model is used to perform semantic segmentation on the target domain dataset. This invention effectively reduces the feature differences between the source and target domains under limited labeling budgets, achieving efficient and accurate cross-domain point cloud semantic segmentation. Attached Figure Description

[0030] Figure 1 A flowchart illustrating the point cloud semantic segmentation method based on active domain adaptation proposed in this embodiment of the invention; Figure 2 This diagram illustrates the structure of the domain classification network model proposed in this embodiment of the invention. Figure 3 This is a schematic diagram illustrating the electronic device proposed in an embodiment of the present invention. Detailed Implementation

[0031] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts of the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions; It is understandable to those skilled in the art that some well-known details may be omitted from the accompanying drawings.

[0032] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0033] The positional relationships depicted in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. Example 1 This embodiment proposes a point cloud semantic segmentation method based on active domain adaptation, such as... Figure 1 The flowchart shown includes the following steps: S1: Obtain the point cloud semantic segmentation dataset and divide the point cloud semantic segmentation dataset into a labeled source domain dataset and an unlabeled target domain dataset; S2: Train the pre-defined semantic segmentation network model based on the labeled source domain dataset and the unlabeled target domain dataset to obtain the initially trained semantic segmentation network model. S3: For target domain datasets without labels, perform voxel-level initial selection and point-level fine screening in sequence to select the target domain sample point subset with the most informational value. S4: Label the subset of target domain sample points with the most informational value, add the labeled subset of target domain sample points with the most informational value to the labeled target domain set, and remove it from the target domain dataset without labels. S5: Based on the labeled source domain dataset and the labeled target domain dataset, perform domain adaptation fine-tuning training on the initially trained semantic segmentation network model to obtain a well-trained semantic segmentation network model. S6: Determine if the annotation budget is exhausted. If so, proceed to step S7; otherwise, use the trained semantic segmentation network model as the initial semantic segmentation network model and repeat steps S3-S5. S7: Based on the trained semantic segmentation network model, perform semantic segmentation on the target domain dataset without labels to obtain the semantic segmentation results of the target domain dataset.

[0034] In this embodiment, the point cloud semantic segmentation dataset uses the publicly available datasets SemanticPOSS (Peking University Semantic Segmentation Dataset), SemanticKITTI (Karlsruhe, Germany Semantic Segmentation Dataset), SynLiDAR (Synthetic LiDAR Dataset), and nuScenes (Autonomous Driving Dataset). Based on these datasets, three representative cross-domain adaptive tasks were constructed: migration from SynLiDAR (synthetic data) to SemanticPOSS (real-world urban scenes), migration from nuScenes (foreign urban scenes) to SemanticPOSS (Chinese urban scenes), and migration from nuScenes (foreign urban scenes) to SemanticKITTI (German urban roads).

[0035] To ensure label compatibility between the source and target domain datasets, a unified category mapping was implemented. The SynLiDAR→SemanticPOSS task mapped to 13 general categories: cars, bicycles, pedestrians, riders, ground, buildings, fences, vegetation, tree trunks, poles, traffic signs, trash cans, and traffic cones. The nuScenes→SemanticPOSS task mapped to 7 categories: vehicles, pedestrians, roads, sidewalks, terrain, man-made objects, and vegetation. The nuScenes→SemanticKITTI task mapped to 6 categories: pedestrians, bicycles, cars, ground, vegetation, and man-made objects.

[0036] In this embodiment, step S2 is as follows: set the cross-entropy loss function, and use the cross-entropy loss function to train the preset semantic segmentation network model during the training process. When the cross-entropy loss function is minimized, the initially trained semantic segmentation network model is obtained. The expression for the cross-entropy loss function is:

[0037] In the formula, Represents the cross-entropy loss function. Represents a single point cloud. This indicates the number of points contained in the point cloud. express X The first in One point, Indicates the total number of categories. express The unique hot tags of the point, Represents the predicted points of the semantic segmentation network model Category of Probability.

[0038] In this embodiment, the voxel-level initial selection process is as follows: The point cloud in the target domain dataset is voxelized to construct a voxel set; Based on the domain uncertainty of voxel internal points and the category distribution weights of voxels, a comprehensive information score is calculated to evaluate the value of voxel information. Based on the comprehensive information score of each voxel, all voxels are sorted in descending order, and the top n voxels are selected as candidate voxels to obtain the initial voxel-level selection results.

[0039] In this embodiment, the process of voxelizing the point cloud in the target domain dataset and constructing a voxel set is as follows: Set a predefined voxel size, calculate the voxel coordinates of each point inside each point cloud in the target domain dataset, and aggregate points with the same voxel coordinates into a voxel; construct a three-dimensional voxel mesh set from all non-empty voxels to obtain the voxel set. The expression for calculating the voxel coordinates is:

[0040] In the formula, Indicates the predefined voxel size. This indicates the floor function. Representing a voxel grid x-axis coordinate value, Representing a voxel grid The y-axis coordinate value, Representing a voxel grid z-axis coordinate value, This represents the x-axis coordinate of a point in the coordinate system. This represents the y-axis coordinate of a point in the coordinate system. This represents the z-axis coordinate of a point in the coordinate system.

[0041] For example, during the model training phase, a predefined voxel size =0.05. The predefined voxel size during the sample selection phase. =0.25.

[0042] In this embodiment, the process of calculating the comprehensive information score used to evaluate the value of voxel information is as follows: The voxel domain uncertainty is calculated based on the domain score of each voxel's interior points. The calculation expression is as follows:

[0043] In the formula, This indicates uncertainty in the voxel domain. Voxel representation midpoint field fractions, Voxel representation The mean of all the points in the region. Indicates the first j Individual factors, Points inside a voxel; Normalizing the voxel-domain uncertainty yields the normalized voxel-domain uncertainty, expressed as:

[0044] In the formula, This indicates the uncertainty of the voxel field after normalization. This represents the minimum value of the uncertainty in the voxel domain within a single point cloud. This represents the maximum value of the uncertainty in the voxel domain within a single point cloud; For example, the interval is normalized to [0, 1].

[0045] Calculate the voxel class weights of points within each voxel using the following expression:

[0046] In the formula, Indicates the class weight of voxels. Voxel representation Belongs to the category The number of points, This indicates the categories in the labeled target domain dataset. The number of points, Voxel representation Total number of points in K Indicates the number of categories; voxels The total number of points in the game must satisfy the following expression:

[0047] Voxel class weights are assigned to smaller values. By assigning higher weights to samples with lower values, the probability of selecting sample points from fewer categories in the labeled samples is effectively increased, thereby mitigating category selection bias.

[0048] The voxel priority score is obtained by multiplying the voxel domain uncertainty by the voxel class weight. This score is used to quantify the overall information content of a voxel. Based on the normalized voxel domain uncertainty and voxel class weight, the comprehensive information score of each voxel is calculated, and the calculation expression is as follows:

[0049] In the formula, A comprehensive information score representing voxels.

[0050] In this embodiment, voxel domain uncertainty and voxel category weights are integrated to propose a comprehensive information scoring method to quantify voxel information in the unlabeled target domain dataset. The algorithm prioritizes the samples that are most effective for domain alignment, while mitigating the selection bias caused by the long-tailed distribution through dynamic class weighting.

[0051] In the voxel selection stage, a comprehensive information score is used as the evaluation criterion for the comprehensive information value of voxels. This score quantifies the spatial domain information and class distribution bias information of voxels. Specifically, the comprehensive information score comprises two core components: voxel domain uncertainty and voxel class weights. Domain uncertainty measures the information value of the target domain data by aggregating the domain uncertainty of points within the same voxel. Higher domain uncertainty indicates a weaker ability of the model to distinguish whether the voxel belongs to the source or target domain; selecting such voxels helps enhance the model's learning of domain-invariant features. Voxel class weights, by assigning higher weights to classes with smaller sample sizes, mitigate class bias from the historical selection process. This strategy, through dynamic class priority adjustment, effectively reduces the impact of long-tail distribution on active learning performance.

[0052] In this embodiment, the point-level fine screening process includes: screening based on information entropy and exclusion based on feature diversity.

[0053] The information entropy-based selection process involves: calculating the information entropy of each point within each candidate voxel in the initial voxel-level selection results, filtering out points with high uncertainty, and obtaining a candidate set of points with high uncertainty. This process is as follows: Each point within a candidate voxel is input into a pre-trained semantic segmentation network model to determine the semantic category to which that point belongs. k The predicted probability; Based on the fact that this point belongs to various semantic categories k The predicted probability is calculated, and the information entropy of that point within each candidate voxel is calculated using the following expression:

[0054] In the formula, Point Belongs to semantic category The probability, Indicates the first A semantic category label, Indicates the first voxel inside the candidate One point, Represents the total number of semantic categories; information entropy value The higher the value, the better the model is at the point. The greater the uncertainty in prediction, the higher the value of these points. These points are usually high-value sample points. Labeling them and using them for training can effectively improve the semantic segmentation performance of the model.

[0055] Based on the information entropy of each point, the points within each candidate voxel are sorted in descending order to obtain a sorted candidate set of uncertain points. .

[0056] By screening for points with high uncertainty and measuring the uncertainty and feature diversity of these points, samples with higher information content can be selected at a lower annotation cost, thereby improving the semantic segmentation performance of the model in the target domain.

[0057] In this embodiment, the exclusion based on feature diversity specifically involves: based on feature diversity, deleting points with similar features from the candidate point set to obtain the most informative subset of target domain sample points; this process is as follows: Step 1: For each point in the candidate point set, extract the high-dimensional feature vector of the point using the pre-trained semantic segmentation network model; Step 2: Sort the candidate points in descending order based on the information entropy of each point; Step 3: Calculate the cosine similarity between the high-dimensional feature vector of the current point and the high-dimensional feature vectors of all points in the candidate point set that are ranked before the current point. The expression is:

[0058] In the formula, This represents the high-dimensional feature vector of the current point. This represents the high-dimensional feature vector of all points in the candidate point set that are ranked before the current point. The cosine similarity between the high-dimensional feature vector of the current point and the high-dimensional feature vectors of all points in the candidate point set that are ranked before the current point. Step 4: Determine if the cosine similarity is lower than a preset threshold. If it is, retain the current point in the point candidate set; otherwise, delete the current point from the point candidate set. Step 5: Repeat steps 3 and 4 to process each point in the candidate point set in turn; after each point in the candidate point set has been processed, the points that are finally retained in the candidate point set are taken as the subset of target domain sample points with the most information value.

[0059] In this embodiment, the expression for determining whether the cosine similarity is lower than a preset threshold is:

[0060] In the formula, Indicates the preset threshold. This indicates the operation of finding the maximum value. This represents the high-dimensional feature vector of the current point. This represents the high-dimensional feature vector of all points in the candidate point set that are ranked before the current point. This represents the cosine similarity between the high-dimensional feature vector of the current point and the high-dimensional feature vectors of all points in the candidate point set that are ranked before the current point.

[0061] While voxel comprehensive information scoring can effectively identify high-information voxels, the selected voxels may still contain informatic points or points with similar features. To address this issue, a point-level fine screening algorithm is introduced as a crucial subsequent processing stage. This algorithm operates on candidate voxels after voxel priority scoring, using fine-grained discrimination to filter out suboptimal sample points while retaining valuable ones. First, to select samples with higher information value, we use information entropy to prioritize high-uncertainty points, ensuring the information value of the samples. Second, a feature diversity mechanism is designed to quantify the differences between sample points in the feature space, actively eliminating redundant points with similar features to guarantee sample diversity. The point-level fine screening algorithm helps to further improve the information value and diversity of the selected sample points.

[0062] This invention utilizes a multi-level, two-stage collaborative screening mechanism involving voxels and points to achieve progressive selection of high-value, diverse samples at both the voxel and point levels. The selected samples are then used for domain adaptation training, thereby significantly improving the semantic segmentation accuracy of the model in the target domain.

[0063] Labeling the subset of target domain sample points that have the highest information value The subset of target domain sample points with the most informational value after labeling is added to the labeled target domain set. Meanwhile, from the target domain dataset that has no labels Remove from the middle. For example, the expression for the above process is:

[0064]

[0065] In the formula, express The corresponding sample's label.

[0066] Based on the labeled source domain dataset and the labeled target domain dataset, the initially trained semantic segmentation network model is fine-tuned for domain adaptation to obtain a well-trained semantic segmentation network model.

[0067] In domain-adaptive fine-tuning training, the cross-entropy loss function is set as follows:

[0068] In the formula, Represents the cross-entropy loss function. Represents a single point cloud. This indicates the number of points contained in the point cloud. express X The first in i One point, Indicates the total number of categories. express The unique hot tags of the point, Represents the predicted points of the semantic segmentation network model Category The probability of.

[0069] In this embodiment, the annotation budget is set manually according to the specific application scenario. For example, it can be set to one percent of the target domain data.

[0070] This embodiment proposes a point cloud semantic segmentation method based on active domain adaptation. First, it evaluates the voxel-level and point-level information of the point cloud sequentially, selecting high-value and diverse sample points for annotation. Then, it uses these selected sample points for domain adaptation training. Firstly, at the voxel level, a voxel priority score index is introduced to identify high-information voxels. This index comprises two components: voxel domain uncertainty and voxel class weight. Specifically, voxel domain uncertainty is calculated by using neighboring points to determine intra-voxel domain differences, while voxel class weight is calculated by using historically selected sample points and intra-voxel sample points to determine voxel class information. Secondly, at the point level, a point-level fine-tuning algorithm is designed to further filter the selected intra-voxel sample points, ensuring that the selected sample points possess both high informatization and diversity. Specifically, information entropy is used to quantify the uncertainty of sample points, and the sample points are sorted from high to low entropy values. Subsequently, a feature diversity mechanism is proposed to eliminate redundant sample points with similar features among voxels, selecting samples with high uncertainty and diversity. Finally, the selected samples are labeled and used for domain adaptation training to obtain a model adapted to the target domain.

[0071] The point cloud semantic segmentation method based on active domain adaptation proposed in this embodiment can significantly improve the semantic segmentation accuracy of cross-domain scenes under limited annotation budget. The segmentation results show excellent performance in detail preservation and category boundary differentiation, providing reliable technical support for 3D vision applications such as autonomous driving, remote sensing surveying, robot navigation and smart cities.

[0072] The point cloud semantic segmentation method based on active domain adaptation proposed in this embodiment is applicable to most semantic segmentation network models, such as SPVCNN, MinkNet and other networks.

[0073] Example 2 In this embodiment, the calculation process of the domain score of voxel interior points in a point cloud semantic segmentation method based on active domain adaptation proposed in Embodiment 1 is described in detail.

[0074] In this embodiment, the calculation process for the domain fraction of points inside a voxel is as follows: Construct a domain classification network model for calculating domain scores; Construct a cross-entropy loss function; train the domain classification network model based on the source domain dataset and the target domain dataset; when the cross-entropy loss function is minimized, the pre-trained domain classification network model is obtained. Based on a pre-trained domain classification network model, predict the domain score of points inside voxels.

[0075] In this embodiment, as Figure 2The structure diagram shown indicates that the domain classification network model includes: a first convolutional layer module, a second convolutional layer module, a third convolutional layer module, a fourth convolutional layer module, a fifth convolutional layer module, a sixth convolutional layer module, a seventh convolutional layer module, an eighth convolutional layer module, a ninth convolutional layer module, a first linear layer, and a first domain score output module, all connected in sequence.

[0076] In this embodiment, any one of the following convolutional layer modules—the first, second, third, fourth, fifth, sixth, seventh, eighth, and ninth—includes a three-dimensional convolution operation, a batch normalization operation, and a linear rectified function operation connected in sequence.

[0077] In this embodiment, the expression for the cross-entropy loss function is:

[0078] In the formula, Represents the cross-entropy loss function. Indicates the total number of domain categories. Represents a single point cloud. This indicates the number of points contained in the point cloud. express The first in One point, express The unique hot tags of the point, Representation domain classification network model prediction points Belongs to domain category The probability of.

[0079] Example 3 This invention also proposes an electronic device, such as... Figure 3 The schematic diagram shown includes a memory 101, a processor 102, and a computer program stored on the memory 101 and running on the processor 102. When the processor 102 executes the computer program, it implements the steps of the point cloud semantic segmentation method based on active domain adaptation proposed in this embodiment.

[0080] Specifically, in this embodiment, the processor 102 may include a central processing unit (CPU) or a specific integrated circuit, or one or more integrated circuits configured to implement this embodiment. The memory 101 may include a mass storage device for data or instructions. It may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 101 may include removable or non-removable (or fixed) media. Where appropriate, the memory 101 may be internal or external to the integrated gateway disaster recovery device.

[0081] Memory 101 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform steps implementing the point cloud semantic segmentation method based on active domain adaptation proposed in this embodiment.

[0082] The embodiments described are merely examples to clearly illustrate the present invention and are not intended to limit the implementation of the invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively describe all possible implementations. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A point cloud semantic segmentation method based on active domain adaptation, characterized in that, Includes the following steps: S1: Obtain the point cloud semantic segmentation dataset and divide the point cloud semantic segmentation dataset into a labeled source domain dataset and an unlabeled target domain dataset; S2: Train the pre-defined semantic segmentation network model based on the labeled source domain dataset and the unlabeled target domain dataset to obtain the initially trained semantic segmentation network model. S3: For target domain datasets without labels, perform voxel-level initial selection and point-level fine screening in sequence to select the target domain sample point subset with the most informational value. S4: Label the subset of target domain sample points with the most informational value, add the labeled subset of target domain sample points with the most informational value to the labeled target domain set, and remove it from the target domain dataset without labels. S5: Based on the labeled source domain dataset and the labeled target domain dataset, perform domain adaptation fine-tuning training on the initially trained semantic segmentation network model to obtain a well-trained semantic segmentation network model. S6: Determine if the annotation budget is exhausted. If so, proceed to step S7; otherwise, use the trained semantic segmentation network model as the initial semantic segmentation network model and repeat steps S3-S5. S7: Based on the trained semantic segmentation network model, perform semantic segmentation on the target domain dataset without labels to obtain the semantic segmentation results of the target domain dataset.

2. The point cloud semantic segmentation method based on active domain adaptation according to claim 1, characterized in that, Step S2 is as follows: Set the cross-entropy loss function. During the training process, use the cross-entropy loss function to train the preset semantic segmentation network model. When the cross-entropy loss function is minimized, the initially trained semantic segmentation network model is obtained. The expression for the cross-entropy loss function is: In the formula, Represents the cross-entropy loss function. Represents a single point cloud. This indicates the number of points contained in the point cloud. express X The first in One point, Indicates the total number of categories. express The unique hot tags of the point, Represents the predicted points of the semantic segmentation network model Category The probability of.

3. The point cloud semantic segmentation method based on active domain adaptation according to claim 1, characterized in that, The voxel-level initial selection process is as follows: The point cloud in the target domain dataset is voxelized to construct a voxel set; Based on the domain uncertainty of voxel internal points and the category distribution weights of voxels, a comprehensive information score is calculated to evaluate the value of voxel information. Based on the comprehensive information score of each voxel, all voxels are sorted in descending order, and the top n voxels are selected as candidate voxels to obtain the initial voxel-level selection results.

4. The point cloud semantic segmentation method based on active domain adaptation according to claim 3, characterized in that, The process of voxelizing the point cloud in the target domain dataset and constructing a voxel set is as follows: Set a predefined voxel size, calculate the voxel coordinates of each point inside each point cloud in the target domain dataset, and aggregate points with the same voxel coordinates into a voxel; construct a three-dimensional voxel mesh set from all non-empty voxels to obtain the voxel set. The expression for calculating the voxel coordinates is: In the formula, Indicates the predefined voxel size. This indicates the floor function. Representing a voxel grid x-axis coordinate value, Representing a voxel grid The y-axis coordinate value, Representing a voxel grid z-axis coordinate value, This represents the x-axis coordinate of a point in the coordinate system. This represents the y-axis coordinate of a point in the coordinate system. This represents the z-axis coordinate of a point in the coordinate system.

5. The point cloud semantic segmentation method based on active domain adaptation according to claim 1, characterized in that, The process of calculating the comprehensive information score used to evaluate the value of voxel information is as follows: The voxel domain uncertainty is calculated based on the domain score of each voxel's interior points. The calculation expression is as follows: In the formula, This indicates uncertainty in the voxel domain. Voxel representation midpoint field fractions, Voxel representation The mean of all the points in the region. Indicates the first Individual factors, Voxel representation The number of midpoints; Normalizing the voxel-domain uncertainty yields the normalized voxel-domain uncertainty, expressed as: In the formula, This indicates the uncertainty of the voxel field after normalization. This represents the minimum value of the uncertainty in the voxel domain within a single point cloud. This represents the maximum value of the uncertainty in the voxel domain within a single point cloud; Calculate the voxel class weights of points within each voxel using the following expression: In the formula, Indicates the class weight of voxels. Voxel representation Belongs to the category The number of points, This indicates the categories in the labeled target domain dataset. The number of points, Voxel representation Total number of points in K Indicates the number of categories; Based on the normalized voxel domain uncertainty and voxel category weights, the comprehensive information score for each voxel is calculated using the following expression: In the formula, A comprehensive information score representing voxels.

6. The point cloud semantic segmentation method based on active domain adaptation according to claim 5, characterized in that, The calculation process for the domain fraction of points inside a voxel is as follows: Construct a domain classification network model for calculating domain scores; Construct a cross-entropy loss function; train the domain classification network model based on the source domain dataset; when the cross-entropy loss function is minimized, the pre-trained domain classification network model is obtained. Based on a pre-trained domain classification network model, predict the domain score of points inside voxels.

7. A point cloud semantic segmentation method based on active domain adaptation according to claim 6, characterized in that, The domain classification network model includes: a first convolutional layer module, a second convolutional layer module, a third convolutional layer module, a fourth convolutional layer module, a fifth convolutional layer module, a sixth convolutional layer module, a seventh convolutional layer module, an eighth convolutional layer module, a ninth convolutional layer module, a first linear layer, and a first domain score output module, all connected in sequence.

8. The point cloud semantic segmentation method based on active domain adaptation according to claim 1, characterized in that, The point-level fine screening process includes: calculating the information entropy of each point within each candidate voxel in the voxel-level preliminary selection results, screening for high-uncertainty points, and obtaining a high-uncertainty candidate point set; this process is as follows: Each point within a candidate voxel is input into a pre-trained semantic segmentation network model to determine the semantic category to which that point belongs. k The predicted probability; Based on the fact that this point belongs to various semantic categories k The predicted probability is calculated, and the information entropy of that point within each candidate voxel is calculated using the following expression: In the formula, Point Belongs to semantic category The probability, Indicates the first k A semantic category label, Indicates the first voxel inside the candidate One point, K Indicates the total number of semantic categories; Based on the information entropy of each point, the points within each candidate voxel are sorted in descending order to obtain a sorted candidate set of uncertain points. S .

9. A point cloud semantic segmentation method based on active domain adaptation according to claim 8, characterized in that, The point-level fine screening process also includes: based on feature diversity, deleting points with similar features from the candidate point set to obtain the most informative subset of target domain sample points; this process is as follows: Step 1: For each point in the candidate point set, extract the high-dimensional feature vector of the point using the pre-trained semantic segmentation network model; Step 2: Sort the candidate points in descending order based on the information entropy of each point; Step 3: Calculate the cosine similarity between the high-dimensional feature vector of the current point and the high-dimensional feature vectors of all points in the candidate point set that are ranked before the current point. The expression is: In the formula, This represents the high-dimensional feature vector of the current point. This represents the high-dimensional feature vector of all points in the candidate point set that are ranked before the current point. The cosine similarity between the high-dimensional feature vector of the current point and the high-dimensional feature vectors of all points in the candidate point set that are ranked before the current point. Step 4: Determine if the cosine similarity is lower than a preset threshold. If it is, retain the current point in the point candidate set; otherwise, delete the current point from the point candidate set. Step 5: Repeat steps 3 and 4 to process each point in the candidate point set in turn; after each point in the candidate point set has been processed, the points that are finally retained in the candidate point set are taken as the subset of target domain sample points with the most information value.

10. An electronic device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the active domain adaptation-based point cloud semantic segmentation program is executed by the processor, it implements the steps of the active domain adaptation-based point cloud semantic segmentation method as described in any one of claims 1-9.

Images