A three-dimensional point cloud semantic segmentation method, device and storage medium

Abstract

This invention proposes a 3D point cloud semantic segmentation method, device, and storage medium, belonging to the field of computer vision and image processing technology. The method includes: acquiring 3D point cloud data of a target scene; constructing a semi-supervised average teacher network (ATM); training the ATM using the point cloud data; using the trained ATM to perform a dual-view uncertainty assessment on each sample in the unlabeled point cloud data of the target scene, obtaining a fusion uncertainty score for each sample; based on this score and the sample confusion score, employing a dual-driven selection strategy to select samples to be labeled from the unlabeled samples; labeling the samples to be labeled and adding them to the labeled dataset; iteratively training the ATM until preset conditions are met to obtain a final point cloud semantic segmentation model suitable for the target scene. Based on this method, this invention also proposes corresponding devices and storage media. This invention achieves efficient point cloud sample selection.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, and specifically relates to a three-dimensional point cloud semantic segmentation method, device and storage medium. Background Technology

[0002] 3D point cloud segmentation is a key technology in computer vision and image processing, aiming to divide point cloud data in 3D space into regions with different geometric properties or physical meanings. This technology has wide applications in autonomous driving environmental perception, robot navigation, and 3D reconstruction. The application of deep learning technology has greatly promoted the progress of point cloud semantic segmentation, but high-performance deep learning models heavily rely on large-scale, high-quality labeled data. However, labeling 3D point clouds is an extremely time-consuming and expensive task, which severely restricts the rapid deployment and iteration of models in real-world scenarios. To reduce labeling costs, Active Learning (AL) has been introduced into point cloud segmentation tasks. Active Learning aims to automatically select the most valuable samples for improving model performance from a large amount of unlabeled data, which are then manually labeled, thereby obtaining the highest possible model performance with the least possible labeling cost. Existing active learning methods mainly rely on the uncertainty of model predictions (such as prediction entropy, marginal confidence, etc.) to select samples, that is, selecting samples that the model is "unsure" about.

[0003] However, existing active learning methods still suffer from the following technical shortcomings in practice: First, the uncertainty assessment dimension is singular: it mainly relies on data-level uncertainty based on the model's predicted probability. In the early stages of training when labeled data is extremely sparse, the model itself is insufficiently trained and unstable, making this single probability-based uncertainty indicator unreliable and unable to accurately reflect the true value of the samples. Second, there is a cognitive blind spot: the model often shows extremely high confidence in some incorrectly predicted samples, which are called "deceptive samples." Traditional selection strategies based on high uncertainty completely ignore these samples, causing the model to be unable to correct its cognitive bias, and errors accumulate during iterative training. Third, active learning and semi-supervised learning do not work well together: many advanced methods combine active learning and semi-supervised learning, using a large amount of unlabeled data to generate pseudo-labels to assist training. However, existing methods fail to effectively address the dynamic changes in model confidence during training when integrating the two, resulting in excessive pseudo-label noise, which actually harms model performance. Fourth, the sample deduplication strategy is rigid: to avoid selecting redundant samples, strategies such as non-maximum suppression are usually used to retain only the most valuable sample in a local region. However, these strategies often use fixed suppression radii and thresholds, which cannot adapt to the geometric complexity and density variations of point cloud data that are not uniformly distributed in space. They are prone to accidentally deleting important samples in complex regions or retaining too many redundant samples in flat regions.

[0004] Therefore, how to more comprehensively and intelligently evaluate the value of point cloud samples in the geometric feature space, effectively explore the blind spots of geometric feature cognition in the model, and achieve efficient deduplication of samples are the technical problems that current active learning methods for point cloud segmentation urgently need to solve. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a three-dimensional point cloud semantic segmentation method, device, and storage medium. Through dual-view uncertainty assessment, a selection strategy driven by both uncertainty and confusion, and a context-aware suppression strategy, efficient point cloud sample selection is achieved.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, this invention proposes a three-dimensional point cloud semantic segmentation method, comprising the following steps: A 3D point cloud data of the target scene is acquired, a semi-supervised average teacher network is constructed for feature extraction and region segmentation of the point cloud data, and the semi-supervised average teacher network is trained using the point cloud data; the semi-supervised average teacher network includes a teacher sub-network and a student sub-network. Using the trained average teacher network, a dual-view uncertainty assessment is performed on each sample in the unlabeled point cloud data of the target scene to obtain the fusion uncertainty score of each sample. The dual-view uncertainty includes data-level uncertainty and model-level uncertainty. Based on the fusion uncertainty score and the sample confusion score, a dual-drive selection strategy is used to select unlabeled samples from the unlabeled samples, wherein the confusion score is used to characterize the degree of confusion between the sample and features of other categories in the feature space; The samples to be labeled are labeled and added to the labeled dataset. The average teacher network is iteratively trained until the preset conditions are met, and the final point cloud semantic segmentation model suitable for the target scene is obtained.

[0007] Secondly, the present invention proposes a three-dimensional point cloud semantic segmentation device, including at least one processor and a memory, wherein the memory stores a computer program, and the computer program implements the three-dimensional point cloud semantic segmentation method when executed by the at least one processor.

[0008] The effects described in the invention are merely those of the embodiments, and not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects: This invention proposes a 3D point cloud semantic segmentation method, device, and storage medium, belonging to the field of computer vision and image processing technology. The method includes the following steps: acquiring 3D point cloud data of a target scene; constructing a semi-supervised average teacher network for feature extraction and region partitioning of the point cloud data; and training the semi-supervised average teacher network using the point cloud data; the semi-supervised average teacher network includes a teacher sub-network and a student sub-network; using the trained average teacher network, performing a dual-view uncertainty assessment on each sample in the unlabeled point cloud data of the target scene to obtain a fusion uncertainty score for each sample, wherein the dual-view uncertainty includes data-level uncertainty and model-level uncertainty; based on the fusion uncertainty score and the sample's confusion score, employing a dual-driven selection strategy to select samples to be labeled from the unlabeled samples, wherein the confusion score characterizes the degree of confusion between the sample and features of other categories in the feature space; labeling the samples to be labeled and adding them to the labeled dataset; iteratively training the average teacher network until preset conditions are met to obtain a final point cloud semantic segmentation model suitable for the target scene. Based on this method, this invention also proposes corresponding devices and storage media. This invention achieves efficient point cloud sample selection through a dual-perspective uncertainty assessment, a selection strategy driven by both uncertainty and confusion, and a context-aware suppression strategy.

[0009] This invention integrates data-level uncertainty (reflecting the stability of point clouds under geometric perturbations) and model-level uncertainty (reflecting the model's confidence in the features of point clouds) to evaluate the value of samples in improving point cloud segmentation accuracy from both data and model perspectives, overcoming the defect that a single uncertainty index is unreliable when the annotation is sparse.

[0010] The confusion point mining strategy proposed in this invention can identify "deceptive regions" that are segmented incorrectly but have high confidence in the model by comparing the similarity between sample features and various regional reference features. This makes up for the blind spots of traditional uncertainty-based methods and significantly improves the model's ability to characterize complex geometric structures.

[0011] The context-aware suppression strategy proposed in this invention adaptively adjusts the suppression radius and similarity threshold based on the local geometric complexity and density of the point cloud. While preserving the details of complex geometric regions, it effectively removes redundant samples, ensuring the diversity and representativeness of the selected samples.

[0012] This invention introduces an exponential moving average update mechanism based on confidence, which dynamically correlates the update speed of the teacher network with the confidence of the model itself. When the model is unstable, the update is slowed down, reducing the accumulation of false label noise and better realizing the synergy between active learning and semi-supervised learning. Attached Figure Description

[0013] Figure 1 This is a flowchart of a three-dimensional point cloud semantic segmentation method proposed in Embodiment 1 of the present invention; Figure 2 This is a framework diagram for implementing a three-dimensional point cloud semantic segmentation method proposed in Embodiment 1 of the present invention; Figure 3 This is a detailed diagram of the segmentation backbone network and MDS prediction network proposed in Embodiment 1 of the present invention; Figure 4 This is the flow diagram of training set preprocessing and data augmentation proposed in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of the dual-view uncertainty measurement and fusion mechanism proposed in Embodiment 1 of the present invention. Figure 6 This is a schematic diagram of the confusion point mining and uncertainty-driven selection strategy proposed in Embodiment 1 of the present invention; Figure 7 This is an initial segmentation effect diagram for a certain scenario proposed in Embodiment 1 of the present invention; Figure 8 This is a visualization of the uncertainty of point cloud data proposed in Embodiment 1 of the present invention; Figure 9 This is a visualization of the obfuscation level of point cloud data proposed in Embodiment 1 of the present invention; Figure 10 This is a segmentation result image after sampling and training using the method proposed in Embodiment 1 of the present invention; Figure 11 A comparative diagram of the semantic segmentation performance of the method of this invention and existing technologies on different datasets is provided. Figure 12 This is a schematic diagram of a three-dimensional point cloud semantic segmentation device according to Embodiment 2 of the present invention. Detailed Implementation

[0014] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the invention. To simplify the disclosure of the invention, components and arrangements of specific examples are described below. Furthermore, reference numerals and / or letters may be repeated in different examples. This repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. It should be noted that the components illustrated in the drawings are not necessarily drawn to scale. Descriptions of well-known components, processing techniques, and processes are omitted in this invention to avoid unnecessarily limiting the invention.

[0015] Example 1 Embodiment 1 of this invention proposes a three-dimensional point cloud semantic segmentation method to solve the technical problems of uncertainty evaluation dimension, feature recognition blind spot, poor coordination between active learning and semi-supervised learning, and rigid sample deduplication strategy in the existing three-dimensional point cloud semantic segmentation.

[0016] Figure 1 This is a flowchart of a three-dimensional point cloud semantic segmentation method proposed in Embodiment 1 of the present invention; Figure 2 This is a framework diagram for implementing a three-dimensional point cloud semantic segmentation method proposed in Embodiment 1 of the present invention; combined with Figure 1 and Figure 2 The process of implementing this invention will be explained together.

[0017] In step S1, 3D point cloud data of the target scene is acquired, a semi-supervised average teacher network for feature extraction and region segmentation of the point cloud data is constructed, and the semi-supervised average teacher network is trained using the point cloud data; the semi-supervised average teacher network includes a teacher subnetwork. and student sub-network To enhance training stability, an exponential moving average (EMA) update mechanism based on confidence level was designed.

[0018] The formula for updating the teacher subnetwork is:

[0019] in, Indicates the first The parameters of the teacher subnetwork at the next iteration; Indicates the first Parameters of the teacher subnetwork during iteration; Indicates the first The parameters of the student subnetwork at the next iteration; Represents the dynamic confidence factor; This indicates the proportion of new knowledge that the teacher subnetwork learns from the student subnetwork in the current step;

[0020]

[0021] in, Indicates the basic attenuation rate; Indicates the adjustment coefficient; This represents the average confidence level of the teacher network regarding the unlabeled data; It is the predicted probability of the teacher subnetwork. It is the pseudo-label threshold; This indicates an unlabeled data pool.

[0022] When the confidence level of the teacher model is high, the update speed is slowed down to maintain stability.

[0023] Figure 4 This is the training set preprocessing and data augmentation flow diagram proposed in Embodiment 1 of the present invention; the point cloud data in the present invention includes labeled points and unlabeled points. After augmenting the labeled and unlabeled points, the augmented labeled and unlabeled points are used simultaneously for semi-supervised average teacher network training.

[0024] In step S2, the trained average teacher network is used to perform a dual-view uncertainty assessment on each sample in the unlabeled point cloud data of the target scene to obtain the fusion uncertainty score of each sample. The dual-view uncertainty includes data-level uncertainty and model-level uncertainty. Figure 3 This is a detailed diagram of the segmentation backbone network and MDS prediction network proposed in Embodiment 1 of the present invention; in order to comprehensively evaluate the sample value, data-level uncertainty and model-level uncertainty are integrated. Figure 5 This is a schematic diagram of the dual-perspective uncertainty measurement and fusion mechanism proposed in Embodiment 1 of the present invention; The method for calculating the data-level uncertainty score is as follows:

[0025] in, , used to measure the The final indicator of the "value" of an unlabeled sample; The data view uncertainty represents the direct uncertainty of the model's prediction for the current sample; the data view uncertainty is calculated based on the prediction probability distribution of the sample by the teacher subnetwork. This indicates the number of perturbations, representing the number of repeated sampling or augmentation operations performed to calculate model stability; Indicates the sample index; Indicates the index of the number of disturbances; This indicates uncertainty in the area view; Indicates the first The weighting coefficients for the sub-disturbances are used to balance the contribution of different views to the total uncertainty.

[0026] The process of determining model-level uncertainties includes: A model confidence score prediction network is constructed and trained in parallel with the student subnetwork. The model confidence score prediction network is composed of a multilayer perceptron. Based on the predicted probability distribution of labeled samples by the student subnetwork, the true value of the model's confidence score is constructed as the training label. The true value is calculated based on the variance and concentration of the predicted probability vector. The model uncertainty score prediction network is trained under supervision using the training labels to learn the model-level uncertainty of the predicted samples; During the inference phase, unlabeled samples are input into the trained model uncertainty score prediction network, which outputs the model-level uncertainty score for that sample. The specific formula is as follows:

[0027] in, Indicates sample The model-level uncertainty score represents the model's lack of confidence, which is a comprehensive indicator. Indicates student subnetwork to samples The maximum predicted probability; Represents the prediction probability vector Concentration index; This represents the variance of the predicted probability vector P(x); Represents the predicted probability vector, derived from the currently labeled set. The probability distributions of each category output by the trained student model; This represents the total number of categories, i.e., the number of semantic categories in the dataset; This represents the true label of the sample.

[0028] In this invention, the decoder features of the model's confidence score prediction network and the student subnetwork are aligned using inverse distance weighted interpolation, specifically: ; in, Indicates the target point The feature values ​​after interpolation update; Indicates the target point. Indicates the index of the target point; for The The nearest neighbor; Indicates the number of nearest neighbors; Indicates the index of neighboring points, ranging from arrive The formula takes into account The nearest surrounding Its characteristics are determined by several points; The nearest neighbor in the decoder representing the student subnetwork eigenvalues; express and Spatial distance between them 1 indicates the first smoothing term. 1 is a very small constant, preventing when The denominator is zero to ensure the stability of numerical calculations. This represents the weighting factor, indicating that points that are closer to each other have a greater weight, and points that are farther apart have a smaller weight.

[0029] The method for calculating the fusion uncertainty score is as follows: ; in, This represents the final fusion score. The algorithm will sort all unlabeled samples in descending order based on this score, and the sample with the highest score will be selected. The data-level uncertainty score represents the uncertainty of a sample in terms of its feature distribution and geometric space. This represents the model-level uncertainty score; Represents the dynamic equilibrium factor, used for coordination. and The ratio between them; ; in, The annotation ratio indicates the proportion of currently labeled samples to the total sample pool; express The preset maximum value; express The preset minimum value, and Used to control the fluctuation range of the balance factor; This represents the attenuation coefficient.

[0030] In step S3, based on the fusion uncertainty score and the sample confusion score, a dual-drive selection strategy is used to select samples to be labeled from the unlabeled samples, wherein the confusion score is used to characterize the degree of confusion between the sample and other categories of features in the feature space; Figure 6 This is a schematic diagram of the confusion point mining and uncertainty-driven selection strategy proposed in Embodiment 1 of the present invention; The dual-drive selection strategy is used to select the sample with the highest final fusion score and to identify and select samples with confusion scores higher than a preset threshold; wherein, the confusion score is used to measure the degree of matching between the sample features and its predicted category prototype, as well as the degree of confusion with other category prototypes.

[0031] The method for calculating the confusion score is as follows: ; in, Indicates the feature-level confidence ratio; This represents the maximum similarity between a sample feature and the predicted class. The maximum similarity of a sample feature to other classes; 2 represents the second smoothing term to prevent the denominator from being zero; if A high value indicates that the feature of this point is more similar to other classes, and it belongs to a potential error point. ; ; in, This represents the feature vector of the current sample; Indicates the predicted category label; The prototype library representing the predicted category, that is, the prototype memory containing elements belonging to the predicted category. The set of all features; This represents the feature vector in the predicted category prototype library; Indicates category The prototype library; Indicates a reference feature in the non-predicted category; Represents the similarity measurement function; This represents iterating through all non-predicted categories.

[0032] After obtaining candidate samples through a dual-drive selection strategy, and before filtering the samples to be labeled, a context-aware suppression operation is performed on the candidate samples to remove redundant information. The context-aware suppression operation includes: The geometric complexity of the local region where the candidate sample is located is calculated based on the degree of anisotropy of the point cloud distribution in the local neighborhood of the candidate sample. The local density of candidate samples is calculated based on the average distance from the candidate sample to its neighboring points; Based on the geometric complexity and local density, the suppression radius and similarity threshold are dynamically adjusted.

[0033] Calculate local geometric complexity And based on the feature values ​​of the local neighborhood Calculate anisotropy: ; This represents the geometric significance score, also known as linearity or anisotropy score; The eigenvalues ​​represent the eigenvalues ​​obtained by performing eigenvalue decomposition on the covariance matrix constructed from the local neighborhood of that point, and typically satisfy the following conditions: ;in It is the principal component, representing the direction in which the distribution is most dispersed; It is the smallest component, usually representing the direction of the normal or the degree to which it is perpendicular to the plane; For the middle and The components between.

[0034] Based on the average distance of k-nearest neighbors : ; Representing the Local density at each point; Represents the point to its The average distance between the nearest neighbors, where the smaller the distance, the higher the density. The larger; This represents a smoothing term to prevent the denominator from being zero.

[0035] Dynamically adjust the suppression parameters and calculate the overall impact score:

[0036] Represents the geometric saliency after normalization, reflecting the complexity of the surface where the point is located; This represents the normalized point density, reflecting the sparseness of the point cloud around that point.

[0037] Suppression radius and similarity threshold Dynamic calculation:

[0038] Representing the The dynamic local radius of each point; These represent the preset maximum and minimum radius values, respectively. Represents the importance indicator value, when The larger the value, the more accurate the calculation. The smaller the value, the more detailed and localized the view becomes in complex areas, which means using a more nuanced and localized perspective to assess uncertainty.

[0039]

[0040] Representing the The adaptive perturbation intensity of a point, i.e. the similarity threshold, determines how much random noise or coordinate offset to apply to the point and its neighborhood when calculating the "regional view uncertainty". Represents the importance indicator value. The larger the value, the more accurate the calculation. The larger it is; and These represent the maximum and minimum values ​​of the perturbation intensity, respectively. These are preset hyperparameters used to limit the fluctuation range of noise, preventing excessive perturbation from causing point cloud structure collapse, or excessive perturbation from making uncertainty difficult to capture. In complex and dense regions (high... (This involves) reducing the suppression radius and increasing the similarity threshold to retain more details.

[0041] The suppression radius is dynamically adjusted based on the geometric complexity and local density. and adaptive perturbation strength In regions with high geometric complexity and high local density, a smaller suppression radius and a higher adaptive perturbation intensity are used to retain more detailed samples; in regions with low geometric complexity and low local density, a larger suppression radius and a lower adaptive perturbation intensity are used to effectively remove redundant samples.

[0042] Finally, based on the above steps, the sampling score of each point is obtained. By sampling from the largest to the smallest score, the highest quality samples can be obtained for model training, thereby achieving efficient point cloud model segmentation performance optimization.

[0043] In step S4, the samples to be labeled are labeled and added to the labeled dataset. The average teacher network is iteratively trained until the preset conditions are met, and the final point cloud semantic segmentation model suitable for the target scene is obtained.

[0044] The samples selected in step S3 are then manually labeled for precise annotation. These newly labeled samples are then added to the already labeled dataset for the next round of iterative training of the average teacher network. Steps S1 to S4 are repeated until the model performance meets the preset conditions, ultimately resulting in a high-performance point cloud segmentation model suitable for the target scene.

[0045] The three-dimensional point cloud semantic segmentation method proposed in Embodiment 1 of this invention achieves efficient point cloud sample selection through a dual-view uncertainty assessment, a selection strategy driven by both uncertainty and confusion, and a context-aware suppression strategy.

[0046] To fully illustrate the implementation process of Embodiment 1 of the present invention, the following description will be based on a first scenario: The backbone model was trained using a small amount of labeled data to perform point cloud segmentation. The segmentation results for the first scene after training are shown below. Figure 7 This is an initial segmentation effect diagram for a certain scenario proposed in Embodiment 1 of the present invention; For unlabeled data, the probability output of the backbone model is used to calculate the data-level uncertainty of each point and predict the model-level uncertainty (MDS) of each point. The two uncertainties are then fused by correlation weighting to obtain the fused uncertainty of each point. Figure 8 This is a visualization of the uncertainty of point cloud data proposed in Embodiment 1 of the present invention; the darker the blue, the higher the uncertainty.

[0047] Sort all unlabeled points by uncertainty from highest to lowest, and select them sequentially until the number of selected points reaches the budget allocated to "uncertainty-driven selection" in this round. For each class of labeled points, sample the feature space based on distribution density.

[0048] The confusion level of each unlabeled point is calculated, and context-aware suppression is performed. Figure 9 This is a visualization of the confusion level of point cloud data proposed in Embodiment 1 of the present invention; the darker the red, the higher the confusion level.

[0049] Points with confusion levels higher than a certain threshold are selected as candidates. The points in the candidate pool are sorted from low to high uncertainty, and selected sequentially until the sampling requirements are met.

[0050] The points selected by the two methods are labeled, and the next round of training of the backbone model is performed. The segmentation effect of the model after training is shown. Figure 10 The image shows the segmentation results after sampling and training using this method, as proposed in Embodiment 1 of the present invention. It can be seen that the segmentation performance is improved compared to the end of the previous round. Figure 11 The diagrams showing the comparison of semantic segmentation performance of the method of this invention with existing technologies on different datasets are provided. Figure 11 Comparative experimental results are presented to illustrate the visualization effects of different methods on point cloud semantic segmentation tasks.

[0051] To fully illustrate the implementation process of Embodiment 1 of the present invention, we will now use autonomous driving environmental perception as an example: In complex urban roads, LiDAR point cloud data often faces the challenge of "long-tail scenarios." Traditional active learning only focuses on the "uncertainty" of classification boundaries, easily missing high-confidence errors caused by the model's "overconfidence." This solution, through a dual-drive selection strategy, aims to identify blind spots in the model's cognition and accurately select key obstacle samples for annotation.

[0052] (1) Initial perception and prediction: The basic segmentation network was trained using a small number of labeled samples. During the inference stage, the model could distinguish large-area ground features, but the division of overlapping areas of pedestrians and traffic facilities was relatively vague, and there were obvious fluctuations in the classification of distant sparse point clouds.

[0053] (2) Dual-view evaluation mechanism: Region view uncertainty (RVU) calculates geometric saliency by analyzing the local eigenvalue distribution of the point cloud. At complex locations such as vehicle corners and guardrail ends, the system automatically increases the disturbance intensity. If the model's prediction changes drastically under perturbation, the area is identified as a "visual blind spot." Model confidence loss (MDS) searches for evidence of "blind confidence" by comparing feature spaces. For example, if a feature is extremely similar to the prototype of a "traffic cone," but the classifier outputs a high probability for "street lamp," the system will... The formula captures such logical contradictions and increases the selection priority of that point.

[0054] (3) Dynamic dual-drive selection strategy: The system selects based on the current annotation ratio. Real-time update of balance factor In the early stages of training, A larger value guides the model to prioritize samples with high MDS scores to correct classification bias; as the model stabilizes, The algorithm gradually decays, and instead tends to select samples with high HMMU scores to refine the edge details of complex targets.

[0055] (4) Context-aware suppression sampling: Before the final annotation task is issued, the system will combine the semantics of the autonomous driving scenario. For example, if the recognition result is extremely unreasonable in physical logic, the system will suppress the interference of such noise points through the context-aware mechanism to ensure that the annotation resources are concentrated on hard samples with actual safety and obstacle avoidance value.

[0056] The "high-value" point cloud data, filtered through the aforementioned dual perspectives and manually labeled, is incorporated into the labeled dataset for incremental training. Through multiple iterations, the backbone model retains its global perception capabilities while rapidly mastering obstacle features in various long-tail scenarios.

[0057] This invention can be widely applied to scenarios involving 3D point cloud semantic segmentation in fields such as autonomous driving, robot navigation, smart cities, and industrial automation.

[0058] In autonomous driving systems, LiDAR point cloud data is a core data source for environmental perception. This invention, even in situations where labeled data is extremely scarce, utilizes an active learning strategy to efficiently filter out point cloud samples with high information value (such as long-tail scenes like distant sparse obstacles, complex intersections, pedestrians, and non-motorized vehicles). This significantly reduces manual labeling costs while rapidly improving the model's semantic understanding of complex driving environments, providing reliable perceptual input for downstream tasks such as path planning and obstacle avoidance.

[0059] In applications such as indoor service robots and warehousing and logistics robots, robots need to understand the categories of objects in their surrounding environment (such as shelves, goods, people, and aisles) in real time. This invention allows robots to rely on only a small number of labeled samples in the initial deployment stage. Through human-computer interaction or proactive inquiry mechanisms, they can continuously filter and learn unknown or confusing areas in new environments, achieving rapid adaptation and generalization, and improving the robot's robustness and operational safety in dynamic environments.

[0060] In city-level 3D reconstruction and high-precision map construction tasks, point cloud data is massive, and manual annotation is extremely costly. This invention can prioritize the screening of structurally complex areas (such as building facades, bridges, and vegetation edges) and semantically ambiguous areas (such as areas where billboards and building walls are confused), guiding annotation resources to focus on the most valuable samples, accelerating the construction process of high-precision semantic maps, and supporting applications such as urban planning, traffic management, and emergency response.

[0061] In industrial production lines, warehouse inventory management, and other scenarios, 3D point clouds are used for target recognition and localization. This invention can quickly identify anomalies or areas to be inspected with only a very small number of labeled samples, assisting in the construction of efficient automated inspection systems, reducing the frequency of manual intervention, and improving production efficiency and inspection accuracy.

[0062] The scope of protection of this invention is not limited to the specific application areas listed in Example 1. Those skilled in the art can make reasonable selections and adjustments based on the actual situation.

[0063] Example 2 This invention also proposes a three-dimensional point cloud semantic segmentation method. Figure 12 This is a schematic diagram of a drone aerial target detection device proposed in Embodiment 2 of the present invention.

[0064] At the hardware level, the electronic device 1200 includes a processor 1210, and optionally, an internal bus 1220, a network interface 1230, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or it may also include non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for its functions.

[0065] The processor 1210, network interface 1230, and memory can be interconnected via an internal bus 1220. This internal bus 1220 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be categorized as an address bus, data bus, control bus, etc. For ease of illustration, only a single bidirectional arrow is used in this diagram, but this does not imply that there is only one bus or one type of bus. The memory is used to store programs. Specifically, the program can include program code, which includes computer operation instructions. The memory can include main memory 12120 and non-volatile memory 1250, and provides instructions and data to the processor 1210.

[0066] Processor 1210 reads the corresponding computer program from non-volatile memory 1250 into memory 12120 and runs it, forming a device for locating the target user at the logical level. Processor 1210 executes the program stored in memory and specifically performs the following: In step S1, three-dimensional point cloud data of the target scene is acquired, a semi-supervised average teacher network for feature extraction and region division of the point cloud data is constructed, and the semi-supervised average teacher network is trained using the point cloud data; the semi-supervised average teacher network includes a teacher sub-network and a student sub-network. In step S2, the trained average teacher network is used to perform a dual-view uncertainty assessment on each sample in the unlabeled point cloud data of the target scene, obtaining a fusion uncertainty score for each sample. The dual-view uncertainty includes data-level uncertainty and model-level uncertainty. In step S3, based on the fusion uncertainty score and the sample confusion score, a dual-drive selection strategy is used to select samples to be labeled from the unlabeled samples, wherein the confusion score is used to characterize the degree of confusion between the sample and other categories of features in the feature space; In step S4, the samples to be labeled are labeled and added to the labeled dataset. The average teacher network is iteratively trained until the preset conditions are met, and the final point cloud semantic segmentation model suitable for the target scene is obtained.

[0067] This device can be integrated into the onboard computer of a drone to achieve real-time target detection; it can also be used as a ground station to process images transmitted back by the drone offline.

[0068] Figure 1It can be applied to processor 1210, or implemented by processor 1210. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the processor or by instructions in the form of software. The processor mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0069] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0070] Example 3 The present invention also proposes a readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, implements the following method steps: In step S1, three-dimensional point cloud data of the target scene is acquired, a semi-supervised average teacher network for feature extraction and region division of the point cloud data is constructed, and the semi-supervised average teacher network is trained using the point cloud data; the semi-supervised average teacher network includes a teacher sub-network and a student sub-network. In step S2, the trained average teacher network is used to perform a dual-view uncertainty assessment on each sample in the unlabeled point cloud data of the target scene, obtaining a fusion uncertainty score for each sample. The dual-view uncertainty includes data-level uncertainty and model-level uncertainty. In step S3, based on the fusion uncertainty score and the sample confusion score, a dual-drive selection strategy is used to select samples to be labeled from the unlabeled samples, wherein the confusion score is used to characterize the degree of confusion between the sample and other categories of features in the feature space; In step S4, the samples to be labeled are labeled and added to the labeled dataset. The average teacher network is iteratively trained until the preset conditions are met, and the final point cloud semantic segmentation model suitable for the target scene is obtained.

[0071] Embodiment 3 of this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory that stores a computer program, which can be executed by a processor to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0072] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks. Alternatively, if the integrated units of this application are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0073] The three-dimensional point cloud semantic segmentation device provided in Embodiment 2 of this application, and the description of the relevant part of the three-dimensional point cloud semantic segmentation storage medium proposed in Embodiment 3 of this application, can be found in the detailed description of the corresponding part of the three-dimensional point cloud semantic segmentation method provided in Embodiment 1 of this application, and will not be repeated here.

[0074] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that the elements inherent in a process, method, article, or apparatus that includes a list of elements are included. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Additionally, portions of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0075] While specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art can make other modifications or variations based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A three-dimensional point cloud semantic segmentation method, characterized in that, Includes the following steps: A 3D point cloud data of the target scene is acquired, a semi-supervised average teacher network is constructed for feature extraction and region segmentation of the point cloud data, and the semi-supervised average teacher network is trained using the point cloud data; the semi-supervised average teacher network includes a teacher sub-network and a student sub-network. Using the trained average teacher network, a dual-view uncertainty assessment is performed on each sample in the unlabeled point cloud data of the target scene to obtain the fusion uncertainty score of each sample. The dual-view uncertainty includes data-level uncertainty and model-level uncertainty. The method for calculating the data-level uncertainty score is as follows: in, ; The data view uncertainty represents the sample; the data view uncertainty is calculated based on the predicted probability distribution of the sample by the teacher subnetwork. Indicates the number of perturbations; Indicates the sample index; Indicates the index of the number of disturbances; This indicates uncertainty in the area view; Indicates the first The weighting coefficients for the second disturbance; The process of determining the model-level uncertainty includes: A model confidence score prediction network is constructed and trained in parallel with the student subnetwork. The model confidence score prediction network is composed of a multilayer perceptron. Based on the predicted probability distribution of labeled samples by the student subnetwork, the true value of the model's confidence score is constructed as the training label. The true value is calculated based on the variance and concentration of the predicted probability vector. The model uncertainty score prediction network is trained under supervision using the training labels to learn the model-level uncertainty of the predicted samples; During the inference phase, unlabeled samples are input into the trained model uncertainty score prediction network, which outputs the model-level uncertainty score for that sample. The specific formula is as follows: in, Indicates sample Model-level uncertainty score; Indicates student subnetwork to samples The maximum predicted probability; Represents the prediction probability vector Concentration index; This represents the variance of the predicted probability vector P(x); Indicates the total number of categories; Indicates the true label of the sample; The method for calculating the fusion uncertainty score is as follows: ； in, Indicates sample Final fusion score; Indicates sample Data-level uncertainty score; Indicates the dynamic equilibrium factor; Indicates sample Model-level uncertainty score; ； in, Indicates the scale of the annotation; express The preset maximum value; express The preset minimum value; Indicates the attenuation coefficient; Based on the fusion uncertainty score and the sample confusion score, a dual-drive selection strategy is used to filter out unlabeled samples for labeling. The confusion score is used to characterize the degree of confusion between the sample and features of other categories in the feature space. The dual-drive selection strategy is used to select the sample with the highest final fusion score and to identify and select samples with confusion scores higher than a preset threshold. The confusion score is used to measure the matching degree between the sample features and its predicted category prototype, as well as the degree of confusion with other category prototypes. The samples to be labeled are labeled and added to the labeled dataset. The average teacher network is iteratively trained until the preset conditions are met, and the final point cloud semantic segmentation model suitable for the target scene is obtained.

2. The three-dimensional point cloud semantic segmentation method according to claim 1, characterized in that, The formula for updating the teacher subnetwork is: in, Indicates the first The parameters of the teacher subnetwork at the next iteration; Indicates the first Parameters of the teacher subnetwork during iteration; Indicates the first The parameters of the student subnetwork at the next iteration; Represents the dynamic confidence factor; This indicates the proportion of new knowledge that the teacher subnetwork learns from the student subnetwork in the current step; in, Indicates the basic attenuation rate; Indicates the adjustment coefficient; This represents the average confidence level of the teacher network regarding the unlabeled data; It is the predicted probability of the teacher subnetwork. It is the pseudo-label threshold; This indicates an unlabeled data pool.

3. The three-dimensional point cloud semantic segmentation method according to claim 1, characterized in that, The method further includes aligning the decoder features of the model's confidence score prediction network and the student subnetwork using inverse distance weighted interpolation, specifically: ； in, Indicates the target point The feature values ​​after interpolation update; Indicates the target point; for The The nearest neighbor; Indicates the number of nearest neighbors; Indicates the index of neighboring points, ranging from arrive ; The nearest neighbor in the decoder representing the student subnetwork eigenvalues; express and Spatial distance between them 1 indicates the first smoothing term.

4. The three-dimensional point cloud semantic segmentation method according to claim 1, characterized in that, The method for calculating the confusion score is as follows: ； ； ； in, Indicates the feature-level confidence ratio; This represents the maximum similarity between the sample features and the predicted class. The maximum similarity between sample features and other classes; 2 represents the second smoothing term; This represents the feature vector of the current sample; Indicates the predicted category; A prototype library representing the predicted categories; This represents the feature vector in the predicted category prototype library; Indicate category The prototype library; This represents a reference feature in the non-predicted category.

5. The three-dimensional point cloud semantic segmentation method according to claim 1, characterized in that, After obtaining candidate samples through the dual-drive selection strategy, and before filtering the samples to be labeled, a context-aware suppression operation is performed on the candidate samples to remove redundant information. The context-aware suppression operation includes: The geometric complexity of the local region where the candidate sample is located is calculated based on the degree of anisotropy of the point cloud distribution in the local neighborhood of the candidate sample. The local density of candidate samples is calculated based on the average distance from the candidate sample to its neighboring points; Based on the geometric complexity and local density, the suppression radius and similarity threshold are dynamically adjusted.

6. A three-dimensional point cloud semantic segmentation device, comprising at least one processor and a memory, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the at least one processor, it implements a three-dimensional point cloud semantic segmentation method as described in any one of claims 1 to 5.

Images