Local consistency based distillation without replay three-dimensional point cloud class incremental semantic segmentation method

Abstract

The application belongs to the technical field of three-dimensional point cloud semantic understanding and artificial intelligence, and discloses a local consistency distillation non-playback three-dimensional point cloud class incremental semantic segmentation method, which comprises the following steps: obtaining a point cloud input teacher-student framework containing only new class labels, extracting old class knowledge features, teacher prediction results, and current prediction features and student prediction results; based on two feature alignment normalization neighborhood distance and soft neighbor preference distribution, outputting local relationship consistency distillation loss; calculating reliability weight according to old features, teacher prediction and preference distribution for adaptive weighting, outputting reliability-aware feature distillation loss; combining inverse frequency and dynamic class uncertainty to construct balanced weight weighted segmentation loss, outputting uncertainty calibration class balanced loss; updating the student model by combining the three losses and outputting the final segmentation result, effectively solving the imbalance problem between new and old classes, effectively alleviating the catastrophic forgetting under the condition of no data playback, and improving the incremental training stability.

Description

Technical Field

[0001] This invention relates to the fields of 3D point cloud semantic understanding and artificial intelligence technology, and in particular to a method for incremental semantic segmentation of 3D point clouds without playback based on local consistency distillation. Background Technology

[0002] With the rapid development of fields such as autonomous driving, robot navigation, digital twins, smart cities, and indoor scene understanding, 3D point cloud semantic segmentation has become a core foundational task in intelligent perception systems. Unlike 2D images, point cloud data is characterized by its disorder, irregularity, and highly variable local density. Models need to accurately model local geometric relationships and contextual semantic relationships at the point level. In practical engineering deployments, the system must continuously introduce new semantic categories based on existing models. If all historical data needs to be re-collected and trained from scratch every time a new category is added, it is not only extremely costly but also difficult to implement due to practical constraints such as data privacy protection and limited storage resources. Therefore, non-replay incremental learning techniques have become an important technical approach, requiring the model to rely only on a small amount of new category data accessible in the current step to complete the learning of new categories and the retention of knowledge of old categories.

[0003] Existing technologies have proposed several solutions to alleviate the forgetting problem: (1) methods based on logit level or parameter regularization to constrain the offset of output probability or parameters; (2) methods based on geometric prior transfer or prototype enhancement to use explicit geometric constraints to help maintain old classes; (3) structural extension methods based on prompts or adapters to introduce learnable modules to achieve new class adaptation; (4) methods based on uncertainty perception or pseudo-label refinement to generate pseudo-labels through teacher models and combine uncertainty weighting.

[0004] However, the aforementioned methods have significant limitations in the context of incremental 3D point cloud scenarios without playback: logit-level regularization ignores the drift of local neighborhood organization relationships in the feature space, leading to fragmentation of old class regions; geometric prior methods rely on manual assumptions, limiting their generalization ability; structural extension methods introduce additional parameters, which is detrimental to lightweight design; and some methods generally lack point-level discrimination of the reliability of the teacher model's response, making it difficult to balance noise suppression and class balance. The disordered nature of 3D point clouds makes local relationships an important carrier for preserving old knowledge. Existing methods lack explicit consistency constraints on the proportional distance and neighbor preference relationships of local neighborhoods, and are easily dominated by the highly unbalanced high-frequency classes in the current batch, thus affecting the supervision loss. Summary of the Invention

[0005] The purpose of this invention is to solve the technical problems in existing non-playback 3D point cloud incremental semantic segmentation techniques, such as the inability to explicitly preserve local neighborhood organization relationships leading to local relationship drift, the lack of point-level reliability in feature distillation resulting in limited adaptation to new classes, and the extreme imbalance of classes in incremental batches leading to unstable supervised optimization.

[0006] Based on this, the present invention provides an incremental semantic segmentation method for 3D point clouds without playback based on local consistency distillation, comprising the following steps: S11. Data acquisition: Acquire the 3D point cloud data of the current stage, wherein the 3D point cloud data only contains the annotation information of the newly added categories; S12. Construct a teacher-student framework: Input the 3D point cloud data into the teacher-student framework. The teacher-student framework includes a frozen teacher model and a student model to be updated. Extract the old class knowledge features and teacher prediction results of the 3D point cloud data through the teacher model. Extract the current prediction features and student prediction results of the 3D point cloud data through the student model. S13. Perform local relation consistency distillation: Based on the old class knowledge features and the current prediction features, align the normalized neighborhood distance distribution and soft neighbor preference distribution, and output the local relation consistency distillation loss. S14. Perform reliability-aware feature distillation: Calculate point-level reliability weights based on the old class knowledge features, the teacher prediction results, and the soft neighbor preference distribution. Use the point-level reliability weights to adaptively weight the point-level feature differences between the old class knowledge features and the current prediction features, and output the reliability-aware feature distillation loss. S15. Perform uncertainty calibration class balance: Construct dynamic balance weights based on the inverse frequency factors of each category and the dynamically updated category-level uncertainty in the current stage of 3D point cloud data. Use the dynamic balance weights to reweight the supervised segmentation loss of the student model and output the uncertainty calibration class balance loss. S16. Joint optimization of teacher-student framework: Construct a total training objective by combining the local relation consistency distillation loss, the reliability-aware feature distillation loss, and the uncertainty calibration class balance loss. Optimize and update the student model using the total training objective. Output the current stage's 3D point cloud semantic segmentation result using the optimized and updated student model.

[0007] As an optional implementation of this application, step S13 specifically includes: for each anchor point, in the feature space corresponding to the old class knowledge features output by the teacher model, calculating the similarity between the anchor point and other points using cosine similarity, and constructing a fixed neighborhood index for the anchor point based on the similarity; reusing the fixed neighborhood index to the student model, establishing the relative distance ratio between the anchor point and the neighbor nodes corresponding to the fixed neighborhood index as a normalized neighborhood distance distribution; establishing the ranking preference of the teacher model for the importance of each neighbor node corresponding to the fixed neighborhood index as a soft neighbor preference distribution; aligning the normalized neighborhood distance distribution and the soft neighbor preference distribution corresponding to the teacher model and the student model respectively, and outputting the local relation consistency distillation loss.

[0008] As an optional implementation of this application, the calculation method of the normalized neighborhood distance distribution includes: calculating the cosine distance between the anchor point and the features of each neighbor node in the teacher model and the student model respectively; smoothing the cosine distance using an exponential function, and normalizing it in combination with temperature parameters to obtain the teacher normalized neighborhood distance distribution induced by the teacher model and the student normalized neighborhood distance distribution output by the student model.

[0009] As an optional implementation of this application, the calculation method of the soft neighbor preference distribution and the local relation consistency distillation loss includes: based on the cosine similarity between the anchor point and its neighbor node features in the teacher model and the student model, and combined with the temperature parameter, a normalized exponential function operation is performed to transform the teacher soft neighbor preference distribution induced by the teacher model and the student soft neighbor preference distribution output by the student model, respectively; the Jensen-Shannon divergence between the teacher normalized neighborhood distance distribution and the student normalized neighborhood distance distribution is calculated as the distance distribution alignment loss; the Kullback-Leibler divergence between the teacher soft neighbor preference distribution and the student soft neighbor preference distribution is calculated as the preference distribution constraint loss; the distance distribution alignment loss and the preference distribution constraint loss are added together to output the local relation consistency distillation loss.

[0010] As an optional implementation of this application, step S14, calculating the point-level reliability weight based on the old class knowledge features, the teacher prediction results, and the soft neighbor preference distribution, specifically includes: calculating the normalized feature magnitude of the old class knowledge features as the feature response intensity; calculating the entropy value of the posterior probability distribution based on the teacher prediction results to obtain the prediction confidence; calculating the normalized entropy complement of the teacher soft neighbor preference distribution as the neighborhood preference concentration; and fusing the feature response intensity, the prediction confidence, and the neighborhood preference concentration to output the point-level reliability weight.

[0011] As an optional implementation of this application, the step of fusing the feature response intensity, the prediction confidence, and the neighborhood preference concentration to output the point-level reliability weight includes: multiplying and fusing the feature response intensity, the prediction confidence, and the neighborhood preference concentration to obtain a product result; mapping the product result using the Sigmoid function; normalizing the result after the mapping operation within the current batch, and outputting the final point-level reliability weight.

[0012] As an optional implementation of this application, in step S14, the point-level reliability weights are used to adaptively weight the point-level feature differences between the old class knowledge features and the current predicted features, and output the reliability-aware feature distillation loss. Specifically, this includes: calculating the norm distance between the old class knowledge features and the current predicted features as the feature difference; multiplying the point-level reliability weights and the feature differences to obtain the weighted difference value at the corresponding feature position; and calculating the mean of the weighted difference values ​​at all feature positions in the current batch to output the reliability-aware feature distillation loss.

[0013] As an optional implementation of this application, step S15 specifically includes: counting the number of points of each semantic category in the current batch of 3D point cloud data, and calculating the inverse frequency factor of each category according to the reciprocal relationship of the number of points of each category; obtaining the prediction entropy value of the teacher model for the old category, and fixing the prediction entropy value as the category-level uncertainty of the old category; obtaining the student prediction probability of the new category in the current batch, calculating the entropy value of the new category, and updating the category-level uncertainty of the new category online according to the entropy value of the new category using an exponential moving average method; combining the category-level uncertainty of the old category and the category-level uncertainty of the new category, and outputting the category-level uncertainty statistics of all categories; multiplying the inverse frequency factor of each category with the corresponding category-level uncertainty statistics to obtain the basic weight corresponding to each category; normalizing the basic weight among all categories in the current batch, and outputting the dynamic balance weight; multiplying the dynamic balance weight with the supervised cross-entropy segmentation loss of the corresponding category, and outputting the uncertainty calibration class balance loss.

[0014] As an optional implementation of this application, step S16 specifically includes: using a preset balance coefficient to perform a weighted summation of the local relation consistency distillation loss, the reliability-aware feature distillation loss, and the uncertainty calibration class balance loss to obtain the total training objective; optimizing and updating the network parameters of the student model based on the backpropagation algorithm according to the total training objective; and freezing the optimized and updated student model at the end of the current incremental stage as the teacher model for the next incremental stage.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention uses Local Relation Consistency Distillation Loss (LRCD) to explicitly preserve the normalized neighborhood distance distribution and soft neighbor preference distribution induced by the teacher model without requiring precise feature matching. This effectively suppresses the complex point cloud local representation drift phenomenon during the incremental process and solves the problems of fragmentation of old class regions and fuzzy class boundaries.

[0016] 2. This invention utilizes Reliability-Aware Feature Distillation Loss (RFD) to calculate point-level reliability weights using three cues: response strength, prediction confidence, and neighborhood preference concentration. This design effectively distinguishes between stable and ambiguous regions, ensuring the stable transfer of high-value prior knowledge while retaining necessary optimization flexibility for learning new regions.

[0017] 3. This invention utilizes Uncertainty-Calibrated Class Balance Loss (UCCB) to fuse inverse class frequencies with dynamically updated class uncertainty to generate dynamic weights. This mechanism precisely reweights the supervised loss, significantly alleviating the optimization convergence difficulties caused by the imbalance between old and new class data in the current incremental phase, and improving the overall training stability in long-sequence multi-step incremental learning. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 A flowchart of an incremental semantic segmentation method for 3D point clouds based on local consistency distillation without playback, provided in an embodiment of the present invention; Figure 2 This is a diagram illustrating the overall framework structure for collaborative training of the teacher model and the student model provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the local relation consistency distillation module structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the reliability-aware distillation module structure provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the uncertainty perception category balancing module structure provided in an embodiment of the present invention; Figure 6 This is a structural diagram of an electronic device disclosed in this invention. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0022] Example like Figure 1 As shown, this invention provides a method for incremental semantic segmentation of 3D point clouds based on local consistency distillation without playback (LRCD-Net).

[0023] The LRCD-Net method proposed in this invention adopts a classic teacher-student framework (such as...). Figure 2 As shown in the diagram, in each incremental learning stage, the model trained in the previous stage is completely frozen as the teacher model. The student model is optimized using only the 3D point cloud dataset containing only the newly added class annotations in the current stage, thus achieving strictly replay-free incremental learning. The teacher model is responsible for providing stable references to old class knowledge, including point-level feature embeddings and class prediction results; the student model is responsible for simultaneously learning new classes and preserving old class knowledge. The core of this framework lies in effectively suppressing catastrophic forgetting without accessing any historical point cloud samples, through local relation constraints in the feature space, reliability-aware point-level feature transfer, and an uncertainty-calibrated class balancing strategy.

[0024] The input to LRCD-Net is a 3D point cloud patch of the current stage, with a size of [missing information]. (in The number of points in the point cloud. (As input feature dimension), each point is defined by its three-dimensional coordinates ( The input point cloud is represented by both semantic labels and color information (RGB). The output of this method is the semantic label of all points in the current input point cloud, with an output size of [size missing]. (in (This represents the cumulative total number of categories up to the current stage). The entire process requires no storage or playback of historical data; model updates can be completed using only a small amount of labeled data from the current stage. This makes it particularly suitable for practical engineering scenarios with limited storage resources, strict privacy protection, and real-time perception systems.

[0025] This method mainly consists of three key modules: Local Relational Consistency Distillation Module (LRCD), Reliability-aware Feature Distillation Module (RFD), Uncertainty-Calibrated Class Balancing Module (UCCB), and a teacher-student joint optimization framework. The Local Relation Consistency Distillation (LRCD) module is the core of this invention. It jointly aligns the teacher-induced normalized neighborhood distance distribution and soft neighbor preference distribution in the feature space, thereby explicitly preserving the knowledge induced by the teacher model at the local structure level and effectively suppressing the local representation drift phenomenon commonly seen during the incremental process. The Reliability-Aware Feature Distillation (RFD) module dynamically evaluates the reliability of teacher responses at different points. It achieves point-level adaptive feature transfer through three cues: response strength, prediction confidence, and neighborhood preference concentration, making knowledge transfer stronger in stable regions and preserving greater flexibility in ambiguous regions. The Uncertainty Calibration Class Balance (UCCB) module is responsible for handling the severe class imbalance problem in the incremental stage. It reweights the supervised loss by dynamically fusing the inverse frequency factor with class-level uncertainty (fixed teacher statistics for old classes and exponential moving average updates for new classes), thereby improving training stability. Finally, the teacher-student joint optimization framework organically integrates the above three modules. It forms a complete end-to-end training objective through the weighted summation of multiple loss terms, achieving stable preservation of the local structure of old classes and efficient adaptation to new classes.

[0026] Specifically, the method includes the following steps: S11. Data acquisition: Acquire the 3D point cloud data of the current stage, wherein the 3D point cloud data only contains the annotation information of the newly added categories; S12. Construct a teacher-student framework: Input the 3D point cloud data into the teacher-student framework. The teacher-student framework includes a frozen teacher model and a student model to be updated. Extract the old class knowledge features and teacher prediction results of the 3D point cloud data through the teacher model. Extract the current prediction features and student prediction results of the 3D point cloud data through the student model. S13. Perform local relation consistency distillation: Based on the old class knowledge features and the current prediction features, align the normalized neighborhood distance distribution and soft neighbor preference distribution, and output the local relation consistency distillation loss. S14. Perform reliability-aware feature distillation: Calculate point-level reliability weights based on the old class knowledge features, the teacher prediction results, and the soft neighbor preference distribution. Use the point-level reliability weights to adaptively weight the point-level feature differences between the old class knowledge features and the current prediction features, and output the reliability-aware feature distillation loss. S15. Perform uncertainty calibration class balance: Construct dynamic balance weights based on the inverse frequency factors of each category and the dynamically updated category-level uncertainty in the current stage of 3D point cloud data. Use the dynamic balance weights to reweight the supervised segmentation loss of the student model and output the uncertainty calibration class balance loss. S16. Joint optimization of teacher-student framework: Construct a total training objective by combining the local relation consistency distillation loss, the reliability-aware feature distillation loss, and the uncertainty calibration class balance loss. Optimize and update the student model using the total training objective. Output the current stage's 3D point cloud semantic segmentation result using the optimized and updated student model.

[0027] Furthermore, the local relational consistency distillation in step S13 will be explained.

[0028] In incremental semantic segmentation tasks involving 3D point clouds without replay, the model can only access the newly added category (new class features) annotation data in each incremental stage, and cannot replay historical point cloud samples. The disorder, uneven density, and high sensitivity of local geometry of the point cloud easily lead to severe drift of local relationships in the feature space. Most existing methods rely on logit-level regularization or global feature alignment, which makes it difficult to explicitly preserve the local neighborhood organization relationships induced by the teacher model at the feature space level. This results in fragmentation of old class regions, blurred class boundaries, and unstable overall segmentation results, which is a key bottleneck restricting the performance of incremental learning without replay.

[0029] Therefore, this invention proposes a Local Relational Consistency Distillation Module (LRCD). This module jointly aligns two complementary relational perspectives in the feature space: teacher-induced normalized neighborhood distance distribution and soft neighbor preference distribution, thereby explicitly preserving teacher-induced local structural knowledge without requiring precise feature matching. Figure 3 As shown.

[0030] The operation steps of this module are as follows: First, for each anchor point, a fixed neighborhood index is constructed in the feature space corresponding to the old class knowledge features output by the teacher model using cosine similarity calculation (preferred neighborhood size K=20, which can be set according to the specific task). This neighborhood index is then directly reused in the student model to ensure that the student model is always aligned with the same teacher reference structure. Next, two complementary relationship perspectives are established: one is a normalized neighborhood distance distribution, used to maintain the relative distance ratio between the anchor point and its neighbors; the other is a soft neighbor preference distribution, used to maintain the teacher model's ranking preference for different neighbors' importance. Finally, Jensen-Shannon divergence is used to align the distance distribution, and Kullback-Leibler divergence is used to constrain the preference distribution. Both are used together to construct the local relationship consistency distillation loss. The relevant formulas are as follows: (1) Cosine similarity calculation (2) Normalized neighborhood distance distribution (3) Soft neighbor preference distribution (4) LRCD loss in, It is the midpoint of the teacher model With point Cosine similarity in feature space, with a value range of [ 1,1]. The first extracted from the teacher model The characteristics of old-class knowledge at each point (dimensions) ). The first extracted from the teacher model The current predicted features of each point. It is a vector dot product. It is the L2 norm of the vector. It is a point With neighboring points Dissimilarity between them The corresponding point in the teacher model The normalized neighborhood distance distribution represents the point The normalized distance distribution within the neighborhood defined by its teacher. It is a point The neighborhood index set is composed of the most similar teachers in the feature space. It consists of several points. It is a point Midpoint of the neighborhood The dissimilarity. It is a very small normal number, which prevents division by zero and keeps the value stable. In the teacher model, the point To neighboring points The probability of soft preference. It is a temperature parameter that controls the degree of softness or hardness of the distribution. The larger the size, the more evenly distributed the distribution; the smaller the size, the sharper the distribution. It is a loss of consistency in local relations during distillation. It represents the total number of points in the current point cloud. The corresponding point in the student model The normalized neighborhood distance distribution. It is the Jensen–Shannon divergence, which measures the similarity between two probability distributions; it is symmetric and bounded. It is the Kullback–Leibler divergence, which measures the divergence from the distribution. arrive Information loss. It is a balance coefficient that controls the weight of the second term (KL divergence) in the loss.

[0031] Furthermore, the distillation of reliability perception features in step S14 will be explained.

[0032] Traditional feature distillation methods apply a uniform distillation intensity to all points, ignoring the reliability differences of teacher responses in stable regions, class boundary regions, and new class regions. This leads to excessive forced alignment of unreliable responses, which in turn limits the student model's ability to adapt to new classes and reduces the overall effectiveness of knowledge transfer.

[0033] Therefore, this invention proposes a Reliability-aware Feature Distillation Module (RFD). This module achieves point-level adaptive feature transfer through three cues: response strength, prediction confidence, and neighborhood preference concentration, enabling intelligent differentiation between reliable and unreliable teacher responses. For example... Figure 4 As shown.

[0034] The operation steps of this module are as follows: First, extract the three reliability cues of the teacher model, including feature response strength (normalized feature magnitude of old-class knowledge features), prediction confidence (entropy value based on the teacher's posterior probability distribution), and neighborhood preference concentration (normalized entropy complement of the soft neighbor preference distribution); then, multiply and fuse the three cues, map them using the Sigmoid function, and normalize them within the current batch to obtain the final point-level reliability weight; finally, use this weight to perform L2 distance-weighted distillation on the point-level feature differences between the teacher model and the student model. The relevant formulas are as follows: (1) Calculation of reliability weight (2) RFD loss in, It is the normalized reliability weight that is ultimately used for weighted feature distillation. It is a point The raw reliability score (unnormalized) is obtained by fusing the three sub-clues and compressing them with Sigmoid. point The characteristic response intensity reflects the relative magnitude of the teacher's feature vector. It is a point The prediction confidence is calculated by supplementing the teacher's prediction entropy for the old category; the larger the value, the more certain the teacher's prediction for that point. It is a point The concentration of relationships is calculated from the entropy of the teacher's neighbor preference distribution. The larger the value, the more concentrated the teacher's neighbor preference for that point. It is a reliability perception feature of distillation loss. The first extracted from the teacher model The old class knowledge characteristics of each point. It is the first extracted from the student model The current predicted features of each point.

[0035] Through the above design, the RFD module can effectively improve the quality of knowledge transfer for teachers. While strengthening knowledge transfer in stable regions, it retains the necessary optimization flexibility for fuzzy regions and new types of regions, which greatly improves the targeting, reliability and overall efficiency of feature distillation.

[0036] Furthermore, the uncertainty calibration balance in step S15 will be explained.

[0037] In the class increment stage, the current batch often has a serious class imbalance. At the same time, the optimization difficulty of the new class and the old class is significantly different. The traditional cross-entropy loss is easily dominated by the high-frequency old class, resulting in insufficient learning of the rare new class. The overall training process is unstable and prone to convergence difficulties.

[0038] Therefore, this invention proposes an Uncertainty-Calibrated Class Balancing Module (UCCB), which achieves precise reweighting of the supervision loss through the dynamic fusion of the inverse frequency factor and category-level uncertainty. Figure 5 As shown.

[0039] The operation steps of this module are as follows: First, count the number of points for each category in the current batch, and calculate the inverse frequency factor for each category based on the reciprocal relationship of the number of points for each category; obtain the prediction entropy value of the teacher model for the old category, and fix the prediction entropy value as the category-level uncertainty of the old category; obtain the predicted probability of students for the newly added category in the current batch, calculate the entropy value of the newly added category, and update the category-level uncertainty of the newly added category online based on the entropy value of the newly added category using the exponential moving average (EMA) method; combine the category-level uncertainty of the old category and the category-level uncertainty of the newly added category, and output the category-level uncertainty statistics of all categories, thereby avoiding noise interference introduced by online calculation. Multiply the inverse frequency factor of each category with the corresponding category-level uncertainty statistics to obtain the basic weight corresponding to each category; normalize the basic weights among all categories in the current batch to obtain the dynamic class balance weights; multiply the dynamic balance weights with the supervised cross-entropy segmentation loss of the corresponding category to output the uncertainty calibration class balance loss. The relevant formulas are as follows: (1) Category balancing weights (2) UCCB loss in, and It is a category index. It is a category The inverse frequency factor. It is a category The final UCCB weights are used for subsequent weighted cross-entropy loss. It represents the number of categories. It is a category Uncertainty factors, It is a student model Predicted as category The probability of; It is a category Uncertainty factors. It is the category balance loss of the uncertainty calibration. It is predicted as a category The final UCCB weights.

[0040] Through the above mechanism, the UCCB module can significantly alleviate the problems of class imbalance and mismatch of optimization difficulty in the incremental stage, enabling the model to maintain a more stable and efficient supervised optimization process in long-term multi-step incremental learning.

[0041] Furthermore, the joint optimization of the teacher-student framework in step S16 is explained.

[0042] A single module cannot simultaneously solve the three core problems of local relation drift, reliable feature distillation distinction, and unstable optimization due to class imbalance, which limits the overall performance and long-term stability of incremental learning without replay.

[0043] Therefore, this invention integrates the three modules LRCD, RFD and UCCB into a complete teacher-student joint optimization framework. This framework strictly follows the no-replay constraint in each incremental stage and forms the end-to-end training objective through the weighted summation of multiple loss terms.

[0044] The framework operates as follows: First, the LRCD loss, RFD loss, and UCCB loss are weighted and summed using a fixed balancing coefficient to obtain the final overall training objective. Then, at the end of each incremental stage, the trained student model is directly frozen as the teacher model for the next stage, achieving continuous knowledge transfer. Finally, through a teacher-student joint optimization mechanism, without relying on historical sample replay, the framework stably preserves the local structural knowledge of the old classes while achieving efficient and rapid adaptation to new classes. The relevant formulas are as follows: Overall training objective (in ) in, It is the total loss. It is a loss of consistency in local relations during distillation. It is a reliability-perceived characteristic of distillation loss. It is the category balance loss of the uncertainty calibration. yes loss weights, yes The loss weight.

[0045] Through the aforementioned joint optimization framework, this invention can simultaneously achieve three major objectives—preservation of local structure, reliable knowledge transfer, and category balance optimization—in real-world deployment scenarios with limited storage and sensitive privacy. This provides a complete, efficient, stable, and easy-to-deploy solution for incremental semantic segmentation of 3D point clouds without playback in practical engineering.

[0046] Example verification 1. Dataset This invention validates the effectiveness of the LRCD-Net method on two widely used indoor 3D point cloud benchmark datasets: S3DIS and ScanNet. S3DIS is a high-quality, large-scale indoor scene dataset, approximately 4.5GB in size (raw format), containing six different office building areas, covering a total area of ​​approximately 6020m², and containing 272 rooms. This dataset is labeled with 13 semantic categories (including common indoor objects such as walls, floors, chairs, tables, doors, windows, and pillars). ScanNet, on the other hand, is a more diverse indoor reconstruction dataset, approximately 1.5TB in size (raw scan data), containing 1513 different scenes covering various indoor environments such as offices, apartments, and shops, and labeled with 20 semantic categories. Both datasets provide 3D coordinates and color information in point cloud form, and the scenes are complex, heavily occluded, and have uneven density distributions, making them classic benchmarks for evaluating incremental semantic segmentation algorithms.

[0047] Unlike traditional fully supervised learning settings, no-replay incremental semantic segmentation requires the model to use only newly added category-labeled data in the current stage when the label space is dynamically expanded, without accessing or replaying training point cloud samples from previous stages. To ensure fair performance testing, all points in both the test and validation sets are assumed to have full labels. Therefore, targeted incremental protocol processing of the S3DIS and ScanNet datasets is necessary for rigorous no-replay incremental training.

[0048] In this invention, a standard replay-free incremental class protocol is adopted on the S3DIS and ScanNet datasets: First, a base class phase is performed, using a fully annotated initial class set to train the initial model; subsequently, in each incremental phase, only the newly added classes for the current phase are provided (…). The labeled data for this process does not save or replay any original training point clouds from previous stages. Following common single-step and multi-step protocols, S3DIS employs three single-step partitioning methods (0-7 / 8-12, 0-9 / 10-12, 0-11 / 12) and a five-step progressive multi-step partitioning; ScanNet employs three single-step partitioning methods (0-14 / 15-19, 0-16 / 17-19, 0-18 / 19). The entire process consists of two steps: First, following standard point cloud preprocessing methods, the original input point cloud is voxel downsampled (voxel size is typically set to 0.05m) to remove redundant points and make the point cloud distribution more uniform, thus providing stable learning input for the model. Second, in the training point set of each incremental stage, only the labeled points of the newly added categories in the current stage are retained; all other historical category point clouds are completely discarded and do not participate in any form of replay or label reconstruction. This approach minimizes storage costs and privacy risks, while also better reflecting real-world scenarios where historical data is difficult to preserve long-term. Through this rigorous no-replay incremental strategy, the network can still achieve stable and efficient 3D point cloud incremental semantic segmentation on S3DIS and ScanNet using only a small amount of new category labeled data in the current stage, which fully verifies the superiority of the present invention in long-term incremental learning.

[0049] The invention was tested on a server equipped with an NVIDIA RTX 4090 GPU (24GB of video memory) and the PyTorch deep learning framework.

[0050] 2. Experimental parameter settings The network is trained for a total of 100 epochs. The batch size in this invention is set to 32, the neighborhood size K in the LRCD module is set to 20, and the initial learning rate is set to 1×10⁻⁶. -3 The AdamW optimizer was used, and a cosine annealing function was employed to gradually decay the learning rate. The balance coefficients of the RFD and UCCB modules were fixed at 0.3 and 0.5, respectively, and the temperature parameter τ in the LRCD module was set to its default value. Experiments were conducted in single-step (… End-to-end training of LRCD-Net is performed under the multi-step (S3DIS five-step progressive) incremental protocol.

[0051] 3. Experimental Results As shown in Table 1, under the single-step incremental semantic segmentation setting of the S3DIS dataset, when using Stage 1 after one incremental update as the main evaluation criterion, the proposed method performs well. and The two task partitioning methods achieved global mIoU of 46.84% and 46.65% respectively, both being the best results among the compared no-replay methods. Specifically, compared with BRDL, the global mIoU under the two settings was improved by 1.83 and 1.13 percentage points respectively; compared with GUA, it was improved by 2.12 and 2.72 percentage points respectively. Meanwhile, under these two settings, the old class mIoU of this method reached 40.40% and 42.21% respectively, both higher than GUA and BRDL, demonstrating better stability in maintaining old class knowledge. Furthermore, in Under these conditions, the global mIoU of this method is even slightly higher than the 46.74% of the joint training upper bound JT. Even with the current settings, the performance difference compared to JT is only 0.09 percentage points, indicating that the proposed method can recover a performance level quite close to that of joint training without accessing historical data. Although in a more emphasis on preserving capabilities... Under the given conditions, the global mIoU of this method is 39.11%, which is lower than BRDL's 42.16% but still slightly higher than GUA's 38.71%. This indicates that the method has a strong overall advantage in most single-step incremental scenarios, while there is still room for further optimization in extreme preservation-oriented tasks.

[0052] Table 2 further demonstrates that the proposed method also exhibits good transfer and generalization capabilities on the ScanNet dataset, which features more complex scene composition and more significant reconstruction noise. In Phase 1, this method achieved a global mIoU of 31.03%, which is superior to BRDL's 29.11%, representing an improvement of 1.92 percentage points; Under the given conditions, this method achieves a global mIoU of 29.58%, which is roughly the same as BRDL's 29.55%, indicating that this method can still maintain stable incremental segmentation performance in more challenging heterogeneous scenarios. Although in Under the given conditions, the proposed method achieves a global mIoU of 30.90%, slightly lower than BRDL's 31.13%, but still within the same competitive tier. Tables 1 and 2 show that the proposed method is not only effective on the structurally regular S3DIS, but also maintains high competitiveness on complex scenarios and with stronger noise interference like ScanNet, thus verifying the method's adaptability to incremental update tasks under different data distribution conditions.

[0053] As shown in Table 3, under the multi-step class increment setting of the S3DIS dataset, although the performance of all methods decreased as the increment rounds progressed from stage 1 to stage 5, the overall class mIoU of our proposed method consistently exceeded that of GUA in each increment stage. The differences between the two methods in stages 1 to 5 were 3.67, 7.40, 6.86, 8.34, and 7.64 percentage points, respectively, indicating that our proposed method has better anti-forgetting ability in long sequence updates. Especially in the final stage 5, the old class mIoU of our proposed method reached 51.75%, significantly higher than GUA's 42.08%; the new class mIoU also reached 21.19%, higher than GUA's 16.79%. This shows that the proposed mechanisms, such as local relation consistency distillation, reliability-aware feature transfer, and uncertainty-calibrated class balancing, can not only improve the comprehensive recognition ability of the old and new classes in a single increment, but also effectively suppress local relation drift and cumulative forgetting in multiple rounds of continuous updates, thus demonstrating stronger long-term stability.

[0054] Tables 1 and 2 present the experimental results of single-step incremental learning, with Table 1 corresponding to the S3DIS dataset and Table 2 corresponding to the ScanNet dataset. Both tables are... , and Under the three task divisions, baseline phase 0 (S) was reported respectively. 0 Phase 1 (S) after completing an incremental update 1 Results; for each setting, the mIoU (%) for the old class, the new class, and the entire class is calculated. Among them, This indicates the number of new categories introduced during the incremental process. These two tables allow for a simultaneous examination of the method's ability to retain knowledge of existing categories and its ability to learn new categories, thus providing a more comprehensive evaluation of the overall performance of the no-replay incremental semantic segmentation method.

[0055] Table 1: Experimental Results of Single-Step Incremental Class Learning on the S3DIS Dataset Table 2: Experimental Results of Single-Step Incremental Class Learning on the ScanNet Dataset Table 3 presents the results of multi-step incremental learning on the S3DIS dataset. The table lists the class-wise IoU between GUA and our proposed method at each incremental stage, from stage 1 to stage 5, along with three summary metrics: old class, new class, and all class, reflecting the performance trend of the model during continuous multi-round category expansion. Category level 0 to category level 12 represent the category-wise IoU for each semantic category; old class represents the average intersection-union ratio (IoU) of all old classes in the current stage; new class represents the average IoU of newly added classes in the current stage; and all class represents the average IoU of all categories in the current stage. Since this table covers the entire update process, it is primarily used to evaluate the method's stability, resistance to forgetting, and ability to continuously model the co-modeling of old and new categories in long-term, no-replay incremental learning.

[0056] Table 3: Experimental Results of Multi-Step Incremental Learning on the S3DIS Dataset In summary, this invention proposes a local relation consistency distillation (LRCD)-based incremental semantic segmentation method for 3D point clouds without replay. Traditional incremental semantic segmentation methods for 3D point clouds typically require access to historical data or rely on replay storage mechanisms during continuous category expansion, leading to high storage overhead, privacy risks, and training complexity. Even without replay, existing methods generally suffer from severe forgetting of old class knowledge, significant drift in local structural relationships, and difficulty in simultaneously learning new and old categories. This invention, by designing multiple collaborative modules, effectively improves semantic segmentation performance in incremental class scenarios without requiring the storage of historical point cloud samples or the replay of training data. First, a local relation consistency distillation module (LRCD) is employed. Around the local neighborhood structure induced by the teacher network, it jointly constrains the local representation of the student network from the perspectives of normalized neighborhood distance distribution and soft neighbor preference distribution, thereby maintaining the stability of the local structural relationships of the point cloud during incremental learning. Secondly, a reliability-aware feature distillation (RFD) module is employed to construct point-by-point reliability weights by integrating teacher feature response strength, prediction confidence, and neighborhood preference concentration. Differential feature transfer is implemented for different points to enhance reliable knowledge transfer and reduce the interference of uncertain regions on model updates. Furthermore, this invention introduces an uncertainty-calibrated class balance (UCCB) module, which fuses class frequency information with class-level uncertainty to dynamically reweight the supervised loss. This enables the model to learn new classes more stably during the incremental phase while maintaining the discriminative ability of old classes. Finally, a teacher-student no-replay incremental learning framework is used to integrate the above modules, achieving collaborative modeling of local structure preservation, feature transfer optimization, and class imbalance correction for 3D point clouds. This further improves the overall segmentation performance of the model during continuous class expansion. Experimental results show that the proposed method achieves competitive results in single-step and multi-step incremental semantic segmentation tasks on the S3DIS and ScanNet datasets. In particular, it demonstrates superior old class preservation ability and global segmentation performance in the S3DIS long-term multi-step incremental evaluation, validating the effectiveness of this invention.

[0057] Furthermore, embodiments of this application also disclose an electronic device, Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0058] Figure 6This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the incremental semantic segmentation method for 3D point clouds based on local consistency distillation without playback disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be a computer.

[0059] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0060] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0061] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0062] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. It can be Windows Server, Netware, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the incremental semantic segmentation method for 3D point clouds based on local consistency distillation without playback disclosed in any of the aforementioned embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0063] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed incremental semantic segmentation method for 3D point clouds based on local consistency distillation without playback. The specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0064] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0065] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software module may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, CD-ROMs (Compact Disc-Read Only Memory), or any other form of storage medium known in the art.

[0066] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0067] The solution provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for incremental semantic segmentation of 3D point clouds without playback based on local consistency distillation, characterized in that, Includes the following steps: S11. Data acquisition: Acquire the 3D point cloud data of the current stage, wherein the 3D point cloud data only contains the annotation information of the newly added categories; S12. Construct a teacher-student framework: Input the 3D point cloud data into the teacher-student framework. The teacher-student framework includes a frozen teacher model and a student model to be updated. Extract the old class knowledge features and teacher prediction results of the 3D point cloud data through the teacher model. Extract the current prediction features and student prediction results of the 3D point cloud data through the student model. S13. Perform local relation consistency distillation: Based on the old class knowledge features and the current prediction features, jointly align the normalized neighborhood distance distribution and the soft neighbor preference distribution, and output the local relation consistency distillation loss. Specifically, this includes: for each anchor point, in the feature space corresponding to the old class knowledge features output by the teacher model, calculating the similarity between the anchor point and other points using cosine similarity, and constructing a fixed neighborhood index for the anchor point based on the similarity; reusing the fixed neighborhood index to the student model, establishing the relative distance ratio between the anchor point and the neighbor nodes corresponding to the fixed neighborhood index as the normalized neighborhood distance distribution; establishing the ranking preference of the teacher model for the importance of each neighbor node corresponding to the fixed neighborhood index as the soft neighbor preference distribution; aligning the normalized neighborhood distance distribution and the soft neighbor preference distribution corresponding to the teacher model and the student model respectively, and outputting the local relation consistency distillation loss. S14. Perform reliability-aware feature distillation: Calculate point-level reliability weights based on the old class knowledge features, the teacher prediction results, and the soft neighbor preference distribution. Use the point-level reliability weights to adaptively weight the point-level feature differences between the old class knowledge features and the current prediction features, and output the reliability-aware feature distillation loss. S15. Perform uncertainty calibration class balance: Construct dynamic balance weights based on the inverse frequency factors of each category and the dynamically updated category-level uncertainty in the current stage of 3D point cloud data. Use the dynamic balance weights to reweight the supervised segmentation loss of the student model and output the uncertainty calibration class balance loss. S16. Joint optimization of teacher-student framework: Construct a total training objective by combining the local relation consistency distillation loss, the reliability-aware feature distillation loss, and the uncertainty calibration class balance loss. Optimize and update the student model using the total training objective. Output the current stage's 3D point cloud semantic segmentation result using the optimized and updated student model.

2. The incremental semantic segmentation method for 3D point clouds based on local consistency distillation without playback as described in claim 1, characterized in that, The calculation method for the normalized neighborhood distance distribution includes: Calculate the cosine distance between the anchor point and the features of each of its neighbor nodes in the teacher model and the student model, respectively. The cosine distance is smoothed using an exponential function and normalized using temperature parameters to obtain the teacher-normalized neighborhood distance distribution induced by the teacher model and the student-normalized neighborhood distance distribution output by the student model.

3. The incremental semantic segmentation method for 3D point clouds based on local consistency distillation without playback as described in claim 2, characterized in that, The calculation methods for the soft neighbor preference distribution and the local relation consistency distillation loss include: Based on the cosine similarity between the anchor point and its neighbor node features in the teacher model and the student model, and combined with the temperature parameter, the teacher soft neighbor preference distribution induced by the teacher model and the student soft neighbor preference distribution output by the student model are obtained respectively. Calculate the Jensen-Shannon divergence between the teacher's normalized neighborhood distance distribution and the student's normalized neighborhood distance distribution, as the distance distribution alignment loss; Calculate the Kullback-Leibler divergence between the teacher soft neighbor preference distribution and the student soft neighbor preference distribution, and use it as the preference distribution constraint loss; The distance distribution alignment loss is added to the preference distribution constraint loss to output the local relation consistency distillation loss.

4. The incremental semantic segmentation method for 3D point clouds based on local consistency distillation without playback as described in claim 3, characterized in that, In step S14, the point-level reliability weight is calculated based on the old class knowledge features, the teacher prediction results, and the soft neighbor preference distribution, specifically including: The normalized feature magnitude of the old knowledge features is calculated as the feature response intensity; The entropy value of the posterior probability distribution is calculated based on the teacher's prediction results to obtain the prediction confidence. The normalized entropy complement of the teacher soft neighbor preference distribution is calculated as the neighborhood preference concentration. The feature response intensity, the prediction confidence, and the neighborhood preference concentration are fused to output a point-level reliability weight.

5. The incremental semantic segmentation method for 3D point clouds based on local consistency distillation without playback as described in claim 4, characterized in that, In the step of fusing the feature response intensity, the prediction confidence, and the neighborhood preference concentration to output a point-level reliability weight: The feature response intensity, the prediction confidence, and the neighborhood preference concentration are multiplied and fused to obtain the product result; The Sigmoid function is used to map the product result; The result of the mapping operation is normalized within the current batch, and the final point-level reliability weight is output.

6. The incremental semantic segmentation method for 3D point clouds based on local consistency distillation without playback, as described in claim 1 or 5, is characterized in that... In step S14, the point-level reliability weights are used to adaptively weight the point-level feature differences between the old class knowledge features and the current predicted features, and the reliability-aware feature distillation loss is output, specifically including: Calculate the norm distance between the old knowledge features and the current predicted features as the feature difference; The point-level reliability weight is multiplied by the feature difference to obtain the weighted difference value at the corresponding feature position; The weighted difference values ​​of all feature locations within the current batch are averaged, and the reliability-aware feature distillation loss is output.

7. The incremental semantic segmentation method for 3D point clouds based on local consistency distillation without playback as described in claim 1, characterized in that, Step S15 specifically includes: Count the number of points in each semantic category in the current batch of 3D point cloud data, and calculate the inverse frequency factor of each category based on the reciprocal relationship of the number of points in each category; Obtain the prediction entropy value of the teacher model for the old category, and fix the prediction entropy value as the category-level uncertainty of the old category; Obtain the predicted probability of students in the newly added category in the current batch, calculate the entropy value of the newly added category, and use the exponential moving average method to update the category-level uncertainty of the newly added category online based on the entropy value of the newly added category; Combine the category-level uncertainty of the old category with the category-level uncertainty of the newly added category, and output the category-level uncertainty statistics of all current categories; The inverse frequency factor of each category is multiplied by the corresponding category-level uncertainty statistic to obtain the basic weight of each category; The basic weights are normalized across all categories in the current batch, and the dynamic balance weights are output. The dynamic balance weights are multiplied with the supervised cross-entropy segmentation loss of the corresponding category to output the uncertainty calibration class balance loss.

8. The incremental semantic segmentation method for 3D point clouds based on local consistency distillation without playback as described in claim 1, characterized in that, Step S16 specifically includes: The total training objective is obtained by weighting and summing the local relation consistency distillation loss, the reliability perception feature distillation loss, and the uncertainty calibration class balance loss using a preset balance coefficient. The network parameters of the student model are optimized and updated based on the overall training objective using the backpropagation algorithm. At the end of the current incremental phase, the optimized and updated student model will be frozen and used as the teacher model for the next incremental phase.

Images