A three-dimensional point cloud-oriented incremental semantic segmentation method and system

Abstract

The application discloses a kind of three-dimensional point cloud-oriented incremental semantic segmentation method and system, by establishing the two-stage segmentation framework of the link of basic training stage and incremental training stage, obtain stable representation in old class in basic stage, in incremental stage, through the old class pseudo-label recovery mechanism of cross-stage class semantic anchoring mechanism, spatial uncertainty and class prototype double constraint, and the local structure relationship distillation mechanism of new and old model, the balance between old class knowledge preservation and new class knowledge learning is realized.It is simultaneously introduced in the joint design of local geometry extraction and global state space modeling in point feature coding process, enhance the point-level representation capability in complex three-dimensional scene.The application explicitly solves the old class forgetting and semantic drift in three-dimensional point cloud class incremental semantic segmentation, improves the segmentation precision of easily confused class in complex scene, enhances the stability and robustness of sequence modeling process.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, 3D point cloud understanding and continuous learning, and specifically to an incremental semantic segmentation method and system for 3D point clouds. Background Technology

[0002] Most existing 3D point cloud semantic segmentation methods are typically based on the closed-set training assumption, meaning that all category samples can be acquired at once during the training phase, and the final model is obtained through uniform training. These methods achieve good results in static scenes, but they struggle to adapt to the application requirements of continuously increasing categories in real-world open environments. In practical deployments, the model needs to gradually learn newly emerging categories in subsequent running phases.

[0003] In existing technologies, during the incremental training phase, model parameters are updated as new categories are learned, leading to the disruption of the original discriminative boundaries of old categories. This manifests as a continuous decline in the recognition ability of old categories, i.e., catastrophic forgetting. Furthermore, in incremental class segmentation tasks, the incremental phase typically only labels new categories, while old categories are often treated as background. This causes points originally belonging to old categories to be gradually absorbed into the background by the model during training, resulting in a significant shift in the semantic representation of old categories and causing confusion between old categories, background, and new categories. Moreover, some existing methods focus on local neighborhood geometry extraction, which can capture the fine-grained structure of point clouds to some extent, but are insufficient in modeling long-distance semantic relationships; other methods emphasize global feature convergence but easily neglect point cloud boundaries, local morphology, and detailed distribution.

[0004] Existing methods, when recovering old class information, primarily generate pseudo-labels by directly relying on model output confidence scores or simple thresholds, lacking consistency verification of local spatial distribution and old class feature centers. This easily leads to the continuous feedback of incorrect predictions to the model, resulting in the accumulation of pseudo-label noise and further exacerbating the forgetting of old classes. Furthermore, existing incremental learning methods focus more on the consistency constraints of class output distribution, while paying less attention to the local coordinate relationships and feature relationships between points in a 3D point cloud. For point cloud segmentation, the old model's understanding of local spatial structure is itself crucial knowledge. Without explicit preservation of such structural relationships, the model is prone to losing its ability to express the geometric morphology of old classes during the incremental phase. Summary of the Invention

[0005] The purpose of this invention is to provide an incremental semantic segmentation method and system for 3D point clouds. By establishing a two-stage segmentation framework that connects the basic training stage and the incremental training stage, a stable representation of the old class is obtained in the basic stage. In the incremental stage, a cross-stage class semantic anchoring mechanism, an old class pseudo-label recovery mechanism with dual constraints of spatial uncertainty and class prototype, and a distillation mechanism for the local structural relationship between the old and new models are used to achieve a balance between preserving old class knowledge and learning new class knowledge, so as to solve the problems mentioned in the background art.

[0006] The present invention provides the following technical solution: an incremental semantic segmentation method for 3D point clouds, comprising the following steps: Step S1: Collect 3D point cloud data and preprocess it to construct the data organization forms for the basic stage and the incremental stage.

[0007] Preferably, the data organization for constructing the basic stage and the incremental stage includes: dividing all training data into a basic class training set and an incremental class training set; using only the basic class training set to train the basic model in the basic stage; and inputting only new class samples from the incremental class training set for continued training in the incremental stage, without calling the original basic class training samples in this stage.

[0008] Step S2: Build a segmentation model and learn basic category representations during the basic training phase.

[0009] Preferably, the construction of the segmentation model and the learning of basic category representations in the basic training phase includes: constructing a segmentation backbone network for class-incremental semantic segmentation, training on the basic class dataset to obtain a basic model, and obtaining the initial segmentation capability of the basic categories after the basic training phase is completed. Preferably, the segmentation backbone network includes: a local geometric feature extraction unit, a cross-stage category semantic anchoring unit, a global context modeling unit, and a segmentation classification head; the local geometric feature extraction unit is used to encode the local neighborhood of the center point, construct the local neighborhood, and extract local geometric features.

[0010] Step S3: Construct a cross-stage category semantic anchoring mechanism.

[0011] Preferably, the constructed cross-stage category semantic anchoring mechanism includes: establishing cross-stage semantic information for each basic category, including category semantic anchor vectors and category feature prototypes; providing a point feature sequence; mapping the category semantic anchor vectors to the same space through linear projection; introducing only the semantic anchor vectors of the basic categories during the basic training stage; introducing the semantic anchor vectors of both the basic categories and the current new categories during the incremental training stage to form a category semantic anchor vector set; and combining the category semantic anchor vector set with the point feature sequence to form an enhanced input.

[0012] Step S4: Perform global context modeling and form enhanced representations in the base phase and incremental phase.

[0013] Preferably, the input features are mapped to content branches and gating branches; one-dimensional convolution and state space modeling are performed on the content branches to obtain forward context features.

[0014] Step S5: Construct complementary enhancement branches of feature dimensions to supplement context information and obtain supplementary context information without changing the point order structure.

[0015] Preferably, a reverse mapping is performed on the feature dimensions of the content branch to construct enhanced branch features that are complementary to the feature dimensions, and the forward context features and enhanced branch features are fused through a gating mechanism.

[0016] Step S6: Perform old class pseudo-label recovery during the incremental training phase.

[0017] Step S7: Use spatial uncertainty constraints to screen candidate points of the old class.

[0018] Preferably, for any point in the incremental stage, its coordinate space is found. Find the nearest neighbor points; calculate the spatial weighted probability of any point with respect to the predicted class of its nearest neighbors; construct an uncertainty index for any point; when the uncertainty index is less than the set uncertainty threshold, proceed to the old class prototype consistency verification.

[0019] Step S8: Perform consistency verification by freezing the category prototype and restore the old class pseudo-labels.

[0020] Preferably, after the basic training phase, the feature prototypes of each basic category are statistically analyzed; during the incremental training phase, the distance between the current point feature and the category prototype is calculated; at the same time, a category adaptive threshold is defined; and the predicted category is restored to the old class pseudo-label or marked as an invalid pseudo-label according to the set conditions.

[0021] Step S9: Preserve the local geometry of the old model through distillation of local structural relationships.

[0022] Preferably, during the incremental training phase, local structural relationship distillation between the new and old models is introduced; the current model and the frozen old model are defined respectively; key points are selected for the input point cloud, and a local neighborhood and a defined local structural relationship vector are constructed for each key point; the corresponding local structural relationships are extracted from the current model and the frozen old model respectively; and then the structural distillation loss is defined.

[0023] Step S10: Jointly optimize the classification loss and the structural distillation loss to obtain the final incremental segmentation model; Step S11: Perform inference on the point cloud to be segmented and output the joint segmentation result of the old and new categories.

[0024] On the other hand, the present invention also provides a system for preserving and restoring old classes in incremental semantic segmentation of 3D point cloud classes, which executes a method for preserving and restoring old classes in incremental semantic segmentation of 3D point cloud classes, including the following functional modules: The data organization and input representation module is used to construct basic class datasets and incremental class datasets, and to represent the input point cloud as a unified point-level attribute form; Local geometric feature extraction module: used to encode the neighborhood of the point cloud and extract fine-grained geometric features; Category semantic anchoring module: used to build and introduce category semantic anchor vectors in the base and incremental phases; Global Context Modeling Module: Used to perform state-space modeling on point feature sequences; Feature Dimension Complementary Enhancement Module: Used to construct complementary context branches through feature dimension inversion mapping; Old category pseudo-label recovery module: used to recover old category pseudo-labels based on spatial uncertainty constraints and category prototype consistency constraints; Structural Relationship Distillation Module: Used to maintain consistency of local structural relationships between the frozen old model and the current model; Model training module: used to update model parameters based on classification loss and structural distillation loss; Segmentation Execution Module: Used to output the point-level semantic segmentation results of the point cloud to be segmented.

[0025] Compared with the prior art, the beneficial effects achieved by the present invention are: (1) A special old class preservation mechanism is established for the class incremental segmentation scenario: This invention is not for static closed set segmentation, but for the problem of continuous learning of new classes under the condition that old class samples are inaccessible. Through basic stage representation construction, incremental stage semantic recovery and structural distillation, the problem of old class forgetting and semantic drift in 3D point cloud class incremental semantic segmentation is explicitly solved.

[0026] (2) Effectively suppresses old class from being absorbed by the background: This invention proposes a dual-guided old class pseudo-label recovery mechanism based on spatial uncertainty and class prototype consistency. Compared with pseudo-label schemes that rely solely on confidence, it can more accurately recover old class points from the background space and reduce semantic degradation caused by old class being mistakenly absorbed into the background.

[0027] (3) Better preservation of local geometric knowledge of old model: This invention constrains the relative coordinate relationship and relative feature relationship of new and old models in the neighborhood of key points by distilling local structural relationship. Compared with the method of only maintaining the consistency of classification output, it is more suitable for 3D point cloud tasks and can more effectively preserve the geometric structure cognition of old categories.

[0028] (4) Local geometry and global context can be modeled collaboratively: This invention combines local neighborhood geometry extraction with global state space modeling to establish a unified representation mechanism between fine-grained local structure and long-range dependency, which helps to improve the segmentation accuracy of easily confused categories in complex scenes.

[0029] (5) Enhance global modeling stability without destroying the point sequence structure: This invention enhances the construction of supplementary context information by complementary feature dimension enhancement branches. Instead of directly changing the topological order of the point sequence, it forms a complementary perspective at the feature dimension level, which enhances the stability and robustness of the sequence modeling process.

[0030] (6) More balanced learning effect between new and old categories: The present invention has achieved better performance for both new categories and overall categories under different increment settings, and is close to the upper limit of joint training, indicating that the present invention can achieve a better balance between maintaining old knowledge and learning new knowledge. Attached Figure Description

[0031] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the overall framework of the incremental semantic segmentation method for 3D point clouds in this embodiment of the invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Example 1 Combination Figure 1 As shown, this invention provides an incremental semantic segmentation method for 3D point clouds, suitable for indoor 3D point cloud scene understanding tasks. In this embodiment, the invention uses an indoor point cloud scene as input. In the basic stage, it first learns basic categories such as background, bookshelves, tables, trash cans, footstools, floors, and walls. Then, in the incremental stage, new categories such as chairs, sofas, and armchairs are introduced. Under the condition that the training samples of the old categories are no longer accessible, the segmentation capability of the old categories is maintained while the new categories are learned. The overall settings, category division methods, and joint segmentation objectives of the basic and incremental stages are consistent with the incremental point cloud semantic segmentation settings in the original document.

[0034] In this embodiment, the method of the present invention includes the following operational steps: Step S1: Collect 3D point cloud data and preprocess it to construct the data organization forms for the basic stage and the incremental stage.

[0035] In this embodiment, by collecting 3D point cloud data of an indoor scene and adding color information and normalized coordinate information to each point, the present invention can handle any input sample. It is constructed in the following form: in, This represents the number of points in the sample. Each point contains nine attributes, namely three-dimensional spatial coordinates. Color components and normalized coordinates .in Used to describe the position of a point in the original 3D scene; Used to provide color and appearance information; This represents the coordinate information after scale normalization, used to assist the model in learning a unified spatial distribution.

[0036] For example, the present invention divides all training data into basic class training sets. and incremental training set Basic training set It includes categories such as background, bookshelves, tables, trash cans, footstools, floors, and walls; incremental training set. It includes categories such as chairs, sofas, and armchairs. The basic stage only uses... Training the base model; the incremental phase only inputs... The new class samples are used for further training, and the original training samples of the base class are no longer called during this stage. In specific preprocessing, this invention can employ the following operations: Random sampling or block sampling of the point cloud is performed to keep the number of input points for each training session within a preset range; The coordinates are translated and scaled. Standardize the color components; During the training phase, random rotation, random jitter, and random scaling are performed on the point cloud to enhance the model's robustness to scene changes. These preprocessing methods are standard operations for point cloud tasks, serving to provide stable input for subsequent local geometry extraction and global context modeling.

[0037] Step S2: Construct a basic segmentation model and learn the local geometric representation and global semantic representation of the old categories in the basic stage.

[0038] In this embodiment, the present invention constructs a segmentation backbone network for class-incremental semantic segmentation, which segments the base class dataset. The model is trained to obtain a base model. This backbone network includes local geometric feature extraction units, cross-stage category semantic anchoring units, global context modeling units, and segmentation / classification heads. Its overall structure corresponds to the local-global joint modeling framework in the document.

[0039] For example, the local geometric feature extraction unit of the present invention is used to encode the local neighborhood of the center point, construct the local neighborhood, and extract local geometric features. That is, for the center point in the input point cloud, a neighborhood query method is used to construct its local point set. Let the local neighborhood input tensor be: in, This indicates the batch size, which is the number of samples in each training batch; This indicates the number of center points extracted in the current batch; This represents the number of neighboring points corresponding to each center point; This represents the dimension of the input feature for each neighboring point. This input feature is typically composed of coordinates, color, and intermediate layer features.

[0040] By first dividing the neighborhood of each center point... The features of the nth neighboring points are rearranged into a form suitable for convolution processing, and then local structural features are extracted by sharing a one-dimensional convolution and several residual convolution blocks. For the nth... The local geometric representation of a center point is defined as: in, Indicates the first The set of neighborhood points of a central point; Indicates the first The first center point corresponds to the Input features of 1 neighboring points; This represents a local feature transformation function composed of shared convolution and residual convolution; This indicates that a max pooling operation is performed along the neighborhood point dimension. Through the above calculation, this invention can compress a set of neighborhood points into a local geometric representation of a center point, thereby obtaining the output feature tensor: ,in, Indicates the dimension of local features.

[0041] In specific implementations, for example, for a center point belonging to the "bookshelf" category, its neighboring points are often distributed near regular planes and boundary corners; while for a center point belonging to the "trash can" category, its neighboring points tend to be located near the cylindrical sidewalls or the edge of the can opening. Through the local geometry extraction unit, the model can learn the differences in local morphology between different categories. That is, after the basic training phase is completed, the model obtains an initial segmentation ability for the basic categories and further statistically analyzes the feature prototype representations of the old categories for use in the subsequent incremental phase of semantic recovery of the old categories.

[0042] Step S3: Construct old category feature prototypes in the basic stage and form category semantic anchoring information that can be inherited across stages.

[0043] In this embodiment, in order to maintain the semantics of the old categories in the subsequent incremental stages, the present invention not only trains the segmentation network in the basic training stage, but also establishes two levels of cross-stage semantic information for each basic category: category semantic anchor vector and category feature prototype.

[0044] For example, this invention constructs category semantic anchor vectors, assuming the basic category set is... And the first The new category set in the next incremental training phase is This vector learns global semantic features at the category level through training, rather than being limited to the local geometric description of a single sample.

[0045] Given a feature sequence of points: ,in, Indicates the sequence length. This represents the feature dimension of each point. For each category... Define a learnable category semantic anchor vector. The category semantic anchor vectors are mapped to the same space through linear projection: ,in, Represents a linear projection matrix; This represents the mapped category semantic anchor vector.

[0046] In the basic training phase, only semantic anchor vectors of the basic categories are introduced; in the second phase... In the incremental training phase, semantic anchor vectors of both the base category and the new category are introduced simultaneously. The semantic anchor vectors of the old category are given priority in context modeling, followed by the semantic anchor vectors of the new category, to mitigate semantic drift of the old category. The set of category semantic anchor vectors is then combined with the point feature sequence to form the enhanced input. : in, Indicates the first The set of category semantic anchor vectors corresponding to the next incremental stage. This indicates a concatenation operation, a mechanism used to maintain consistency in category semantics during cross-stage training.

[0047] For example, the present invention further constructs a category feature prototype, which, after training in the basic stage, is used for each category. The mean of all labeled points in the intermediate feature space is calculated to obtain the feature prototype of this category. : in, Indicates category The characteristic prototype; Represents the mathematical expectation; Indicates belonging to a category The representation of points in the feature space. For example, in this embodiment, the prototype vector of the "bookshelf" category reflects the structural and texture features common to points of this category in multiple training samples, while the prototype vector of the "ground" category reflects commonalities such as large flat surfaces, low curvature, and continuous layout. The above category prototypes will serve as stable references for semantic recovery of old categories in subsequent incremental stages.

[0048] Step S4: Perform global context modeling and form enhanced representations in the base phase and incremental phase.

[0049] In this embodiment, to establish long-range dependencies between points, the present invention performs state-space modeling on the point feature sequence, thereby enhancing the input. Input a global context modeling unit to establish long-range dependencies between points.

[0050] For example, the input features are first mapped to content branches. and gated branches : in, Represents the feature sequence of the input points; This represents the content branch mapping parameter matrix; Represents the gating branch mapping parameter matrix; Indicates content branching characteristics; This indicates the gating branch feature.

[0051] Then, one-dimensional convolution and state space modeling are performed on the content branches to obtain the forward context features. : in, One-dimensional convolution represents the forward content branch; Represents the forward state space model; This represents the result of forward context modeling.

[0052] Furthermore, the update process of the state-space model can be written as: in, Indicates the first in the sequence One location; Indicates the first One input feature; Indicates the first One hidden state; Indicates the first One output; Indicates state transition parameters; Indicates the input mapping parameters; This represents the output mapping parameters. Through this modeling process, the model can encode the spatial semantic relationships between distant points into the representation of the current point. In a complete case, for example, the local neighborhood of a "chair" point may be similar to the local neighborhood of an "armchair," but by combining the spatial distribution of distant points, the model can use the global context to determine whether the point is more likely to belong to a regular chair or an armchair.

[0053] Step S5: Construct complementary enhancement branches of feature dimensions to supplement contextual information, so as to obtain supplementary contextual information without changing the point order structure.

[0054] In this embodiment, in order to enhance the stability of global modeling and obtain complementary feature perspectives without changing the topological order of point feature sequences, the present invention does not directly reverse the spatial order of point sequences, but performs reverse mapping on the feature dimensions of content branches to construct complementary enhancement branches.

[0055] For example, for any point's content feature vector It is expressed as follows: Its reverse mapping result Defined as: in, These respectively indicate that the feature of this point is in the 1st to the 1st... Components on each channel; This represents the vector after the feature dimensions have been reversed.

[0056] After performing this operation on the entire content branch feature sequence, the enhanced branch output is obtained. : in, Represents the inversion of the feature dimension; This indicates that the original channel order has been restored; This indicates an enhanced branched convolutional transform; This represents an enhanced branch state-space modeling unit; This indicates that the branch output is enhanced.

[0057] Finally, a gating mechanism is used to fuse the forward context features and the enhanced branch features to obtain the fused context enhancement amount. Features after residual update : in, Represents the Sigmoid function; This represents element-wise multiplication. In this invention, the design provides a supplementary perspective without altering the spatial topology of the point sequence, thereby mitigating the representational bias caused by single-path modeling and enhancing the stability of the global representation without destroying the point sequence structure.

[0058] Step S6: Perform old class pseudo-label recovery during the incremental training phase.

[0059] In this embodiment, the invention enters the incremental phase, using only samples of the new category. Training is then performed. Basic class samples are no longer included in the current training phase. Since old classes lack explicit annotations during the incremental phase, this invention performs pseudo-label recovery on background points that may belong to the old class.

[0060] For example, after the basic stage is completed, the present invention fixes the model of the previous stage as the old model. And the model that is currently being trained is denoted as At this point, the original training samples of the base class are removed, and only the incremental class dataset is input. The model trains on samples from existing datasets. For example, in this case, the incremental phase only inputs new category data such as "chair," "sofa," and "armchair," instead of the original samples from basic categories such as "bookshelf," "table," "trash can," "floor," and "wall." Since the old categories no longer have true labels in the incremental phase, points that originally belonged to the old categories are often treated as background in the training labels. Without processing, the model will gradually absorb these points into the background category, leading to semantic drift of the old categories. Therefore, this embodiment introduces a dual-guided old category pseudo-label recovery mechanism in the incremental phase.

[0061] Step S7: Screen candidate points of the old class through spatial uncertainty constraints.

[0062] In this embodiment, for any point in the incremental phase First, find its coordinate space Let there be nearest neighbors. Point Its first Normalized geometric weights among nearest neighbors Indicates the first The nearest neighbors were predicted as category. The probability of point For category The spatially weighted probability is defined as: in, Point Under spatial neighborhood constraints, it belongs to the category The weighted probability.

[0063] Furthermore, an uncertainty index is constructed for this point: in, Point The uncertainty of prediction; This represents the entropy calculated based on the spatially weighted category probabilities; This represents the entropy obtained by predicting the distribution based on the neighborhood average.

[0064] when At that time, the point is considered to be a point of low uncertainty, where, This represents the uncertainty threshold. Only points that meet this condition will proceed to the next step of old class prototype consistency verification.

[0065] In this case, suppose a certain "bookshelf" old class point is initially predicted as background by the model in the incremental stage, but multiple nearby points show a stable prediction pattern similar to the bookshelf. Then its uncertainty value is low, so it is retained as an old class recovery candidate point.

[0066] Step S8: Perform consistency verification by freezing the category prototype and restore the old class pseudo-labels.

[0067] In this embodiment, after the basic training phase is completed, the present invention performs training on each basic category. Statistical analysis of its characteristic prototypes: ,in, Indicates category The characteristic prototype; Represents the mathematical expectation; Indicates belonging to a category Point features.

[0068] During the incremental training phase, candidate points that have passed the uncertainty screening are... Take its current feature representation Each of the old category prototypes Calculate the cosine distance: in, Represents the cosine distance between point features and category prototypes; Represents the dot product of vectors; and Let represent the Euclidean norms of the two vectors respectively.

[0069] To accommodate the differences in feature distribution within different old categories, a category-adaptive threshold is defined: in, Indicates the magnification factor; Indicates category The degree of discreteness of the characteristics; Indicates feature dimension; Indicates category In the Variance across each feature dimension.

[0070] If the candidate point satisfies This indicates that the point is similar to the old category in the feature space. If the prototype is close enough, it can be restored as an old category pseudo-label; otherwise, it is marked as an invalid label and will not participate in the old category supervision.

[0071] In this embodiment, if the distance between a candidate point's feature and the "bookshelf" prototype is significantly less than the distance between it and the "ground" or "wall" prototypes, and the corresponding adaptive threshold is met, then the point is relabeled as a "bookshelf" pseudo-label and is no longer considered background. Thus, this embodiment completes the recovery of old class semantics from the background space, forming a dual-guided old class semantic recovery mechanism based on spatial uncertainty and class prototype consistency.

[0072] Step S9: Preserve the local geometry of the old model through structural relation distillation.

[0073] In this embodiment, to further suppress the forgetting of old categories, the present invention introduces local structural relationship distillation between the old and new models during the incremental training phase. Let the current model be... Freeze the old model as In order to avoid Disruption during the learning of new categories Based on the existing local geometric relationships of the old categories, this invention introduces structural relationship distillation.

[0074] For example, first select the input point cloud. There are 1 key point, and a size of 1 is built for each key point. The local neighborhood of . For the . The key points, whose local structural relationship vectors are defined as follows: in, Indicates the first Local structural relationship vectors of key points; Indicates key points and the first Attention weights between neighboring points; Indicates the first The key point and the first The relative coordinate offset of each neighboring point; Indicates the first The key point and the first The relative feature offset between neighboring points.

[0075] Then, from the current model respectively And freeze old models Extract the corresponding local structural relationships and The structural distillation loss is defined as: in, Indicates structural distillation loss; This represents the squared Euclidean distance. Through this loss term, Being constrained to maintain as much as possible when learning new categories Similar local structural relationships are expressed.

[0076] In this case, even though a large number of old category samples such as "bookshelf" and "table" were not seen during the incremental phase, It will still try to get as close as possible to the local structure. Understanding the geometric relationships of the old class reduces forgetting and preserves the old model's ability to recognize local geometric topological relationships.

[0077] Step S10: Jointly optimize the classification loss and the structural distillation loss to obtain the final incremental segmentation model.

[0078] In this embodiment, the total loss during the incremental training phase is defined as: in, Indicates the total loss; The classification loss includes both the supervision generated by the true labels of the new categories and the supervision generated by the pseudo labels of the old categories. Indicates structural distillation loss; The balancing coefficient is used to adjust the weight of the structural distillation term in the overall objective. By jointly optimizing the total loss, a final model that can simultaneously segment the base class and the new class is obtained.

[0079] For example, in actual training, the AdamW optimizer can be used, and the basic and incremental training phases can be performed according to the training strategy in the paper. In the implementation given in the paper, the model is based on PyTorch, trained on a single NVIDIA 4090 GPU, with 100 epochs and an initial learning rate of... The weight decays to And segmented learning rate decay is performed during training; in the dual-guided pseudo-label recovery module, the uncertainty threshold is at Scheduling within a specified range. In the patented embodiments, these parameters can be adjusted according to different hardware platforms and data scales, but do not affect the technical essence of the present invention.

[0080] Step S11: Perform inference on the point cloud to be segmented and output the joint segmentation result of the old and new categories.

[0081] In this embodiment, after the model training is completed, a new 3D point cloud to be segmented is input into the trained model. The model first obtains fine-grained local representations through the local geometric feature extraction unit, then obtains global enhanced features through the category semantic anchoring and global context modeling units, and finally outputs point-level semantic labels by the classification head. This label set simultaneously covers the old categories learned in the basic stage and the new categories learned in the incremental stage.

[0082] For example, in a preferred embodiment, when an indoor scene containing "walls, floors, bookshelves, tables, chairs, and sofas" is input, the final model can continue to correctly identify the "walls, floors, bookshelves, and tables" already learned in the basic stage; at the same time, it can identify the "chairs and sofas" newly learned in the incremental stage; and it can also avoid misclassifying some old category points into the background. Thus, this invention achieves a better balance between preserving old categories and learning new categories. Experimental results in the manuscript also show that this method achieves better overall performance than the comparison methods under various incremental settings.

[0083] Example 2 In this embodiment, the present invention also provides an incremental semantic segmentation system for 3D point clouds, including the following functional modules: Data organization and input representation module: used to collect or read 3D point cloud data and organize each point into a nine-dimensional representation containing 3D coordinates, color information and normalized coordinates; it also performs sampling, normalization and data augmentation operations; and divides the data into basic stage datasets and incremental stage datasets.

[0084] Local geometric feature extraction module: used to construct a local neighborhood for the center point, and perform local geometric encoding on the neighborhood points through shared convolution and residual convolution, outputting local geometric features at the center point level.

[0085] Category semantic anchoring module: It is used to construct category semantic anchor vectors for base categories and incremental categories, and inject corresponding semantic anchoring information in the base stage and incremental stage respectively to maintain the unified semantic space of the old and new categories.

[0086] Global Context Modeling Module: This module maps point feature sequences to content branches and gating branches, and extracts long-range contextual dependencies based on the state-space model.

[0087] Feature Dimension Complementary Enhancement Module: This module performs feature dimension inversion mapping on content branch features to form complementary enhancement branches, and performs gated fusion with forward context features to improve global modeling stability.

[0088] Old Class Pseudo-Label Recovery Module: Used to perform spatial uncertainty assessment and class prototype consistency verification on candidate background points during the incremental phase, thereby recovering valid old class pseudo-labels.

[0089] Structural Relations Distillation Module: This module calculates the structural relation differences between the current model and the frozen old model in the local neighborhood and generates a distillation loss term to preserve the old category geometric relation knowledge.

[0090] Model training module: Used to perform parameter updates based on classification loss and structural distillation loss, completing the basic training phase and incremental training phase.

[0091] Segmentation Execution Module: This module receives the point cloud to be segmented after training, performs forward inference, and outputs point-level semantic label results.

[0092] The old class preservation and semantic recovery method for incremental semantic segmentation of 3D point cloud classes in Example 1 is implemented through the system of this embodiment.

[0093] To address the problems in existing technologies regarding incremental semantic segmentation of 3D point clouds, such as the easy forgetting of old category knowledge, easy drift of old category semantics, easy misabsorption of old category points into the background, difficulty in preserving local structural relationships, and difficulty in co-modeling local geometry and global context, this invention establishes a two-stage segmentation framework that connects the basic training stage and the incremental training stage. In the basic stage, stable representations of old categories are obtained. In the incremental stage, a cross-stage category semantic anchoring mechanism, an old category pseudo-label recovery mechanism with dual constraints of spatial uncertainty and category prototype, and a distillation mechanism for local structural relationships between the old and new models are used to achieve a balance between preserving old category knowledge and learning new category knowledge. Simultaneously, this invention introduces a joint design of local geometry extraction and global state space modeling during point feature encoding to enhance point-level representation capabilities in complex 3D scenes.

[0094] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0095] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An incremental semantic segmentation method for 3D point clouds, characterized in that: The following steps are included: Step S1: Collect 3D point cloud data and preprocess it to construct the data organization forms for the basic stage and the incremental stage; Step S2: Build a segmentation model and learn basic category representations during the basic training phase; Step S3: Construct a cross-stage category semantic anchoring mechanism; Step S4: Perform global context modeling and form enhanced representations in the basic and incremental phases; Step S5: Construct complementary enhancement branches of feature dimensions to supplement contextual information and obtain supplementary contextual information without changing the point order structure; Step S6: Perform old class pseudo-label recovery during the incremental training phase; Step S7: Use spatial uncertainty constraints to screen candidate points of the old class; Step S8: Perform consistency verification by freezing the category prototype and restore the old class pseudo-labels; Step S9: Preserve the local geometry of the old model through distillation of local structural relationships; Step S10: Jointly optimize the classification loss and the structural distillation loss to obtain the final incremental segmentation model; Step S11: Perform inference on the point cloud to be segmented and output the joint segmentation result of the old and new categories.

2. The incremental semantic segmentation method for 3D point clouds according to claim 1, characterized in that: The data organization methods for constructing the basic and incremental phases in step S1 include: All training data is divided into a basic training set and an incremental training set. In the foundational phase, only the foundational training set is used to train the foundational model; The incremental phase only inputs new class samples from the incremental class training set for continued training, and the original training samples of the base class are no longer called during this phase.

3. The incremental semantic segmentation method for 3D point clouds according to claim 2, characterized in that: Step S2, which involves building a segmentation model and learning basic category representations during the basic training phase, includes: A segmentation backbone network for incremental semantic segmentation of classes is constructed, and a basic model is obtained by training on a basic class dataset. After the basic training phase is completed, the basic model obtains the initial segmentation ability of the basic classes. The segmentation backbone network includes: a local geometric feature extraction unit, a cross-stage category semantic anchoring unit, a global context modeling unit, and a segmentation classification head; The local geometric feature extraction unit is used to encode the local neighborhood of the center point, construct the local neighborhood, and extract local geometric features.

4. The incremental semantic segmentation method for 3D point clouds according to claim 3, characterized in that: The cross-stage category semantic anchoring mechanism constructed in step S3 includes: For each base category, establish cross-stage semantic information including category semantic anchor vectors and category feature prototypes; Given a feature sequence of points; Map category semantic anchor vectors to the same space using linear projection; During the basic training phase, only semantic anchor vectors of basic categories are introduced; During the incremental training phase, semantic anchor vectors of both the base category and the current new category are introduced simultaneously to form a set of category semantic anchor vectors. The set of category semantic anchor vectors is combined with the point feature sequence to form an enhanced input.

5. The incremental semantic segmentation method for 3D point clouds according to claim 4, characterized in that: Step S4 includes: Map the input features to content branches and gating branches; One-dimensional convolution and state space modeling are performed on the content branches to obtain forward context features.

6. The incremental semantic segmentation method for 3D point clouds according to claim 5, characterized in that: Step S5 includes: performing reverse mapping on the feature dimensions of the content branch, constructing enhanced branch features that are complementary to the feature dimensions, and fusing the forward context features and enhanced branch features through a gating mechanism.

7. The incremental semantic segmentation method for 3D point clouds according to claim 6, characterized in that: Step S7 includes: For any point in the incremental phase, find its coordinate space. One nearest neighbor point; Calculate the spatially weighted probability of any point relative to the predicted classes of its nearest neighbors; Then construct an uncertainty index for any point; When the uncertainty index is less than the set uncertainty threshold, proceed to the old prototype consistency verification.

8. The incremental semantic segmentation method for 3D point clouds according to claim 7, characterized in that: Step S8 includes: After the basic training phase is completed, the feature prototypes of each basic category are statistically analyzed. During the incremental training phase, the distance between the current point feature and the category prototype is calculated; Simultaneously define category-adaptive thresholds; Based on the set conditions, the predicted category will be restored to the old pseudo-label or marked as an invalid pseudo-label.

9. The incremental semantic segmentation method for 3D point clouds according to claim 8, characterized in that: Step S9 includes: In the incremental training phase, distillation of the local structural relationships between the old and new models is introduced; Define the current model and freeze the old model separately; Select key points from the input point cloud and construct a local neighborhood and definitional structural relationship vector for each key point; Extract the corresponding local structural relationships from the current model and the frozen old model respectively; Redefine structural distillation loss.

10. An incremental semantic segmentation system for 3D point clouds, executing the incremental semantic segmentation method for 3D point clouds according to any one of claims 1-9, characterized in that: Includes the following functional modules: The data organization and input representation module is used to construct basic class datasets and incremental class datasets, and to represent the input point cloud as a unified point-level attribute form; Local geometric feature extraction module: used to encode the neighborhood of the point cloud and extract fine-grained geometric features; Category semantic anchoring module: used to build and introduce category semantic anchor vectors in the base and incremental phases; Global Context Modeling Module: Used to perform state-space modeling on point feature sequences; Feature Dimension Complementary Enhancement Module: Used to construct complementary context branches through feature dimension inversion mapping; Old category pseudo-label recovery module: used to recover old category pseudo-labels based on spatial uncertainty constraints and category prototype consistency constraints; Structural Relationship Distillation Module: Used to maintain consistency of local structural relationships between the frozen old model and the current model; Model training module: used to update model parameters based on classification loss and structural distillation loss; Segmentation Execution Module: Used to output the point-level semantic segmentation results of the point cloud to be segmented.

Images