Three-dimensional point cloud incremental learning method and system based on structure guidance and dynamic routing

Abstract

The disclosure provides a three-dimensional point cloud incremental learning method and system based on structure guidance and dynamic routing, relating to the technical field of three-dimensional vision, comprising: obtaining original three-dimensional point cloud data and multi-view two-dimensional depth maps; feature extraction is performed on them respectively to obtain three-dimensional topological features and two-dimensional visual features; a three-dimensional semantic prompt vector is extracted and injected into the two-dimensional visual features in the form of a residual for each view to obtain enhanced visual features; similarity calculation is performed based on the enhanced visual features and global three-dimensional topological features to obtain dynamic aggregation weights for each view, and the enhanced visual features of each view are weighted and fused based on the dynamic aggregation weights to obtain an optimized visual representation; text semantic embedding features of text prompt words are extracted, and the optimized visual representation is matched with the text semantic embedding features for similarity to obtain a classification prediction result. The disclosure enhances the features of high-quality discriminative views and suppresses the interference of low-quality noise views.

Description

Technical Field

[0001] This disclosure relates to the field of 3D vision technology, specifically to a method and system for incremental learning of 3D point clouds based on structure guidance and dynamic routing. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] With the rapid development of deep learning in the field of 3D vision, incremental class learning of 3D point clouds has gradually become a research hotspot. Incremental class learning requires the model to continuously learn new categories from new data while maintaining the ability to recognize previously learned categories. When each new category has only a very small number of training samples, the task further evolves into few-shot incremental class learning. In this scenario, the core challenge facing the model is how to strike a balance between stability and plasticity: it must quickly adapt to new concepts while preventing catastrophic forgetting of previous knowledge.

[0004] Recent research has begun to introduce pre-trained large-scale vision-language models, such as CLIP, to replace traditional 3D backbone networks. These methods render 3D point clouds as 2D multi-view depth maps, extract independent visual features from each viewpoint using a frozen visual encoder, then fuse multi-view information through simple heuristic aggregation operations (such as average pooling), and finally align it with text features to achieve classification. However, this existing approach still suffers from two key structural flaws: (i) Irreversible Degradation of Geometric Information. Flattening a 3D object with a complete spatial topology into a structureless 2D planar projection fundamentally discards the spatial continuity between different parts of the object, global geometric relationships, and structural complexity information. Since depth maps lack color and texture cues found in natural images, inferring object categories solely from isolated viewpoints is inherently ambiguous. For example, the bottom and side projections of a table may present completely different structural features in a depth map; a single viewpoint cannot capture the object's complete geometric semantics. This loss of geometric information cannot be fully recovered through subsequent processing, resulting in inherent ambiguity in the extracted 2D features when expressing the structural attributes of 3D objects.

[0005] (II) Suboptimal Heuristic Viewpoint Aggregation. Existing methods generally employ static global pooling when fusing multi-viewpoint features, assuming that all views contribute equally to the final classification. This assumption has serious flaws in practical applications. Since different parts of an object may produce highly occluded or feature-poor projections in 3D space (such as the bottom projection of a symmetrical object), treating these low-quality views the same as views containing rich discriminative features inevitably introduces a large amount of semantic noise during the fusion process. Under conditions of few samples, discriminative features are already extremely scarce; this indiscriminate aggregation method essentially drowns out weak, effective signals in noise, becoming a key bottleneck limiting incremental learning performance. Summary of the Invention

[0006] To address the aforementioned issues, this disclosure proposes a method and system for incremental learning of 3D point clouds based on structure guidance and dynamic routing. Without introducing catastrophic forgetting, it introduces two lightweight, learnable cross-modal interaction modules to achieve rapid adaptation to new categories, while simultaneously realizing 3D spatial enhancement of 2D features and dynamic noise reduction optimization through viewpoint fusion.

[0007] According to some embodiments, the present disclosure adopts the following technical solutions: Incremental learning methods for 3D point clouds based on structure guidance and dynamic routing include: Obtain the raw 3D point cloud data and perform multi-view projection rendering to obtain a multi-view 2D depth map; The multi-view 2D depth map is input into the 2D visual encoder, and the output is the 2D visual feature; the original 3D point cloud is input into the 3D point cloud encoder, and the output is the global 3D topological feature. The global 3D topological features are adapted by 3D feature projection to obtain 3D semantic cue vectors. The 3D semantic cue vectors are then injected into the 2D visual features in the form of residuals from one viewpoint to obtain enhanced visual features. A modal projection matrix is ​​constructed, and similarity is calculated based on enhanced visual features and global 3D topological features to obtain dynamic aggregation weights for each viewpoint. The enhanced visual features of each viewpoint are then weighted and fused based on the dynamic aggregation weights to obtain an optimized visual representation. Given image category text prompts, extract the text semantic embedding features of the text prompts, and perform similarity matching between the optimized visual representation and the text semantic embedding features to obtain the classification prediction results.

[0008] As one embodiment, the step of acquiring the original 3D point cloud data and performing multi-view projection rendering to obtain a multi-view 2D depth map includes: By uniformly arranging virtual cameras in different positions around the object, multiple multi-view 2D depth maps are generated from the original 3D point cloud; where the value of each pixel in the 2D depth map is the distance mapping from the object surface point corresponding to that pixel to the camera plane.

[0009] As one embodiment, the step of inputting a multi-view 2D depth map into a 2D visual encoder and outputting 2D visual features; and inputting the original 3D point cloud into a 3D point cloud encoder and outputting global 3D topological features includes: The rendered multi-view 2D depth maps are input into the frozen 2D visual encoder to obtain independent 2D visual features for each viewpoint. v Each perspective, its characteristics are represented as follows:

[0010] in, Represents a two-dimensional visual encoder. This represents the depth map of the i-th viewpoint obtained from the rendering. Indicates the total number of viewpoints; The original 3D point cloud is input into the frozen 3D point cloud encoder to obtain global 3D topological features:

[0011] in, This represents a 3D point cloud encoder. This represents the original 3D point cloud input.

[0012] As one embodiment, the step of performing 3D feature projection adaptation on global 3D topological features to obtain a 3D semantic cue vector includes: A lightweight cue adapter is designed for 3D feature projection adaptation. The lightweight cue adapter adopts a bottleneck structure multilayer perceptron, specifically including a linear dimensionality reduction layer, a nonlinear activation function, and a linear dimensionality increase layer. The output of the adapter is a 3D semantic cue vector with the same dimension as the 2D visual features. This 3D semantic cue vector is not globally shared, but dynamically generated for each input point cloud instance.

[0013] As one embodiment, the step of injecting the three-dimensional semantic cue vector into the two-dimensional visual features in residual form on a per-view basis to obtain enhanced visual features includes: The 3D semantic cue vector generated by the adapter is injected into the 2D visual features in residual form, view-by-view. For the ... v The enhanced visual features obtained from each perspective are:

[0014] in, It is a learnable scalar scaling factor used to adjust the injection intensity of 3D prior information; For lightweight prompt adapters, It is a two-dimensional visual feature; This refers to the global 3D topological features extracted from the original 3D point cloud.

[0015] As one embodiment, the construction of the modal projection matrix, based on enhanced visual features and global 3D topological features, performs similarity calculation to obtain dynamic aggregation weights for each viewpoint, including: Three sets of learnable modal projection matrices are set up to map the global 3D topological features to query vectors in the attention mechanism, and to map the enhanced visual features of each viewpoint to key vectors and value vectors. Through the scaling dot product attention mechanism, the similarity between the 3D query vector and the key vectors of each 2D viewpoint is calculated as the dynamic aggregation weight of each viewpoint.

[0016] According to some embodiments, the present disclosure adopts the following technical solutions: A 3D point cloud incremental learning system based on structure guidance and dynamic routing includes: The data acquisition module is used to acquire raw 3D point cloud data and perform multi-view projection rendering to obtain a multi-view 2D depth map. The feature extraction module is used to input multi-view 2D depth maps into a 2D visual encoder and output 2D visual features; and to input the original 3D point cloud into a 3D point cloud encoder and output global 3D topological features. The feature enhancement module is used to perform 3D feature projection adaptation on the global 3D topological features to obtain 3D semantic cue vectors. The 3D semantic cue vectors are then injected into the 2D visual features in the form of residuals on a per-view basis to obtain enhanced visual features. The feature fusion module is used to construct a modal projection matrix, perform similarity calculation based on enhanced visual features and global 3D topological features to obtain dynamic aggregation weights for each viewpoint, and perform weighted fusion of enhanced visual features from each viewpoint based on the dynamic aggregation weights to obtain an optimized visual representation. The point cloud classification module is used to extract the text semantic embedding features of a given image category text prompt, and then perform similarity matching between the optimized visual representation and the text semantic embedding features to obtain the classification prediction result.

[0017] According to some embodiments, the present disclosure adopts the following technical solutions: A computer program product includes a computer program that, when executed by a processor, implements the aforementioned structure-guided and dynamic routing-based incremental learning method for 3D point clouds.

[0018] According to some embodiments, the present disclosure adopts the following technical solutions: A non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the aforementioned three-dimensional point cloud incremental learning method based on structure guidance and dynamic routing.

[0019] According to some embodiments, the present disclosure adopts the following technical solutions: An electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the structure-guided and dynamic routing-based 3D point cloud incremental learning method.

[0020] Compared with the prior art, the beneficial effects of this disclosure are as follows: This disclosed method for incremental learning of 3D point clouds based on structure guidance and dynamic routing injects 3D topological priors into each 2D visual feature map in a residual manner, enabling isolated views that originally only contained local depth and contour information to perceive the overall spatial structure of objects. The design of a zero-initialization scaling factor ensures that the integrity of the pre-trained semantic priors is not compromised in the early stages of training. This mechanism allows even highly occluded or feature-poor viewpoints to achieve geometric semantic enhancement through 3D cues, fundamentally solving the problem of irreversible information loss during 3D-to-2D projection.

[0021] This disclosure presents a structure-guided and dynamic routing-based incremental learning method for 3D point clouds, achieving intelligent multi-view fusion with topology awareness. Using global 3D topological features as the query, a cross-modal attention mechanism dynamically evaluates the topological consistency of each 2D viewpoint and assigns aggregation weights accordingly. This mechanism has clear physical interpretability: high-discriminative views receive high weights due to their high fit with the 3D structure, while low-quality, noisy views receive weights approaching zero due to domain offset. Compared to the indiscriminate average pooling of existing methods, the dynamic routing of this disclosure fundamentally avoids the problem of effective signal dilution by noise, and its performance is particularly outstanding under conditions of few samples.

[0022] This disclosed incremental learning method for 3D point clouds based on structure guidance and dynamic routing combines high training efficiency with convenient engineering deployment. It requires training only a very small number of parameters, such as the cue adapter and the routing projection matrix, resulting in a very small proportion of the total system parameters. During training, the entire backbone network is frozen, eliminating the need to store historical samples, and the incremental session training time is significantly shorter than traditional fine-tuning methods. After incremental learning is complete, only lightweight module parameters need to be saved for deployment, resulting in extremely low storage overhead and making it suitable for practical applications in resource-constrained scenarios.

[0023] This disclosure presents a structure-guided and dynamic routing-based incremental learning method for 3D point clouds, significantly improving classification accuracy in incremental learning with few samples. In cross-domain ShapeNet-to-ModelNet benchmark tests, this method achieves 87.2% classification accuracy on the base class task, maintaining best performance across all six incremental sessions. Particularly in the last incremental session (after introducing 24 new classes), it maintains 78.1% accuracy, representing an absolute improvement of 11.0 percentage points compared to the existing best method, RAIL's 67.1%. This gain directly benefits from the enhanced 2D feature space perception capability of the structure-to-semantic prompting module and the effective filtering of semantic noise by the structure-to-viewpoint routing module. The synergistic effect of these two modules enables the model to extract highly discriminative features even under extremely limited sample conditions.

[0024] This disclosure presents a structure-guided and dynamic routing-based incremental learning method for 3D point clouds, effectively alleviating the stability-plasticity dilemma. This method achieves an F-score of 69.3, comprehensively measuring the ability to retain old classes and learn new classes, far exceeding RAIL's 61.3 and SAVC's 47.1. Simultaneously, the relative accuracy decrease is only 11.0%. In contrast, although the PackNet method achieves a lower decrease rate of 9.5%, its F-score is only 22.4, indicating that it maintains the stability of old classes at the cost of severely sacrificing the ability to learn new classes. This disclosure achieves a better balance between stability and plasticity by strictly freezing the backbone network to ensure stability and only updating lightweight cross-modal modules, avoiding the structural defect of "neglecting one aspect" in existing methods. Attached Figure Description

[0025] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0026] Figure 1 This is an overall architecture diagram of the 3D point cloud incremental learning method based on structure guidance and dynamic routing, according to an embodiment of this disclosure. Detailed Implementation

[0027] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0028] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0029] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0030] Example 1 One embodiment of this disclosure provides a method for incremental learning of 3D point clouds based on structure guidance and dynamic routing. The method includes the following steps: Step 1: Obtain the original 3D point cloud data and perform multi-view projection rendering to obtain a multi-view 2D depth map; Step 2: Input the multi-view 2D depth map into the 2D visual encoder and output the 2D visual features; input the original 3D point cloud into the 3D point cloud encoder and output the global 3D topological features. Step 3: Perform 3D feature projection adaptation on the global 3D topological features to obtain 3D semantic cue vectors. Inject the 3D semantic cue vectors into the 2D visual features in the form of residuals from one viewpoint to obtain enhanced visual features. Step 4: Construct a modal projection matrix, perform similarity calculation based on enhanced visual features and global 3D topological features to obtain dynamic aggregation weights for each viewpoint, and perform weighted fusion of enhanced visual features for each viewpoint based on the dynamic aggregation weights to obtain an optimized visual representation; Step 5: Given image category text prompts, extract the text semantic embedding features of the text prompts, and perform similarity matching between the optimized visual representation and the text semantic embedding features to obtain the classification prediction results.

[0031] As one embodiment, this disclosure presents a 3D point cloud incremental learning method based on structure guidance and dynamic routing. It addresses the inherent correlation between two major shortcomings of existing solutions described in the background: geometric information degradation leads to unreliable semantic representations of some 2D perspectives; and heuristic aggregation further mixes these unreliable representations with reliable ones, amplifying the adverse effects of noise. This disclosure abandons the approach of relying on independent processing of 2D perspectives, fundamentally reintroducing 3D topological structure as guidance throughout the entire process of perspective enhancement and fusion. It proposes a 3D point cloud incremental learning method, namely the S2PR framework. The S2PR framework includes a feature extraction backbone network, a structure-to-semantic cueing module, a structure-to-viewpoint routing module, and a fusion module. This method fundamentally eliminates catastrophic forgetting by strictly freezing all heavy feature extraction backbone networks (3D point cloud encoder, 2D visual encoder, and text encoder), while introducing two lightweight, learnable cross-modal interaction modules to achieve rapid adaptation to new categories: the structure-to-semantic cueing module and the structure-to-viewpoint routing module. The overall technical solution and specific implementation are described in detail below.

[0032] Step 1: Obtain the original 3D point cloud data and perform multi-view projection rendering to obtain a multi-view 2D depth map; First, for any input 3D point cloud sample, multi-view projection rendering is performed. By uniformly placing virtual cameras around the object at different locations, multiple 2D depth maps with different viewpoints are generated from the 3D point cloud. The value of each pixel in the 2D depth map is the distance mapping from the corresponding object surface point to the camera plane. This disclosure does not strictly limit the number of viewpoints, typically using 6 to 12 evenly distributed viewpoints to balance coverage integrity and computational efficiency.

[0033] Step 2: Input the multi-view 2D depth map into the 2D visual encoder and output the 2D visual features; input the original 3D point cloud into the 3D point cloud encoder and output the global 3D topological features. The overall technical architecture disclosed herein follows the design principle of "frozen backbone and lightweight adaptation". It includes three strictly frozen feature extraction backbone networks: a 3D point cloud encoder, a 2D visual encoder, and a text encoder, which are used to extract global 3D topological features, 2D visual features, and text semantic embedding features, respectively.

[0034] Furthermore, a 3D point cloud encoder is used to extract global 3D topological features from raw 3D point cloud data. This 3D point cloud encoder can employ a point cloud-based neural network architecture (such as DGCNN), with the raw 3D point cloud coordinates as input and a fixed-dimensional global geometric description vector as output. After pre-training, the 3D point cloud encoder maintains its parameters frozen throughout the entire training and inference process disclosed herein, and does not participate in any gradient updates. The original 3D point cloud is input into the frozen 3D point cloud encoder to obtain the global 3D topological feature vector:

[0035] in, This represents a 3D point cloud encoder. This indicates the input of the original 3D point cloud.

[0036] Furthermore, the 2D visual encoder employs an image encoder (such as CLIPViT-B / 32) from a pre-trained vision-language model. Its input consists of multiple 2D depth maps from various perspectives generated by rendering from a 3D point cloud, and its output is independent 2D visual features for each perspective. This 2D visual encoder is also strictly frozen and used only as a feature extractor. Specifically, the rendered multi-view depth maps are input into the frozen 2D visual encoder to obtain independent visual feature vectors for each perspective. For the... v From a single perspective, its two-dimensional visual features are represented as follows:

[0037] in, Represents a two-dimensional visual encoder. This represents the depth map of the i-th viewpoint obtained from the rendering. Indicates the total number of viewpoints.

[0038] Furthermore, the text encoder employs the same pre-trained vision-language model. Its input is a text cue template constructed from category names (such as "an image, projection, or sketch of [category]"), and its output is text semantic embedding vectors for each category, which are subsequently used for similarity matching with visual features to achieve classification.

[0039] In this disclosed technical solution, the three-dimensional features are not treated as an independent modality parallel to the two-dimensional features, but rather positioned as a "structural prior" to guide and enhance the two-dimensional visual flow. This role fundamentally determines the design logic of the subsequent two cross-modal modules: the three-dimensional features do not directly participate in classification, but rather serve as a "cue source" and "query signal" to collaboratively optimize the representation quality of the two-dimensional features.

[0040] Step 3: Perform 3D feature projection adaptation on the global 3D topological features to obtain 3D semantic cue vectors. Inject the 3D semantic cue vectors into the 2D visual features in the form of residuals from one viewpoint to obtain enhanced visual features. Specifically, this disclosure sets up a structure-to-semantic module. The core function of the structure-to-semantic prompting module is to inject three-dimensional topological priors into each isolated two-dimensional visual feature, so that each planar view can obtain global spatial perception capability, fundamentally solving the problem of geometric information degradation.

[0041] The specific structure and working principle of this structure to the semantic module are as follows: Step 31: 3D Feature Projection Adaptation. Since the 3D point cloud encoder and the 2D visual encoder come from different pre-training paradigms, their feature space distributions differ significantly. To achieve effective cross-modal information injection, this disclosure designs a learnable, lightweight cue adapter. This adapter is implemented using a bottleneck-structured multilayer perceptron, specifically comprising a linear dimensionality reduction layer, a nonlinear activation function, and a linear dimensionality increase layer. The bottleneck structure effectively reduces the number of parameters and prevents overfitting in small-sample scenarios. The mathematical expression of the adapter is:

[0042] in, and This is the learnable parameter matrix of the adapter. and For bias vectors, The activation function is non-linear (using ReLU or GELU). The middle dimension of the bottleneck structure is typically set to 1 / 4 to 1 / 8 of the input dimension to achieve a balance between expressive power and parameter efficiency.

[0043] The adapter outputs a three-dimensional semantic cue vector with the same dimensions as the two-dimensional visual features. This cue vector is not globally shared, but is dynamically generated for each input point cloud instance, thus possessing instance-specific characteristics.

[0044] Step 32: Residual Injection and Zero Initialization Strategy. The 3D semantic cue vector generated by the adapter is injected into the 2D visual features in residual form, view-by-view. For the... v From each perspective, the enhanced visual features are obtained as follows:

[0045] in, It is a learnable scalar scaling factor used to adjust the injection intensity of 3D prior information; For lightweight prompt adapters, It is a two-dimensional visual feature; This refers to the global 3D topological features extracted from the original 3D point cloud.

[0046] Furthermore, the scaling factor employs a strict zero-initialization strategy, meaning its value is initialized to zero at the start of training. The physical significance of this design is that, in the early stages of training, before the adapter has learned to generate effective 3D cues, zero initialization ensures that the enhanced features are completely equivalent to the original frozen 2D features, thus protecting the integrity of the pre-trained semantic prior and preventing irreversible initial damage to the feature space caused by randomly initialized adapters. As training progresses, it gradually learns suitable non-zero values, adaptively adjusting the injection ratio of cross-modal cues.

[0047] Through the aforementioned structure-to-semantic cues mechanism, the features of each 2D view no longer merely reflect local depth and contour information, but are "cued" with the complete 3D topological context of the object. This means that even information-poor projection views (such as feature-symmetric bases) can gain the ability to perceive the overall structure of the object through 3D cues, fundamentally alleviating semantic ambiguity.

[0048] Step 4: Construct a modal projection matrix, perform similarity calculation based on enhanced visual features and global 3D topological features to obtain dynamic aggregation weights for each viewpoint, and perform weighted fusion of enhanced visual features for each viewpoint based on the dynamic aggregation weights to obtain an optimized visual representation; Specifically, this disclosure sets up a structure-to-view routing module to implement this process. The core function of the structure-to-view routing module is to replace the traditional static average pooling and realize topology-aware dynamic multi-view fusion, fundamentally solving the semantic noise dilution problem.

[0049] The specific structure and working principle of the structure-to-view routing module are as follows: Step 41: Cross-modal projection alignment. To achieve effective interaction between 3D queries and 2D semantic information, this disclosure sets three sets of learnable modal projection matrices, which respectively map global 3D topological features to query vectors in the attention mechanism, and map enhanced visual features from each viewpoint to key vectors and value vectors:

[0050] in, The geometric query vector is the projected representation of the 3D features. and The first v Key vectors and value vectors for each viewpoint. Projection matrix. , and All dimensions are , For feature dimension, For the dimension of attention head.

[0051] This disclosure uses global 3D topological features as queries rather than keys or values, embodying a profound design philosophy: the 3D structure prior acts as the "questioner," actively retrieving feature information from each 2D viewpoint that best matches its topological structure. This positioning transforms viewpoint fusion from passive equal-weighted averaging to proactive topology-aware retrieval.

[0052] Step 42: Cross-modal attention weight calculation. Using a scaled dot product attention mechanism, the similarity between the 3D geometric query and the key vectors of each 2D viewpoint is calculated, serving as the dynamic aggregation weight for each viewpoint.

[0053] in, Scaling factor for the dimension of the attention head This is used to prevent the Softmax function from entering the saturation region due to excessively large dot product values. The Softmax function ensures that the sum of the weights for all views is 1.

[0054] The calculation of the aforementioned attention weights has a clear geometric and physical meaning: the dot product essentially measures the "topological consistency" between the 3D topological expectation and the 2D visual reality. A viewpoint that closely matches the 3D structure (such as a projection that clearly shows the geometric relationships of the object's components) has a high similarity to the 3D query in the mapping space, and therefore receives a large attention weight. Conversely, a highly occluded or feature-poor viewpoint (such as the projection of the object's bottom surface) has a significant domain shift between its features and the 3D query in the latent space, resulting in low similarity and naturally receiving an attention score close to zero. This mechanism theoretically achieves adaptive filtering of semantic noise.

[0055] Step 43: Weighted Feature Aggregation. Based on the dynamically calculated routing weights described above, the enhanced visual features from multiple perspectives are weighted and fused to obtain the final optimized visual representation:

[0056] The fused features are strictly dominated by the most topologically discriminative perspective, and the contribution of low-quality noise perspectives is effectively suppressed, thus providing a high signal-to-noise ratio feature foundation for subsequent incremental classification.

[0057] Step 5: Given the image category text prompt, extract the text semantic embedding features of the text prompt, and perform similarity matching between the optimized visual representation and the text semantic embedding features to obtain the classification prediction result.

[0058] The text encoder described in step 2 is used to extract features from the text prompts for a given image category. The text encoder is the same pre-trained visual-language model. Its input is a text prompt template constructed from the category name (such as "an image, projection, or sketch of [category]"), and its output is the text semantic embedding vectors for each category. The optimized visual representation features are matched with the text embeddings of each category output by the text encoder using cosine similarity to complete the classification prediction.

[0059] Unlike traditional multi-head self-attention, the routing module disclosed herein employs a cross-modal attention mechanism: queries originate from the three-dimensional modality, while keys and values ​​originate from the two-dimensional modality. This asymmetric design ensures that the consistency criterion for viewpoint selection is always determined by the pure three-dimensional topological prior, which is not degraded by two-dimensional projection, thus avoiding the circular argument dilemma of "evaluating a two-dimensional viewpoint with a two-dimensional viewpoint".

[0060] As one embodiment, the training process and optimization strategy of the network architecture disclosed herein are as follows: The training process disclosed herein optimizes only the learnable parameters in the two cross-modal interaction modules mentioned above, while the three backbone networks remain frozen throughout. The training objective consists of the following two parts: Main loss function: Contrastive learning loss. The InfoNCE contrastive loss is used as the main supervision signal for classification, and it is defined as follows:

[0061] in, Represents cosine similarity. This is the temperature hyperparameter (usually taken as 0.1). For optimized visual features, For text embedding corresponding to the real category, This is the set of all categories seen up to the current session. The goal of this loss function is to bring visual features closer to the correct category text embedding while distancing them from other category text embeddings. This represents the text embedding vector corresponding to category C. c This indicates a category index.

[0062] Auxiliary regularization term: Attention entropy regularization. In scenarios with extremely few samples, dynamic routing mechanisms may experience premature attention collapse, where a single viewpoint is assigned a weight close to 1.0, while other viewpoints are completely ignored. To mitigate this problem, an attention entropy regularization term is introduced:

[0063] in, V Indicates the number of multiple views. v This represents the view index, with values ​​ranging from 1, 2, ... V ; Indicates the first v The attention weights for each view are calculated by the cross-modal attention in the structure-view routing module, satisfying the following conditions: =1, and a v ≥0.

[0064] This regularization term encourages the model to maintain a more balanced attention distribution when the differences in viewpoint quality are not significant by penalizing low-entropy distributions (i.e., excessive attention concentration), thereby enhancing the robustness of training and the model's ability to generalize to different viewpoint combinations.

[0065] Total loss function: The final optimization objective is a weighted combination of the two losses mentioned above:

[0066] in, To balance the hyperparameters, it is used to adjust the relative strength of the regularization term.

[0067] For optimizer configuration, the Adam optimizer was used, with a learning rate of 0.001, weight decay of 0.0001, and a batch size of 16. The basic training phase iterated for 10 epochs, and each incremental session trained for 20 epochs. Due to the extremely small number of trainable parameters and the frozen backbone network, the training time for incremental sessions was significantly shorter than traditional fine-tuning methods.

[0068] As one embodiment, the application process of the incremental learning method disclosed herein is divided into two stages, as follows: (1) Basic Session Phase: Train all learnable lightweight parameters using the base class dataset. At this point, the learnable parameters (including the cue adapter parameters, projection matrix, and scaling factor) have achieved initial optimization on the base class data. The 3D encoder, 2D visual encoder, and text encoder remain frozen.

[0069] (2) Incremental Session Phase: When new category data arrives, training continues only on the aforementioned lightweight parameters. Since the feature extraction backbone network is frozen, the visual and textual feature representations of the old categories remain unchanged, and the memory of old knowledge is fully preserved. Learning of new categories is achieved through further updates to the adapter parameters. After training is complete, only the updated lightweight module parameters need to be saved; there is no need to store old samples or expand the model capacity.

[0070] The parameter freezing strategy disclosed herein has a dual significance: Firstly, for visual and textual backbones, the freeze operation fundamentally prevents catastrophic forgetting because the feature extraction logic of the old categories remains completely unchanged. Secondly, for the 3D encoder, the freeze operation ensures the consistency of the 3D topology prior extraction, providing a stable geometric reference for cross-modal cues and routing. This design achieves a physical decoupling of "stability" and "flexibility": stability is guaranteed by the frozen backbone, while flexibility is achieved by lightweight adaptation modules.

[0071] The structure-to-semantic prompting module and the structure-to-view routing module form a synergistic effect. The prompting module injects 3D spatial awareness into each 2D view, improving the overall feature quality of each view and reducing the quality differences between views. The routing module further filters high-discrimination views, achieving refined noise filtering. The two modules work together to ensure a high signal-to-noise ratio for multi-view fusion from both "enhancement" and "filtering" dimensions. Ablation experiments verify that using either the prompting module or the routing module alone can improve performance, but the performance gain from using both modules together is significantly higher than the sum of their individual gains, demonstrating their synergistic effect.

[0072] Example 2 One embodiment of this disclosure provides a 3D point cloud incremental learning system based on structure guidance and dynamic routing, including: The data acquisition module is used to acquire raw 3D point cloud data and perform multi-view projection rendering to obtain a multi-view 2D depth map. The feature extraction module is used to input multi-view 2D depth maps into a 2D visual encoder and output 2D visual features; and to input the original 3D point cloud into a 3D point cloud encoder and output global 3D topological features. The feature enhancement module is used to perform 3D feature projection adaptation on the global 3D topological features to obtain 3D semantic cue vectors. The 3D semantic cue vectors are then injected into the 2D visual features in the form of residuals on a per-view basis to obtain enhanced visual features. The feature fusion module is used to construct a modal projection matrix, perform similarity calculation based on enhanced visual features and global 3D topological features to obtain dynamic aggregation weights for each viewpoint, and perform weighted fusion of enhanced visual features from each viewpoint based on the dynamic aggregation weights to obtain an optimized visual representation. The point cloud classification module is used to extract the text semantic embedding features of a given image category text prompt, and then perform similarity matching between the optimized visual representation and the text semantic embedding features to obtain the classification prediction result.

[0073] Example 3 One embodiment of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned three-dimensional point cloud incremental learning method based on structure guidance and dynamic routing.

[0074] Example 4 One embodiment of this disclosure provides a non-transitory computer-readable storage medium for storing computer instructions. When these computer instructions are executed by a processor, they implement the aforementioned three-dimensional point cloud incremental learning method based on structure guidance and dynamic routing.

[0075] Example 5 One embodiment of this disclosure provides an electronic device, including a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the 3D point cloud incremental learning method based on structure guidance and dynamic routing.

[0076] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0077] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0078] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A 3D point cloud incremental learning method based on structure guidance and dynamic routing, characterized in that, include: Obtain the raw 3D point cloud data and perform multi-view projection rendering to obtain a multi-view 2D depth map; The multi-view 2D depth map is input into the 2D visual encoder, and the 2D visual features are output. The original 3D point cloud is input into the 3D point cloud encoder, and the output is the global 3D topological features. The global 3D topological features are adapted by 3D feature projection to obtain 3D semantic cue vectors. The 3D semantic cue vectors are then injected into the 2D visual features in the form of residuals from one viewpoint to obtain enhanced visual features. A modal projection matrix is ​​constructed, and similarity is calculated based on enhanced visual features and global 3D topological features to obtain dynamic aggregation weights for each viewpoint. The enhanced visual features of each viewpoint are then weighted and fused based on the dynamic aggregation weights to obtain an optimized visual representation. Given image category text prompts, extract the text semantic embedding features of the text prompts, and perform similarity matching between the optimized visual representation and the text semantic embedding features to obtain the classification prediction results.

2. The 3D point cloud incremental learning method based on structure guidance and dynamic routing as described in claim 1, characterized in that, The process of acquiring the original 3D point cloud data and performing multi-view projection rendering to obtain a multi-view 2D depth map includes: By uniformly arranging virtual cameras in different positions around the object, multiple multi-view 2D depth maps are generated from the original 3D point cloud; where the value of each pixel in the 2D depth map is the distance mapping from the object surface point corresponding to that pixel to the camera plane.

3. The 3D point cloud incremental learning method based on structure guidance and dynamic routing as described in claim 1, characterized in that, The process involves inputting a multi-view 2D depth map into a 2D visual encoder and outputting 2D visual features. The original 3D point cloud is input into a 3D point cloud encoder, and the output is the global 3D topological features, including: The rendered multi-view 2D depth maps are input into the frozen 2D visual encoder to obtain independent 2D visual features for each viewpoint. v Each perspective, its characteristics are represented as follows: in, Represents a two-dimensional visual encoder. This represents the depth map of the i-th viewpoint obtained from the rendering. Indicates the total number of viewpoints; Input the original 3D point cloud into the frozen 3D point cloud encoder to obtain global 3D topological features: in, This represents a 3D point cloud encoder. This represents the original 3D point cloud input.

4. The 3D point cloud incremental learning method based on structure guidance and dynamic routing as described in claim 1, characterized in that, The step of performing 3D feature projection adaptation on global 3D topological features to obtain a 3D semantic cue vector includes: A lightweight cue adapter is designed for 3D feature projection adaptation. The lightweight cue adapter adopts a bottleneck structure multilayer perceptron, specifically including a linear dimensionality reduction layer, a nonlinear activation function, and a linear dimensionality increase layer. The output of the adapter is a 3D semantic cue vector with the same dimension as the 2D visual features. This 3D semantic cue vector is not globally shared, but dynamically generated for each input point cloud instance.

5. The 3D point cloud incremental learning method based on structure guidance and dynamic routing as described in claim 1, characterized in that, The step of injecting the three-dimensional semantic cue vector into the two-dimensional visual features in residual form on a per-view basis to obtain enhanced visual features includes: The 3D semantic cue vector generated by the adapter is injected into the 2D visual features in residual form, view-by-view. For the ... v The enhanced visual features obtained from each perspective are: in, It is a learnable scalar scaling factor used to adjust the injection intensity of 3D prior information; For lightweight prompt adapters, It is a two-dimensional visual feature; It represents global three-dimensional topological features.

6. The 3D point cloud incremental learning method based on structure guidance and dynamic routing as described in claim 1, characterized in that, The construction of the modal projection matrix involves similarity calculation based on enhanced visual features and global 3D topological features to obtain dynamic aggregation weights for each viewpoint, including: Three sets of learnable modal projection matrices are set up to map the global 3D topological features to query vectors in the attention mechanism, and to map the enhanced visual features of each viewpoint to key vectors and value vectors. Through the scaling dot product attention mechanism, the similarity between the 3D query vector and the key vectors of each 2D viewpoint is calculated as the dynamic aggregation weight of each viewpoint.

7. A 3D point cloud incremental learning system based on structure guidance and dynamic routing, characterized in that, include: The data acquisition module is used to acquire raw 3D point cloud data and perform multi-view projection rendering to obtain a multi-view 2D depth map. The feature extraction module is used to input multi-view 2D depth maps into a 2D visual encoder and output 2D visual features. The original 3D point cloud is input into the 3D point cloud encoder, and the output is the global 3D topological features. The feature enhancement module is used to perform 3D feature projection adaptation on the global 3D topological features to obtain 3D semantic cue vectors. The 3D semantic cue vectors are then injected into the 2D visual features in the form of residuals on a per-view basis to obtain enhanced visual features. The feature fusion module is used to construct a modal projection matrix, perform similarity calculation based on enhanced visual features and global 3D topological features to obtain dynamic aggregation weights for each viewpoint, and perform weighted fusion of enhanced visual features from each viewpoint based on the dynamic aggregation weights to obtain an optimized visual representation. The point cloud classification module is used to extract the text semantic embedding features of a given image category text prompt, and then perform similarity matching between the optimized visual representation and the text semantic embedding features to obtain the classification prediction result.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the three-dimensional point cloud incremental learning method based on structure guidance and dynamic routing as described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the three-dimensional point cloud incremental learning method based on structure guidance and dynamic routing as described in any one of claims 1-6.

10. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the three-dimensional point cloud incremental learning method based on structure guidance and dynamic routing as described in any one of claims 1-6.

Images