Dynamic expansion prompt generation-based continual learning method, system, device and storage medium

Abstract

This invention discloses a continuous learning method, system, device, and storage medium based on a dynamically expanded prompt generator. The method includes: dividing a dataset into data for different tasks according to continuous learning settings to obtain continuous learning data; initializing a prompt-based continuous learning network and training a model using the data from the first task; updating the parameters of the model trained on the first task data and expanding the generator's modules; keeping the old boosted generator parameters frozen and training the updated model using the new task data; repeating the model update and parameter training process, and mapping image features to pseudo-task identity numbers to assist the model in selecting corresponding parameters. The system includes a training module, a parameter update module, an optimization module, and a mapping module. This invention effectively reduces catastrophic forgetting and promotes seamless acquisition of new tasks, greatly improving the performance of continuous learning.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a continuous learning method, system, device, and storage medium based on a dynamic extended cue generator. Background Technology

[0002] Deep neural network models should be continuously updated and evolved, much like the human learning process. However, acquiring new knowledge often leads to the unintentional forgetting of previously learned tasks, a phenomenon known as catastrophic forgetting. To mitigate this problem, researchers have proposed the task setting of Continual Learning (CL) and have drawn on various strategies, such as regularization-based, architecture-based, and replay-based methods, to alleviate this issue.

[0003] Regularization-based methods, such as LwF, typically minimize forgetting by constraining changes in certain network parameters. Architecture-based methods, like DualNet, promote the learning of new knowledge by allocating additional parameters for new tasks. Replay-based methods, such as RainbowMemory, excel at preserving past knowledge by retaining and replaying a portion of old data. Furthermore, replay strategies are applicable to various continuous learning methods, significantly improving model performance and outperforming methods that rely solely on regularization or architecture. However, in real-world scenarios, replay-driven strategies require retaining previous data, which presents practical challenges.

[0004] Recently, prompt learning has been widely applied in various fields due to its excellent performance. Initially emerging in natural language processing, prompt learning has demonstrated compelling model adaptation capabilities. With the development of VisionTransformer (ViT), prompt learning has also been applied to computer vision. Existing techniques highlight the unique advantages of prompt learning in continuous learning. Notably, L2P and DualPrompt use prompt pools to match prompts, thereby reducing the forgetting of previous knowledge. DAP introduces a prompt generator, overcoming the limitations of a finite prompt pool and enabling the creation of instance-level prompts. These methods effectively encode knowledge from earlier tasks into the prompts, allowing pre-trained models to quickly adapt to new tasks.

[0005] Despite the progress made in cue-based learning, current methods, including those using cue pools or cue generators, require updating parameters shared across all tasks when acquiring new knowledge, which inevitably leads to the forgetting of previously learned knowledge. Summary of the Invention

[0006] Purpose of the invention: The purpose of this invention is to provide a continuous learning method, system, device, and storage medium based on a dynamic extended prompt generator.

[0007] Technical solution: The continuous learning method based on a dynamic extended prompt generator described in this invention is characterized by the following steps:

[0008] (1) Prepare a dataset according to the continuous learning paradigm, and divide the dataset into data for different tasks according to the continuous learning settings to obtain continuous learning data.

[0009] (2) Initialize the prompt-based continuous learning network and train the model using the data from the first task;

[0010] (3) Update the parameters of the model trained based on the data from the first task and expand the module of the prompt generator;

[0011] (4) Use the old booster generator parameters frozen and train the updated model with the new task data;

[0012] (5) Repeat the model update and parameter training process in step (4), train the model parameters, and map the image features to pseudo-task identity numbers to help the model select the corresponding parameters.

[0013] Furthermore, the datasets prepared in step (1) include eight types: S-CIFAR100, S-Pets, S-EuroSAT, S-RESISC45, S-CropDis, S-ISIC, S-ChestX, and Split ImageNet-R, with corresponding task division numbers of 10, 7, 5, 9, 7, 3, 2, and 10, respectively.

[0014] Furthermore, step (2) adopts the ViT model architecture.

[0015] Furthermore, the ViT model utilizes the Transformer architecture to handle vision tasks, including a patch embedding layer, a set of L Transformer layers, and a classifier.

[0016] Furthermore, the classifier classification process includes:

[0017] First, the dataset is divided into n patch tokens through an embedding layer, with each patch representing a d-dimensional vector;

[0018] Next, a trainable classification token [CLS] is attached to these patch tokens to create the complete input token set E0, making E l The input token for the l-th layer Transformer has the following structure:

[0019] E l =[{CLS;PATCH1,...,PATCH n}]∈R (n+1)×d

[0020] Token E l It will enter the l-th layer Transformer and generate E for use in the next layer. l+1 .

[0021] Furthermore, the Transformer layer includes a multi-head self-attention module, a feedforward network, layer normalization, and residual connections. Each Transformer layer l is assigned its own cue generator G. l DEG extends Gl by introducing new, task-specific generators, resulting in a set of generators:

[0022]

[0023] Where n is the total number of tasks, and to determine which generator to use for the current task, DEG calculates a pseudo-task identifier I derived from the category features. task .

[0024] Further, step (5) includes:

[0025] Using the pre-trained ViT model backbone as a feature extractor, the model processes the encoder input x and extracts class features from the output of the [CLS] token. These class features are projected into an n-dimensional space through a learnable projection matrix Π, from which the most likely task identifier is selected as the pseudo-task identifier. Let f0 represent the pre-trained encoder and Π represent the projection matrix. The pseudo-task identifier is calculated as follows:

[0026]

[0027] in,[.] i Let f0(x) represent the i-th element of the vector, and let [f0(x)]0 represent the feature vector corresponding to the [CLS] token.

[0028] The continuous learning system based on a dynamic extended prompt generator described in this invention includes:

[0029] The training module is used to initialize the prompt-based continuous learning network and train the model using the data from the first task.

[0030] The parameter update module is used to update the parameters of the model trained based on the data from the first task, and to expand the prompt generator module.

[0031] The optimization module is used to train the updated model using new task data while keeping the old booster generator parameters frozen.

[0032] The mapping module is used to repeat the model update and parameter training process, train model parameters, and map image features to pseudo-task identity numbers to help the model select the corresponding parameters.

[0033] Furthermore, when the processor executes the program, it implements a continuous learning method based on a dynamic extended hint generator.

[0034] Furthermore, the computer program is designed to implement a continuous learning method based on a dynamic extended hint generator at runtime.

[0035] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: By retaining previous knowledge and activating task-specific information through pseudo-task identifiers, the present invention effectively reduces catastrophic forgetting and promotes the seamless acquisition of new tasks, greatly improving the performance of continuous learning. Attached Figure Description

[0036] Figure 1 This is a framework diagram of the present invention;

[0037] Figure 2 This is a comparison diagram between the present invention and existing learning methods. Detailed Implementation

[0038] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0039] like Figure 1 As shown, the present invention includes the following steps:

[0040] (1) Prepare a dataset according to the continuous learning paradigm, and divide the dataset into data for different tasks according to the continuous learning settings to obtain continuous learning data.

[0041] (2) Initialize the prompt-based continuous learning network and train the model using the data from the first task;

[0042] (3) Update the parameters of the model trained based on the data from the first task and expand the module of the prompt generator;

[0043] (4) Use the old booster generator parameters frozen and train the updated model with the new task data;

[0044] (5) Repeat the model update and parameter training process in step (4), train the model parameters, and map the image features to pseudo-task identity numbers to help the model select the corresponding parameters.

[0045] The ViT (Vision Transformer) model was chosen for training the data. The ViT model utilizes the Transformer architecture to handle visual tasks, achieving significant results by dividing images into patch tokens. ViT typically consists of a patch embedding layer, a set of L Transformer layers, and a classifier. During classification, the input image x is first segmented into n patch tokens by the embedding layer, each patch representing a d-dimensional vector. Then, a trainable classification token [CLS] is appended to these patch tokens to create the complete input token set E0. Let E... l The input token for the l-th layer Transformer has the following structure:

[0046] E l =[{CLS;PATCH1,...,PATCH n}]∈R (n+1)×d

[0047] These tokens E l It will enter the l-th layer Transformer and generate E for use in the next layer. l+1 .

[0048] Each Transformer layer mainly consists of a Multi-Head Self-Attention (MHSA) module, a feedforward network (MLP), layer normalization (LN), and residual connections. Specifically, the input token E l Convert to E l+1 The process is as follows:

[0049]

[0050] Finally, the classification head, a single feedforward network, processes the [CLS] tokens from the last Transformer layer to generate predicted labels.

[0051]

[0052] in, This represents the predicted category probability distribution.

[0053] The essence of continuous learning lies in endowing a single model with the ability to perform multiple tasks over time, while maintaining its expertise in previously learned tasks. In incremental learning-like scenarios, the data for each task comes from different sources within the same domain, represented as D = {D1, ..., D2}. nTypically, each dataset is specific to its task. The dataset D for the t-th task... t It consists of input-label pairs {(x, y)}, each in its own space X t ×Y t Defined in [the standard]. Our goal is to train a model to handle incremental tasks. In this paper, the model includes a patch embedding layer f. p A set of Transformer layers f b and a classifier f θ , integrated into

[0054] Training for each task involves training based on dataset D. t The model f is tuned to enable it to handle new tasks while maintaining its performance on all previous tasks {D1, ..., D}. t-1 The ability in}. This process requires through D t The training set is used to update the model to incorporate knowledge of new tasks, while the model is evaluated across all available datasets D = {D1, ..., D2}. t The performance on the} is to ensure that its capabilities are sustained.

[0055] In prompt-based continuous learning methods, such as L2P and DualPrompt, prompts are selected from a predefined pool of prompts to tune a ViT-Base model pre-trained on ImageNet. A prompt is essentially a series of d-dimensional tokens of length p appended to the input tokens. Therefore, the modified input can be represented as:

[0056]

[0057] Here, s indexes the set of hints in the hint pool, while o represents the length of the hint sequence, typically between 1 and 4, as shown in L2P and DualPrompt.

[0058] DAP proposes a method to incorporate cue-based learning into continuous learning, without being limited to a finite set of cues. Unlike selecting cues from a predefined cue pool, cue-based learning... It is generated on the fly by an adaptive prompt generator G. This generator includes a series of operations, such as input transpose (T), LN, MLP, and linear transformation (LT), to process the input token E∈R. (n+1)×d And generate adaptive prompts P a As shown below:

[0059]

[0060] Here, φ represents the linear layer that predicts the affine transformation parameters, and e is the conditional input embedding of φ. Although DAP's instance-specific hints offer more flexibility, similar to L2P and DualPrompt, it still requires updating shared parameters for new tasks, which may lead to the forgetting of previously acquired knowledge.

[0061] like Figure 2 As shown, the core idea of ​​DEG is to address the problem of updating models for new tasks. Updating shared parameters often affects the model's performance on previous tasks. DEG mitigates this problem by dynamically expanding the cue generator. As the number of tasks increases, this approach ensures that new modules can integrate new knowledge without overwriting parameters that encapsulate previous knowledge.

[0062] Within the context of the DAP framework, each Transformer layer l is assigned its own hint generator G. l DEG extends this concept. Instead of reusing and updating G in different tasks... ; By generating hints, DEG extends G by introducing new, task-specific generators. l This results in a set of generators:

[0063]

[0064] Where n is the total number of tasks. To determine which generator to use for the current task, DEG calculates a pseudo-task identifier I derived from the categorical features. task Using a pre-trained ViT backbone as a feature extractor, the model processes the encoder input x and extracts categorical features from the output of the [CLS] token. These features are then projected into an n-dimensional space through a learnable projection matrix Π, from which the most probable task identifier is selected as the pseudo-task identifier. Formally, let f0 represent the pre-trained encoder, Π represent the projection matrix, and the pseudo-task identifier is calculated as follows:

[0065]

[0066] in,[.] i Let f0(x) represent the i-th element of the vector, and let [f0(x)]0 represent the feature vector corresponding to the [CLS] token.

[0067] Therefore, using pseudo-task identifiers, the generator is selected. Available for use by all layers l = 1, ..., L, thereby generating task-specific hints for layer l. The formula is as follows:

[0068]

[0069] By using This method generates task-specific cues, enabling the model to seamlessly adapt to new tasks while preserving the integrity of previously learned knowledge.

[0070] To ensure fair comparison, the experiments used a ViT-B / 16 model pre-trained on ImageNet, a standard choice in the advanced vision research community. The study covered multiple cross-domain datasets, each with unique characteristics and partitions. We compared our method with two classes of replay-free methods: regularization-based methods (EWC and LwF) and cue-based methods (L2P, DualPrompt(DP), and DAP), as detailed in Table 1. In Table 2, we evaluated the performance of methods using replay strategies at different buffer sizes, comparing buffered methods (ER, BiC, GDumb, DER++, and Co2L) with the aforementioned bufferless methods. To ensure fairness, all methods were initialized with the same ImageNet pre-trained ViT-B / 16 model. Furthermore, architecture-based methods (SupSup, DualNet, RPSNet, and DynaER) were also evaluated in Table 3. To demonstrate the relative performance of these methods, we also include FT-seq (simple sequential training) and Sup-FT (supervised fine-tuning on independently and identically distributed data for all tasks), the latter often considered the upper limit of performance for incremental learning models. It's important to note that our experiments showed varying results for the Sup-FT implementation across different research reports; therefore, we uniformly selected the best results from all papers to more accurately represent the performance upper limit. For example, the best results for SplitCIFAR-100 are from DAP, while the ImageNet-R results are from DualPrompt.

[0071] Table 1. Average accuracy of different methods on different datasets.

[0072]

[0073] Table 2 Forgetting metrics for different methods on different datasets

[0074]

[0075] To ensure the accuracy and reliability of the results, each experiment was performed three times, and the mean and standard deviation were reported. We adopted two widely accepted metrics for sustained learning: mean accuracy (Avg.Acc ↑) to measure overall performance, and forgetting (Forgetting ↓) to evaluate the model's ability to retain previously learned information. Given the challenges of migrating methods from different architectures to Transformer-based models, we introduced the Diff metric (Diff = Acc).Sup-FT -Acc Method (A lower Diff value indicates better performance), used to compare the performance gap with supervised fine-tuning performed on all task data, which is considered the upper limit of the model's performance.

[0076] Comparative experiments were conducted on eight different datasets to evaluate different methods from multiple perspectives. Tables 1, 2, and 3 focus on regularization-based methods, while Tables 3 and 4 compare repetition-based and architecture-based methods. DEG consistently delivers excellent results in these continuous learning evaluations.

[0077] As shown in Tables 1, 2, and 3, the performance of several incremental learning methods on different datasets was evaluated. Cue-based methods (L2P, DP, DAP, and DEG) consistently outperformed regularization-based methods. However, cue pooling methods (L2P, DP) exhibited significant performance degradation on datasets with low domain similarity (such as ISIC2018), due to the inherent limitations of the cue pool size.

[0078] Table 3 Performance Comparison of Methods with Different Memory Requirements

[0079]

[0080] Table 4 Comparison of parameter increases in dynamic expansion methods

[0081]

[0082] Conversely, regularization methods typically suffer from overall performance degradation due to algorithmic limitations. In contrast, suggestion generation methods demonstrate significant performance advantages by eliminating the constraints of a finite suggestion pool, enabling models to effectively adapt to a variety of downstream tasks. Notably, our DEG method sets new benchmarks on eight datasets and surpasses Sup-FT on five of them, marking a significant advancement in the field of continuous learning.

[0083] As shown in Table 2, DEG also outperforms competing methods in terms of forgetting rate, supporting its high average accuracy. By dynamically expanding specific network parts, DEG ensures that each task benefits from the optimal, customized cue generator. DEG is the first method to achieve zero forgetting on datasets with fewer tasks, effectively preserving prior knowledge.

[0084] Table 4 compares the performance of DEG and architecture-based methods on CIFAR-100, detailing their setups, accuracy, and number of additional parameters. Considering the differences in backbone networks used by these methods, we employed the Diff metric for a fair comparison. The results show that cue-based methods (such as DEG) outperform architecture-based methods in both experimental accuracy and the Diff metric, while retaining the advantage of no repetition. In particular, DEG achieves state-of-the-art performance despite a small increase in parameters, and this increase is considered acceptable given the resulting performance improvement.

[0085] The present invention also provides an apparatus comprising a memory and a processor, wherein: the memory is used to store a computer program capable of running on the processor; and the processor is used to execute, when running the computer program, the steps of the continuous learning method based on a dynamic extension boosting generator as described above.

[0086] The present invention also provides a storage medium storing a computer program, which, when executed by at least one processor, implements the steps of the continuous learning method based on a dynamic expansion boosting generator as described above.

Claims

1. A continuous learning method based on a dynamic extended prompt generator, characterized in that, Includes the following steps: (1) Prepare a dataset according to the continuous learning paradigm, and divide the dataset into data for different tasks according to the continuous learning settings to obtain continuous learning data; (2) Initialize the prompt-based continuous learning network and train the model using the data from the first task; (3) Update the parameters of the model trained based on the data from the first task and expand the module of the prompt generator; (4) Keep the old cue generator parameters frozen and train the updated model using the new task data; (5) Repeat the model update and parameter training process in step (4), train the model parameters, and map the image features to the pseudo-task identity number to help the model select the corresponding parameters; Step (2) adopts the ViT model architecture; The ViT model architecture utilizes the Transformer architecture to handle visual tasks, including a patch embedding layer, a set of L Transformer layers, and a classifier. The classifier classification process includes: First, the dataset is segmented through the embedding layer. n Each patch token represents one patch token. d dimensional vector; Next, a trainable classification token [ These are attached to the patch tokens to create the complete set of input tokens. ,make Indicates the first The input token for the Transformer layer has the following structure: ， Token Will enter the Layer Transformer generates the material for use by the next layer. ; The Transformer layer includes a multi-head self-attention module, a feedforward network, layer normalization, and residual connections. Each Transformer layer... Each of them has its own prompt generator. The dynamically expanded hint generator (DEG) extends the functionality by introducing new, task-specific generators. This resulted in a set of generators: ， in, n Given the total number of tasks, in order to determine which generator to use for the current task, DEG computes pseudo-task identifiers derived from categorical features. ; Step (5) includes: Using the pre-trained ViT model backbone as a feature extractor, the model processes the encoder input. x and from the category token [ Extract categorical features from the output of [the system], and these categorical features are processed by a learnable projection matrix. Projected to n In a 3D space, the most likely task identifier is selected as the pseudo-task identifier, and then... This represents the pre-trained encoder. The formula for calculating the pseudo-task identifier, representing the projection matrix, is as follows: ， in, The vector represents the first One element, Representation and classification tokens [ The corresponding feature vector.

2. The continuous learning method based on a dynamic extended prompt generator according to claim 1, characterized in that, The datasets prepared in step (1) include eight types: S-CIFAR100, S-Pets, S-EuroSAT, S-RESISC45, S-CropDis, S-ISIC, S-ChestX, and Split ImageNet-R, with corresponding task division numbers of 10, 7, 5, 9, 7, 3, 2, and 10, respectively.

3. A continuous learning system based on a dynamic extended prompt generator, implemented using the method described in any one of claims 1-2, characterized in that, include: The training module is used to initialize the prompt-based continuous learning network and train the model using the data from the first task. The parameter update module is used to update the parameters of the model trained based on the data from the first task, and to expand the prompt generator module. The optimization module is used to train the updated model using new task data while keeping the old booster generator parameters frozen. The mapping module is used to repeat the model update and parameter training process, train model parameters, and map image features to pseudo-task identity numbers to help the model select the corresponding parameters.

4. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the continuous learning method based on a dynamic extended prompt generator according to any one of claims 1 to 2.

5. A storage medium storing a computer program, characterized in that, The computer program is designed to implement the continuous learning method based on a dynamic expansion hint generator according to any one of claims 1 to 2 at runtime.

Images