An incremental learning method for generating replays based on variational autoencoder

Abstract

The application discloses an incremental learning method for generating replay based on a variational autoencoder, and particularly relates to the technical field of artificial intelligence, and comprises the steps of adding an incremental data set, constructing an incremental generation model, and training and updating the model, wherein the incremental generation model comprises a generation model and a classification model, the update strategy of the classification model comprises a trivial generation replay strategy, a no-forgetting generation replay strategy and a no-storage generation replay strategy; an efficient and high-quality incremental generation method based on a VAE generation framework is used to generate historical data for replay of the classification model, so that the anti-forgetting capability of the incremental classification network is improved, and meanwhile, some existing incremental learning strategies at the model layer are combined, so that the recognition accuracy of the incremental classification is further improved from the aspects of data and model, and the problems of low training efficiency and difficult training of the replay method based on the GAN generation model are solved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to an incremental learning method for generating replays based on variational autoencoders. Background Technology

[0002] Among existing incremental learning methods, many classic approaches apply deep learning to solve computer vision problems. Methods based on replaying historical data currently appear to have the best performance, but directly replaying real historical samples introduces many problems, such as storage space constraints, poor privacy, and imbalance between old and new task samples. Deep Generative Replay (DGR) addresses this by referencing the parallelism between the human hippocampus and generative models. It proposes using a Generative Adversarial Network (GAN) to generate historical data for a classification model, while simultaneously using the classification model to label the generated data. This data serves as a substitute for real historical data and is mixed with real data for the new task to participate in model training. Dynamic Generative Memory (DGM) proposes using a binary mask applied to the generator. The mask is updated cumulatively with each task and introduces a model expansion mechanism: when the number of "idle" nodes in the generator decreases, nodes are added or the mask is reinitialized to enhance the expressive power of the generative model. Generative Feature Replay (GFR) proposes replaying data by generating features rather than generating data. This involves decoupling the classification model into a feature extractor and a classifier, introducing a generative artificial intelligence (GAN) to generate historical features. These features, along with those extracted by the feature extractor, serve as input to the classifier, thus mitigating the forgetting problem. Updates to the generative model itself are also based on alignment with the outputs of historical generative models. A common characteristic of these methods is that they are all replay methods based on GAN generative models, and they still suffer from low training efficiency and difficulty in training. To address these issues, a technical solution is proposed. Summary of the Invention

[0003] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide an incremental learning method based on variational autoencoder (VAE) for generating replay data. This method utilizes an efficient and high-quality incremental generation method based on the VAE generation framework to generate historical replay data for the classification model, thereby improving the anti-forgetting ability of the incremental classification network. Simultaneously, it combines some existing incremental learning strategies at the model layer, starting from both the data and model levels, to further improve the recognition accuracy of incremental classification, thus solving the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] An incremental learning method based on variational autoencoder for generative replay includes adding an incremental dataset, constructing an incremental generative model, and training and updating the model.

[0006] Incremental generative model: The incremental generative model includes a generative model and a classification model. The generative model is M = (Φ, Θ), where Φ is the encoder, Θ is the decoder, and M is the generative model. The generative model trained on the t-th task (t = 1, 2, ..., T) is M. t =(Φ t ,Θ t Model M t =(Φ t ,Θ t Only in the training set After training on all tasks is completed, the target model M = M0 is obtained. T =(Φ T ,Θ T ), where Φ and Θ correspond to parameters φ and θ respectively, and the encoder Φ fits the distribution q. φ The distribution fitted by the decoder Θ is p θ The classification model is M θ Where θ are the parameters of the model, and the classification model trained on the t-th task is... After all tasks are trained, the target model is obtained.

[0007] Model training and updates: Incremental conditional models are used to generate historical data for the classification model when training on new tasks, and the model is trained together with the new task data. The classification model is a general neural network. The output of the input data is represented by o. t Identify, model Prediction of input data using ot -1 The classification model is labeled, and its update strategies include trivial generation and replay strategy, forget-free generation and replay strategy, and storage-free generation and replay strategy.

[0008] As a further aspect of this invention, the generative model (Φ,Θ) utilizes a conditional variational autoencoder trained using an image incremental generation method based on knowledge reconstruction and feedback reinforcement. The generative model is updated during training through knowledge reconstruction and feedback reinforcement. The data generated by the current generative model is used... Identifiers, data generated by historical models This indicates that the data generated by the current generative model will not be used in the training of the current classification model.

[0009] As a further aspect of this invention, the incremental dataset is added as follows: Let the training dataset for the entire incremental learning task be... It contains a total of T tasks, of which This is the training dataset corresponding to the t-th task, with a total of N. t There are n samples, containing a set of categories {Y}. t},satisfy It is the i-th sample image and its corresponding category label, y t i ∈{Y t}, Y :t For the categories contained in the first to the tth tasks, (x :t ,y :t The dataset for task t is the data sample-label pairs contained in task 1 through task t. The generated datasets for the first t tasks are In particular, This is the test dataset corresponding to the t-th task, with a total of N′. t During the training phase, T task sequences arrive for each sample.

[0010] As a further aspect of this invention, the trivial generation and replay strategy uses generated data to replace real historical data. The new data is directly combined with the generated historical data to train the model. The historical classification model is not saved; only the historical decoder is saved. The training process focuses only on the classification accuracy of the examples. Cross-entropy loss is used to constrain the network's discrimination of the input. The cross-entropy loss function formula for the trivial generation and replay strategy is as follows:

[0011]

[0012] in, Let the cross-entropy loss function be the trivial generation replay strategy. N is the sum of the number of generated samples and the number of real samples.

[0013] As a further aspect of the present invention, the forget-free generation and replay strategy saves the historical model during training, the historical model including the decoder Θ. t-1 and classification models When training the t-th task, t>0, given the label y of the historical category. ∶t-1 Let z be the noise randomly sampled from a Gaussian distribution, and let y be... ∶t-1 Enter z together into the reserved Θ t-1 In this process, the generator produces historical data, which is simultaneously input into the saved database. and the current model In this process, we obtain the outputs of the old and new models regarding the historical generated data. and Then, knowledge transfer is performed. The knowledge transfer loss function for the forgetting-free generation and replay strategy is:

[0014]

[0015] in, The knowledge transfer loss function is used to generate a replay strategy without forgetting.

[0016] As a further aspect of the present invention, the overall loss function during training of the forgetting-free generation and replay strategy is:

[0017]

[0018] In the formula: Let λ be the overall loss function during training of the forget-free generation and replay strategy, and let λ be the weight hyperparameter of the knowledge transfer loss term of the forget-free generation and replay strategy.

[0019] As a further aspect of this invention, the storageless generation and replay strategy performs knowledge transfer by aligning the outputs of the old and new models with respect to data from the new task. Simultaneously, it increases the alignment of the input attention map, performing this on the new task data and replacing this new task data with generated historical data. The knowledge transfer loss function of the storageless generation and replay strategy is:

[0020]

[0021] In the formula, For a storageless replay strategy, the knowledge transfer loss function is provided. For input image x in model M t The predicted Grad-CAM heatmap vector is for category c, where l is... The length.

[0022] As a further aspect of the present invention, the overall loss function during training of the storageless generation and replay strategy is a weighted sum of the cross-entropy loss term of the trivial generation and replay strategy, the knowledge transfer loss term during training of the forgetting-free generation and replay strategy, and the knowledge transfer loss term of the storageless generation and replay strategy. The formula for the overall loss function during training of the storageless generation and replay strategy is as follows:

[0023]

[0024] In the formula: Let β be the overall loss function during training of the storageless generation and replay strategy, and let β be the weight hyperparameter of the knowledge transfer loss term of the storageless generation and replay strategy.

[0025] As a further aspect of the present invention, the general neural network used in the classification model includes, but is not limited to, ResNet18.

[0026] The technical effects and advantages of this invention's incremental learning method based on variational autoencoder-based replay generation are as follows: It can adaptively generate historical data according to the amount of data for the new task during the incremental learning process using an incremental conditional variational autoencoder, thereby assisting in the training of the target model and improving its anti-forgetting ability; it can also use the generated data to assist in the training of existing incremental learning methods, further improving the performance of existing methods; specifically, it trains an incremental conditional variational autoencoder and proposes three replay strategies to train the target model: a trivial replay method that directly replays the generated data, a forgetting-free replay method using additional incremental learning strategies, and a storage-free replay method, to solve the problem of class increment without sample replay. Attached Figure Description

[0027] Figure 1 This is a flowchart of an incremental learning method for generating replays based on a variational autoencoder, according to the present invention. Detailed Implementation

[0028] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described technical solutions are only a part of the present invention, and not all of it. Based on the content of this invention, all other technical solutions obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0029] This invention discloses an incremental learning method for generative replay based on variational autoencoders, addressing the problem of category incremental learning. During model inference, it directly predicts the category of the input image without providing manually specified task identifiers. The task objective is to improve the performance of the target model M. θ In dataset I = I 1 ∪I 2 ∪...∪I T The recognition accuracy achieved in this area, compared to incremental task learning, is equivalent to simultaneously predicting the task to which a sample belongs and the specific category within that task, even for more challenging tasks.

[0030] Figure 1 A flowchart of the incremental learning method for generative replay based on variational autoencoders of this invention is provided, which includes adding incremental datasets, constructing incremental generative models, and training and updating the models.

[0031] Incremental generative model: The incremental generative model includes a generative model and a classification model. The generative model is M = (Φ, Θ), where Φ is the encoder, Θ is the decoder, and M is the generative model. The generative model trained on the t-th task (t = 1, 2, ..., T) is M. t =(Φ t ,Θ t Model Mt =(Φ t ,Θ t Only in the training set After training on all tasks is completed, the target model M = M0 is obtained. T =(Φ T ,Θ T ), where Φ and Θ correspond to parameters φ and θ respectively, and the encoder Φ fits the distribution q. φ The distribution fitted by the decoder Θ is p θ The classification model is M θ Where θ are the parameters of the model, and the classification model trained on the t-th task is... After all tasks are trained, the target model is obtained.

[0032] Model training and updates: Incremental conditional models are used to generate historical data for the classification model when training on new tasks, and the model is trained together with the new task data. The classification model is a general neural network. The output of the input data is represented by o. t Identify, model Prediction of input data using o t-1 The classification model is labeled, and its update strategies include trivial generation and replay strategy, forget-free generation and replay strategy, and storage-free generation and replay strategy.

[0033] Specifically, the generative model (Φ,Θ) utilizes a conditional variational autoencoder trained using an image incremental generation method based on knowledge reconstruction and feedback reinforcement. The generative model is updated during training through knowledge reconstruction and feedback reinforcement. The data generated by the current generative model is used... Identifiers, data generated by historical models This indicates that the data generated by the current generative model will not be used in the training of the current classification model.

[0034] It should be noted that the incremental image generation method based on knowledge reconstruction and feedback consolidation includes defining the entire incremental learning task, adding knowledge reconstruction loss terms and feedback consolidation loss terms to the loss function, and incorporating knowledge reconstruction loss and feedback consolidation loss into the training loss function of the VAE. The overall loss function consists of the regular training loss, knowledge reconstruction loss, and feedback consolidation loss, and the formula for the overall loss function is as follows:

[0035]

[0036] In the formula: For the overall loss function, This is the standard training loss term for VAE. For knowledge reconstruction loss term, To consolidate the loss item, Λ R and Λ F These are the weight hyperparameters for the knowledge reconstruction loss term and the weight hyperparameters for the feedback consolidation loss term, respectively.

[0037] The standard training loss term for a VAE is defined as the sum of the intrinsic reconstruction loss term and the variational loss term. The formula is:

[0038]

[0039] In the formula: For the intrinsic reconstruction loss term, This is the variational loss term;

[0040] In the formula for the overall loss function, the weight hyperparameters of the knowledge reconstruction loss term and the feedback consolidation loss term satisfy: Λ R >0, Λ F >0;

[0041] The definition process of the knowledge reconstruction loss term is as follows:

[0042] After training on the (T-1)th task is completed, save the decoder Θ. T-1 And freeze its parameters, preserving all historical knowledge learned from Task 1 to Task T-1 in the decoder Θ. T-1 middle;

[0043] When training the T-th task, Θ T With Θ T-1 Initialization was performed, but historical data D could not be accessed. :T-1 Gaussian noise Z is sampled from a standard Gaussian distribution, and the Gaussian noise Z is compared with the historical task category labels Y from task 1 to task T-1. :T-1 The data is input into the history decoder Θ respectively. T-1 and the current decoder Θ T In the middle, decoder Θ T-1 Output generated samples related to historical tasks. Decoder Θ T Output reconstructed samples of historical data By alignment and To motivate Θ T The reconstruction of historical knowledge ensures its consistency with Θt-1;

[0044] The formula for defining the knowledge reconstruction loss term is:

[0045]

[0046] In the formula: It is a discrete uniform distribution. These are sample labels uniformly sampled from the set of labels for all previously learned historical tasks.

[0047] The definition of feedback consolidation loss is as follows: During training for each task, for the data (x) of the t-th task... t ,y t encoder Φ t Encode it to obtain the latent variable z t Decoder Θ t For the latent variable z t Decoding yields reconstructed samples Reconstructed sample Re-enter Φ t Obtain the encoding of the reconstructed sample In the VAE's regular loss term In the process, the latent variable z will be obtained. t The KL divergence of the Gaussian distribution interval is used to guide the encoder, mapping the distribution of the real data to a Gaussian distribution.

[0048] The formula for defining feedback consolidation loss is:

[0049]

[0050] It needs further elaboration that the generative model uses an image incremental generation method based on knowledge reconstruction and feedback reinforcement for updating. It utilizes a conditional variational autoencoder based on this method as the generative model, which is updated during training through knowledge reconstruction and feedback reinforcement. The data generated by the current model uses... Identifiers, data generated by historical models This indicates that the data generated by the current generative model is not used in the training of the current classification model. The classification model is a general neural network such as ResNet18. The output of the input data is represented by o. t Identify, model Prediction of input data using o t-1 The model can be updated using only replay data or by combining some model-level incremental learning strategies. The training of the generative model and the classification model is relatively independent, each performing its own function. The generative model efficiently provides historical data to the classification model at the data level, while the classification model can use other incremental learning strategies at the model level to further enhance the incremental recognition performance of the target classification model.

[0051] It should be noted that the incremental dataset is added as follows: Let the training dataset for the entire incremental learning task be... It contains a total of T tasks, of which This is the training dataset corresponding to the t-th task, with a total of N. t There are n samples, containing a set of categories {Y}. t},satisfy It is the i-th sample image and its corresponding category label, y t i ∈{Y t}, Y :t For the categories contained in the first to the tth tasks, (x :t ,y :t The dataset for task t is the data sample-label pairs contained in task 1 through task t. The generated datasets for the first t tasks are In particular, This is the test dataset corresponding to the t-th task, with a total of N′. t During the training phase, T task sequences arrive for each sample.

[0052] Furthermore, the trivial generation and replay strategy uses generated data to replace real historical data. The new data is directly combined with the generated historical data to train the model. It does not save the historical classification model, only the historical decoder. The training process focuses only on the classification accuracy of the examples, using cross-entropy loss to constrain the network's discrimination of the inputs. The cross-entropy loss function formula for the trivial generation and replay strategy is:

[0053]

[0054] in, Let the cross-entropy loss function be the trivial generation replay strategy. N is the sum of the number of generated samples and the number of real samples.

[0055] Specifically, the forget-free generative replay strategy preserves a historical model during training, which includes the decoder Θ. t-1 and classification models When training the t-th task, t>0, given the label y of the historical category. ∶t-1 Let z be the noise randomly sampled from a Gaussian distribution, and let y be... ∶t-1 Enter z together into the reserved Θ t-1 In this process, the generator produces historical data, which is simultaneously input into the saved database. and the current model In this process, we obtain the outputs of the old and new models regarding the historical generated data. and Then, knowledge transfer is performed. The knowledge transfer loss function for the forgetting-free generation and replay strategy is:

[0056]

[0057] in, The knowledge transfer loss function is used to generate a replay strategy without forgetting.

[0058] Specifically, the overall loss function during training of the forget-free generation and replay strategy is:

[0059]

[0060] In the formula: Let λ be the overall loss function during training of the forget-free generation and replay strategy, and let λ be the weight hyperparameter of the knowledge transfer loss term of the forget-free generation and replay strategy.

[0061] Specifically, the storageless generative replay strategy transfers knowledge by aligning the outputs of the old and new models with respect to data from the new task. It also increases the alignment of the input attention map, performing this on the new task data and replacing it with generated historical data. The knowledge transfer loss function of the storageless generative replay strategy is:

[0062]

[0063] In the formula, For a storageless replay strategy, the knowledge transfer loss function is provided. For input image x in model M t The predicted Grad-CAM heatmap vector is for category c, where l is... The length.

[0064] Specifically, the overall loss function for training the storageless generation and replay strategy is a weighted sum of the cross-entropy loss term of the trivial generation and replay strategy, the knowledge transfer loss term of the forgetting-free generation and replay strategy, and the knowledge transfer loss term of the storageless generation and replay strategy. The formula for the overall loss function for training the storageless generation and replay strategy is as follows:

[0065]

[0066] In the formula: Let β be the overall loss function during training of the storageless generation and replay strategy, and let β be the weight hyperparameter of the knowledge transfer loss term of the storageless generation and replay strategy.

[0067] It should be noted that the general neural networks used in the classification models include, but are not limited to, ResNet18.

[0068] In incremental classification tasks constructed on the MNIST and SVHN datasets, the method proposed in this invention can effectively resist model forgetting and improve the performance of existing incremental learning methods.

[0069] As a classic replay method based on GAN generative models, Deep Generative Replay (DGR) references the parallelism between the human brain's hippocampus and generative models. It proposes using a Generative Adversarial Model (GAN) to generate historical data for a classification model, while simultaneously using the classification model to label the generated data. This data serves as a substitute for historical real data and is mixed with real data for the new task to participate in the model's training on the new task. Dynamic Generative Memory (DGM) addresses the issue that DGR, besides its low training efficiency, is prone to severe "semantic drift." This means that the quality of images generated during each memory replay largely depends on images generated in previous replays, leading to significant quality loss over time. Therefore, it proposes using a binary mask applied to the generator, which is updated cumulatively with the task. Simultaneously, a model expansion mechanism is introduced: when the number of "idle" nodes in the generator decreases, nodes are added or the mask is reinitialized to enhance the expressive power of the generative model. Generative Feature Replay (GFR) argues that generative models based on generative replay are difficult to train, highly dependent on initialization, and require a lot of data to train. Therefore, it proposes to replay by generating features rather than generating data. That is, the classification model is decoupled into a feature extractor and a classifier, and a generative model (GAN) is introduced to generate historical features. These features, together with the features extracted by the feature extractor, are used as input to the classifier, thereby mitigating the forgetting problem.

[0070] While GAN-based replay methods have been continuously improved and perfected, they still suffer from low training efficiency and difficulty in training. The updates to the generative model itself are based on aligning with the output of the historical generative model. The amount of data replayed by the generative replay method can be adjusted according to the number of new tasks, avoiding class imbalance. As long as the quality of the generated samples is good and the training is simple and efficient, the network can achieve lifelong learning. Moreover, the incremental network obtained in this way not only has recognition capabilities but also generation capabilities, which is closer to human learning. Humans can not only recognize and judge but also create and visualize the knowledge they have learned.

[0071] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0072] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An incremental learning method for generative replay based on variational autoencoders, comprising adding an incremental dataset, constructing an incremental generative model, and training and updating the model, characterized in that, Incremental generative models: Incremental generative models include generative models and classification models. The generative model is... ,in For encoder, Let M be the decoder and M be the generative model, in the... The generative model trained above is ,Model Only in the training set After training on all tasks is completed, the target model is obtained. , and The corresponding parameters are respectively and encoder The fitted distribution is decoder The fitted distribution is The classification model is as follows, where These are the parameters of the model, the first... The classification model trained on each task is The target model is obtained after all tasks have been trained. ; Model training and updates: Incremental conditional models are used to generate historical data for the classification model when training on new tasks, and the model is trained together with the new task data. The classification model is a general neural network. Output of input data Identify, model Prediction of input data The update strategies for the classification model include trivial generation and replay strategy, forget-free generation and replay strategy, and storage-free generation and replay strategy. The incremental dataset is added as follows: Let the training dataset for the entire incremental learning task be denoted as . It contains a total of One task, of which It is the first The training datasets for each task total [number] The sample contains a set of categories. ,satisfy , , It is the first Each sample image and its corresponding category label , For the first task to the... The categories contained in each task, For the first task to the... The data sample-label pairs contained in the task, the first The generated dataset for each task is ,forward The generated dataset for each task is , , It is the first The test datasets for each task total [number] One sample, during the training phase The tasks arrive sequentially; The storageless generative replay strategy transfers knowledge by aligning the outputs of the old and new models with respect to data from the new task. It also increases the alignment of the input attention maps, performing this on the new task data and replacing it with generated historical data. The knowledge transfer loss function for the storageless generative replay strategy is: ； In the formula, For a storageless replay strategy, the knowledge transfer loss function is provided. The model predicts the category of the input image. , Input image In the model , The above prediction category is Grad-CAM heatmap vectors for The length.

2. The incremental learning method for generating replays based on a variational autoencoder according to claim 1, characterized in that, The generative model M=(Φ,Θ) utilizes a conditional variational autoencoder trained using an image incremental generation method based on knowledge reconstruction and feedback reinforcement. During training, the generative model is updated through knowledge reconstruction and feedback reinforcement. The data generated by the current generative model is... Identifiers, data generated by historical models This indicates that the data generated by the current generative model will not be used in the training of the current classification model.

3. The incremental learning method for generating replays based on a variational autoencoder according to claim 1, characterized in that, The trivial generation and replay strategy uses generated data to replace real historical data. The new data is directly combined with the generated historical data to train the model. It does not save the historical classification model, only the historical decoder. The training process focuses only on the classification accuracy of the examples, using cross-entropy loss to train the model to correctly predict the class of the input sample. The cross-entropy loss function formula for the trivial generation and replay strategy is: ； in, Let be the cross-entropy loss function for the trivial generation replay strategy, where , ; This is the sum of the number of generated samples and the number of real samples.

4. The incremental learning method for generating replays based on a variational autoencoder according to claim 3, characterized in that, The forget-free generative replay strategy saves the historical model during training, which includes the decoder. and classification models During training , For each task, the label for a given history category is... The noise randomly sampled from the Gaussian distribution is ,Will and Enter them together into the reserved folder In Historical data is generated in the middle, and this data is simultaneously input into the saved... and the current model In this process, we obtain the outputs of the old and new models regarding the historical generated data. and Then, knowledge transfer is performed through the following steps. The knowledge transfer loss function of the forgetting-free generation and replay strategy is: ； in, , For historical classification models, For the current classification model, To generate a knowledge transfer loss function for a forget-free replay strategy, It is a classification model.

5. The incremental learning method for generating replays based on a variational autoencoder according to claim 4, characterized in that, The overall loss function during training of the forget-free generation and replay strategy is: In the formula: The overall loss function during training of the forgetting-free replay policy. The weight hyperparameters are used to generate the knowledge transfer loss term for the forgetting-free replay strategy.

6. The incremental learning method for generating replays based on a variational autoencoder according to claim 5, characterized in that, The overall loss function for training a storageless generation and replay strategy is a weighted sum of the cross-entropy loss term of the trivial generation and replay strategy, the knowledge transfer loss term of the forgetting-free generation and replay strategy, and the knowledge transfer loss term of the storageless generation and replay strategy. The formula for the overall loss function for training a storageless generation and replay strategy is: ； In the formula: The overall loss function during training of the storageless generation replay policy. The weight hyperparameters are used for the knowledge transfer loss term of the storageless replay strategy.

7. The incremental learning method for generating replays based on a variational autoencoder according to claim 1, characterized in that, The general neural network used in the classification model is ResNet18.

Images