Training method of continuous learning model and non-transient computer readable medium

Abstract

The invention provides a training method of a continuous learning model and a non-transitory computer readable medium for executing the method. The method comprises the following steps: when a current training process is a first task of continuous learning, training an encoder and a self-attention layer in an essence generation program according to original data of the first task, otherwise, freezing parameters of the encoder and the self-attention layer, and executing the essence generation program to convert the original data into data essence; and adding the data essence into the essence memory area, and repeatedly executing the training program until the continuous learning model converges. The training program comprises the steps of obtaining a training batch from the original data, updating the playback memory area according to the training batch, training the continuous learning model according to the playback memory area and the essence memory area, and updating the data essence in the essence memory area when the current training process is a first task.

Description

TECHNICAL FIELD

[0001] The present application relates to Artificial Intelligence (AI) and Machine Learning (ML), and in particular to a method for training a model using data essence. BACKGROUND

[0002] Catastrophic forgetting is a big concern for AI / ML models in real-world applications. Catastrophic forgetting refers to the phenomenon that a model gradually forgets previously learned data when trained on new data, which leads to a trade-off between performance on new and old data, and thus a decrease in classification accuracy.

[0003] Traditional methods to deal with performance degradation usually involve retaining a small number of important data and adding these retained important data to the training when training on new data. Although this method can reduce the decrease in performance to some extent, the effect is limited. Another method is to encode the old data and add the encoded old data to the new training. The limitation of this method is that the encoding model needs to be trained on the target data domain in advance. However, in the context of continuous learning, the future data domain cannot be known in advance, so it is difficult to prepare similar data in advance for training. SUMMARY

[0004] Therefore, the purpose of the present application is to further reduce the performance degradation of the model caused by the transition training between new and old data.

[0005] A training method of a continuous learning model according to an embodiment of the present application is executed by an operation device, comprising the following steps: initializing a replay memory area, an essence memory area and a continuous learning model. When the current training process is the first one of a plurality of tasks of continuous learning, training the encoder and the self-attention layer in the essence generation program according to the original data of one of the plurality of tasks, otherwise freezing the parameters of the encoder and the self-attention layer. Execute the essence generation program to convert the original data into data essence, and add the data essence to the essence memory area. Repeat the training program until the continuous learning model converges, the training program comprising: obtaining a training batch from the original data; updating the replay memory area according to the training batch, wherein the replay memory area before updating comprises a plurality of data of old training batches; training the continuous learning model according to the replay memory area and the essence memory area; and when the current training process is the first task of continuous learning, updating the data essence in the essence memory area.

[0006] Preferably, the essence generation procedure comprises: reducing dimensionality of the original data by the encoder to generate a first feature map, wherein the first feature map comprises a plurality of positions; generating a second feature map by the self-attention layer according to similarities between any two of the plurality of positions; generating a plurality of noises by a noise generation module according to dimensionality and size of the first feature map; and adding the plurality of noises to the second feature map to generate the data essence.

[0007] Preferably, updating the replay memory according to the training batch comprises: taking a candidate data from the data in the training batch; adding the candidate data to the replay memory; and deleting data least important to the continual learning model from the replay memory when the number of data in the replay memory exceeds an upper limit.

[0008] Preferably, training the continual learning model according to the replay memory and the essence memory comprises: initializing a first model and a second model according to an architecture of the continual learning model; training the first model according to the replay memory and the essence memory; training the second model according to the essence memory; and calculating a linear combination of the first model and the second model as the continual learning model.

[0009] A non-transitory computer readable medium according to an embodiment of the present application stores instructions for causing operations when executed by a computing device, the operations comprising: initializing a replay memory, an essence memory, and a continual learning model. When a current training process is a first one of a plurality of tasks of continual learning, training an encoder and a self-attention layer in an essence generation procedure according to original data of one of the tasks, otherwise freezing parameters of the encoder and the self-attention layer. Executing the essence generation procedure to convert the original data to a data essence, and adding the data essence to the essence memory. Repeating a training procedure until the continual learning model converges, the training procedure comprising: taking a training batch from the original data; updating the replay memory according to the training batch, wherein the replay memory before the updating comprises data of old training batches; training the continual learning model according to the replay memory and the essence memory; and when the current training process is the first one of the tasks of continual learning, updating the data essence in the essence memory.

[0010] Preferably, the essence generation procedure comprises: reducing dimensionality of the original data by the encoder to generate a first feature map, wherein the first feature map comprises a plurality of positions; generating a second feature map by the self-attention layer according to similarities between any two of the plurality of positions; generating a plurality of noises by a noise generation module according to dimensionality and size of the first feature map; and adding the plurality of noises to the second feature map to generate the data essence.

[0011] Preferably, wherein updating the replay memory according to the training batch comprises: taking a candidate data from the plurality of data of the training batch; adding the candidate data to the replay memory; and deleting the data least important to the continual learning model from the replay memory when the number of data in the replay memory exceeds an upper limit.

[0012] Preferably, wherein training the continual learning model according to the replay memory and the essence memory comprises: initializing a first model and a second model according to the architecture of the continual learning model; training the first model according to the replay memory and the essence memory; training the second model according to the essence memory; and calculating a linear combination of the first model and the second model as the continual learning model.

[0013] In summary, the present disclosure proposes a training method of a continual learning model and a non-transitory computer readable medium for executing the method. The method proposed by the present disclosure does not limit the type of encoder and can use a model publicly downloaded on the Internet as the encoder. In the first task of the continual learning model training phase, the method proposed by the present disclosure fine-tunes the parameters of the encoder and the self-attention layer, so that the continual learning model can meet the current task. The present disclosure refers to this method as the first session adaption.

[0014] The above description of the present disclosure and the following description of the embodiments are used to demonstrate and explain the spirit and principles of the present disclosure, and provide further explanation of the claims of the present disclosure.

[0015] The figures simply illustrate

[0016] Figure 1 is a flowchart of a training method of a continual learning model according to an embodiment of the present disclosure;

[0017] Figure 2 and Figure 3 are a schematic diagram and a flowchart of an essence generation procedure according to an embodiment of the present disclosure, respectively;

[0018] Figure 4 is a flowchart of a training procedure according to an embodiment of the present disclosure;

[0019] Figure 5 is a flowchart of a method of updating a replay memory according to an embodiment of the present disclosure;

[0020] Figure 6 is a flowchart of an experience fusion algorithm according to an embodiment of the present disclosure; and

[0021] Figure 7 and Figure 8 are a module diagram of the first model and the second model, respectively.

[0022] Symbol explanation:

[0023] T 1-T6, U 1-U4, W1-W4, W21-W25, W31-W34: Step 11: Encoder

[0024] 13: Self-attention layer

[0025] 15: Noise generation module

[0026] 17: Adder

[0027] 21: First feature generator

[0028] 23: Second feature generator

[0029] 25: Classifier

[0030] B: Data batch

[0031] First feature map

[0032] Second feature map

[0033] Noise

[0034] Data essence(s)

[0035] e: Data essence(s) (single)

[0036] r: Raw data (single)

[0037] First model

[0038] Second model

[0039] Continual learning model Embodiment

[0040] The detailed features and advantages of the present application are described in detail in the embodiments below, and the contents are sufficient to enable any person skilled in the art to understand the technical content of the present application and to implement it, and any person skilled in the art can easily understand the related purposes and advantages of the present application according to the contents disclosed in the specification, claims and drawings. The following examples are further detailed to illustrate the present application, but not to limit the scope of the present application in any way.

[0041] The present disclosure provides a method for training a continual learning model, which is suitable for being executed by a computing device. In an embodiment, the computing device can be at least one of a personal computer, a network server, a central processing unit (CPU), a graphic processing unit (GPU), a microcontroller (MCU), an application processor (AP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system-on-a-chip (SOC), a deep learning accelerator, or any electronic device having similar functions. The present disclosure does not limit the hardware type of the computing device.

[0042] Figure 1 is a flowchart of the method for training a continual learning model according to an embodiment of the present disclosure, which comprises steps T1-T6. Continual learning (CL) refers to continuously updating a model by sequentially performing multiple tasks in time order, Figure 1 The virtual codes corresponding to the method shown in Table 1. Please refer to Figure 1 and Table 1.

[0043] Table 1, virtual codes of the method for training a continual learning model.

[0044]

[0045] As shown in step T1 and line 01, the computing device initializes a replay memory area an essential memory area ε, and a continual learning model a replay memory area for storing training data, the essential memory area ε is used for storing data essence extracted from the training data. The memory area ε can be a physical or virtual storage space. The present disclosure does not limit the use of hardware or software to implement the memory area ε. In an embodiment, a network attached storage (NAS) can be used to implement the memory area ε.

[0046] As shown in step T2 and lines 02, 03, the operation device determines whether the current training process is the first task. If the determination is yes, step T3 is performed. If the determination is no, step T4 is performed. The basis for distinguishing tasks is the pre-set CL parameter, for example, one task can correspond to N data sets or N classifications in the same data set, which is not limited by the present application.

[0047] As shown in step T3 and line 04, the operation device trains the encoder and the self-attention layer (Self-Attention Layer, SAlayer) according to the original data of the first task. The original data is, for example, at least one of the CIFAR-10, CIFAR-100, Tiny ImageNet, and ImageNet data sets, which is not limited by the present application. The encoder is, for example, a deep learning model for image classification trained on a large-scale data set, such as EfficientNet, ResNet, VGG, Vision Transformer, etc. The self-attention layer is, for example, a self-attention module in the Self-Attention Generative Adversarial Networks (SAGAN).

[0048] As shown in step T4 and line 06, the operation device freezes the parameters of the encoder and the self-attention layer. After steps T3 or T4 are completed, step T5 is performed.

[0049] As shown in steps T5 and lines 07-08, the operation device performs the Essence Generation procedure EssenceGeneration() to convert the original data R i of the i-th task into the data essence E i of the i-th task, and adds the data essence E i to the essence memory ε. In the application scenario of continuous learning, it is usually impossible to obtain the original data used in the previous training task. To avoid the performance decline of the continuous learning model due to catastrophic forgetting in subsequent training tasks, the data essence is extracted from the original data of the current task and stored.

[0050] As shown in steps T6 and lines 09-15, the operation device repeatedly performs the training procedure until the continuous learning model converges.

[0051] The following is an example of the continuous learning modelFigure 2 and Figure 3 The way of generating the essence is explained. Figure 4 to Figure 8 The details of the training procedure are explained.

[0052] Figure 2 and Figure 3 are the flowchart and the schematic diagram of the essence generation procedure according to an embodiment of the present application, respectively.

[0053] In step U1, the encoder 11 reduces the dimension of the original data R to generate a first feature map In an embodiment, the first feature map is an image outputted by the convolutional layer in the encoder 11, which includes multiple positions. In order to reduce the computation cost of the computing device, the original data R i may be divided into multiple data batches B and then inputted to the encoder 11 one by one.

[0054] In step U2, the self-attention layer generates a second feature map For example, a 2x2 first feature map is shown in Table 2 below, which has four positions A, B, C, D. According to such a first feature map, an attention map of 4x4 can be generated, which has 16 values (A, A), (A, B), …, (D, D), as shown in Table 3, where (X, Y) represents the correlation between position X and position Y. After performing the softmax operation, the dot product operation, and the 1x1 convolution operation on the attention map, the second feature map

[0055] Table 2, an example of a first feature map.

[0056] A B C D

[0057] Table 3, an example of an attention map.

[0058] (A, A) (A, B) (A, C) (A, D) (B, A) (B, B) (B, C) (B, D) (C, A) (C, B) (C, C) (C, D) (D, A) (D, B) (D, C) (D, D)

[0059] In step U3, the noise generation module 15 generates multiple noises according to the dimension and the size of the first feature map In an embodiment, the noise generation module 15 generates multiple Laplacian noises where τ is the difference between the maximum value and the minimum value in the data batch B, |B| is the number of data in the data batch B, and λ is a parameter adjusted by the user, and the smaller the value of λ is, the stronger the added noise is. In another embodiment, the noise generation module 15 generates multiple Gaussian noises. The purpose of adding noise is to simulate the fuzzy effect of human memory.

[0060] In step U4, the arithmetic unit executes adder 17 to convert multiple noise signals. Add a second feature map To generate data essence

[0061] Figure 4 This is a flowchart of a training procedure according to an embodiment of the present invention, including steps W1 to W4.

[0062] In step W1 (corresponding to row 10 of Table 1), the computing device obtains training batch B from the raw data. Training batch B may contain, for example, N data / images, N categories of data / images, or N datasets of data / images; this invention is not limited thereto.

[0063] In step W2 (corresponding to row 11 of Table 1), the computing device updates the playback memory area according to training batch B. Replay memory area before update This includes multiple datasets from older training batches. Please refer to [link / reference] for details on step W2. Figure 5 .

[0064] In step W3 (corresponding to row 12 of Table 1), the processing unit uses the playback memory area... and the essential memory area ε training continuous learning model Please refer to the implementation details of step W3. Figure 6 In this invention, the algorithm corresponding to step W3 is called Experience Fusion.

[0065] In step W4 (corresponding to rows 13 to 15 of Table 1), when the current training process is the first task of continuous learning, the computing device updates the data essence in the essential memory region ε; otherwise, the computing device does not operate. After executing step W4, if the continuous learning model... If convergence has not yet occurred, return to step W1, and the computing device obtains the next training batch B to continue training the continuous learning model.

[0066] Figure 5 This is a flowchart of a method for updating the playback memory area according to an embodiment of the present invention, including steps W21 to W25. Table 4 shows the virtual code corresponding to this method. Please refer to it as well. Figure 5 And Table 4.

[0067] Table 4 updates the virtual code for the replay memory area.

[0068]

[0069]

[0070] In step W21 (corresponding to row 41 of Table 4), the computing device retrieves data from the training batch. Select one candidate data b from multiple data sets.

[0071] In step W22 (corresponding to row 42 of Table 4), the processing unit adds candidate data b to the playback memory area.

[0072] In step W23 (corresponding to row 43 of Table 4), the processing unit determines the playback memory area. The amount of data in Does it exceed the upper limit s? If yes, proceed to step W24. If no, proceed to step W25.

[0073] In step W24 (corresponding to row 44 of Table 4), the processing unit retrieves data from the playback memory area. Remove for continuous learning models The least important data. In one embodiment, in the continuous learning model Use training batches During training, the difference in loss before and after training is measured for each candidate data b. If the loss decreases, the importance score of candidate data b increases. Therefore, the least important data has the lowest importance score. Through this mechanism, the importance score can be determined from the training batches. The most important data is selected and stored in the playback memory area. middle.

[0074] In step W25, the computing device determines the training batch. Check if there is any more data. If yes, return to step W21; otherwise, proceed to step W3.

[0075] Figure 6 This is a flowchart of an empirical fusion algorithm according to an embodiment of the present invention, including steps W31 to W34. Table 5 is the virtual code of the empirical fusion algorithm. Please refer to it as well. Figure 6 And Table 5.

[0076] Table 5. Virtual code of the experience fusion algorithm.

[0077]

[0078]

[0079] In step W31 (corresponding to row 51 of Table 5), the computing device uses the continuous learning model. Architecture initialization first model and the second model Figure 7 andFigure 8 are the first model and the second model , respectively. As shown in FIGS. 1 and 2, the first model Figure 7 and the second model Figure 8 are both generated based on the architecture of the model , but are trained respectively according to different data. The continual learning model includes a first feature generator 21, a second feature generator 23, and a classifier 25. In an embodiment, the first feature generator 21 and the second feature generator 23 can be implemented by using the following examples: ResNet, VGG, MLP, Vision Transformer, but the present application is not limited thereto. In step W32 (corresponding to row 52 of Table 5), the operation device trains the first model according to the essence memory area ε and the replay memory area

[0080] As shown in FIG. 3, the first feature generator 21 generates the first feature according to the original data r, the second feature generator 23 generates the second feature according to the input, and the classifier 25 classifies according to the concatenation result of the first feature and the second feature, and finally outputs the classification result. It should be noted that there are two cases for the input of the second feature generator 23: if the current training task is the first task, the input of the second feature generator 23 is the second feature map Figure 7 generated by the encoder 11 and the attention layer 13 according to the original data r; if the current training task is the second or subsequent task, the input of the second feature generator 23 is the original data r itself. In an embodiment, the following loss function is used as the loss function of the first model

[0081] , where is the cross entropy loss function. In step W33 (corresponding to row 53 of Table 5), the operation device trains the second model

[0082] according to the essence memory area ε. Since the data essence e is different from the original data r, the model architecture needs to be modified so that it can recognize the data essence e, and restore the data essence e to the type of the original data r. The modified model architecture is shown in FIG. 4. Figure 8 ​​​As shown, the generative model 19 generates a data anchor e according to the data essence, the first feature generator 21 generates the first feature according to the data anchor, the second feature generator 23 generates the second feature according to the data essence, the classifier 25 classifies according to the concatenation result of the first feature and the second feature, and finally outputs the classification result. In an embodiment, the generative model 19 can be implemented by using the following examples: generative adversarial network (GAN), variational autoencoder (VAE), autoregressive model, or transformer-based model (such as Image GPT). In an embodiment, the loss function used by the classifier 25 is as follows: As the second model loss function.

[0083] In step W34 (corresponding to row 54 of Table 5), the operation device calculates the linear combination of the first model and the second model as the continuous learning model Since the first model and the second model have similar architectures, the parameters of the two models corresponding to each other can be multiplied by weights α and (1-α) respectively and then added as the parameters of the continuous learning model In an embodiment, α = 0.5.

[0084] Table 6, accuracy comparison of existing methods and the present application.

[0085]

[0086]

[0087] RM:

[0088] GDumb: A Simple Approach that Questions Our Progress in Continuous Learning

[0089] EWC++: Riemannian Walk for Incremental Learning: Understanding

[0090] Forgetting and Intrinsicality

[0091] ER-MIR: Online continual learning with maximally interfered

[0092] Retrieval

[0093] BIC: Large scale incremental learning

[0094] CLIB: Online continual learning on class incremental blur task configuration with anytime inference

[0095] iCaRL: Incremental classifier and representation learning.

[0096] FOSTER: Feature boosting and compression for class-incremental

[0097] learning.

[0098] In Table 6, joint training represents the process of continual learning that can take all the data at any time, so its value is the upper limit of the accuracy. Therefore, the closer the accuracy rate is to the value of joint training, the better the continual learning model can solve the problem of catastrophic forgetting. As shown in Table 6, the training method of the continual learning model proposed according to the present application is the best in the performance of each data set compared with various existing methods.

[0099] Table 7, comparison table of different encoders.

[0100]

[0101]

[0102] Table 7 presents the accuracy of the continual learning model on the target dataset when trained with different sets of data. As shown in Table 7, the average accuracy of the continual learning model is proportional to the size of the dataset used to train the encoder. In other words, if the pre-training dataset of the encoder is large enough, or if a foundation model is used as the encoder, the accuracy of the continual learning model is expected to be improved.

[0103] In summary, the present disclosure proposes a training method for a continual learning model and a non-transitory computer-readable medium for executing the method. The method proposed by the present disclosure does not limit the type of encoder and can use a publicly downloaded model on the Internet as the encoder. In the first task of the training phase of the continual learning model, the method proposed by the present disclosure fine-tunes the parameters of the encoder and the self-attention layer, so that the continual learning model can meet the current task. The present disclosure refers to this method as the first session adaption.

[0104] In addition, in order to reduce the performance degradation of the model caused by the conversion training of new and old data, the present disclosure uses data essence to train the continual learning model. The concept is to simulate the way humans store memories. The less important part of the memory will gradually become blurred, while the more important part will remain clear. Using this concept, the method proposed by the present disclosure converts old data into highly refined data essence and stores it. In the training of new tasks, the data essence is restored to the original data form by a generative model, and then the continual learning model is trained with the original data of the new task to help the model retain the knowledge of the data it has learned and reduce the performance degradation caused by model updating. This mechanism is similar to the way humans recall events from blurred memories through clear parts.

[0105] Although the present disclosure is disclosed with the above-mentioned embodiments, it is not intended to limit the present disclosure. Any changes and modifications made without departing from the spirit and scope of the present disclosure are within the scope of the present disclosure. For the scope of protection of the present disclosure, please refer to the appended claims.

Claims

1. A training method for a continuous learning model, characterized in that, Executed by a computing device, and including: Initialize a playback memory area, an essential memory area, and a continuous learning model; When the current training process is the first of multiple tasks in a continuous learning process, train the encoder and self-attention layer in an essential generation program based on the original data of one of the multiple tasks; otherwise, freeze the parameters of the encoder and the self-attention layer. Execute the essence generation procedure to convert the original data into a data essence, and add the data essence to the essence memory area; and Repeat a training procedure until the continuous learning model converges, the training procedure comprising: Obtain a training batch from the original data; The replay memory is updated based on the training batch, wherein the replay memory before the update includes multiple data from the old training batch; The continuous learning model is trained based on the playback memory region and the essential memory region; and When the current training process is the first of the multiple tasks in continuous learning, update the data essence in the essence memory area.

2. The training method for the continuous learning model as described in claim 1, characterized in that, The essence generation process mentioned above includes: The encoder is used to reduce the dimensionality of the original data to generate a first feature map, wherein the first feature map includes multiple locations; The self-attention layer generates a second feature map based on multiple similarities between any two of the multiple locations; A noise generation module generates multiple noises based on the dimensions and size of the first feature map; and The multiple noises are added to the second feature map to generate the data essence.

3. The training method for the continuous learning model as described in claim 1, characterized in that, The updating of the replay memory region based on the training batch includes: Select a candidate data from the multiple data in the training batch; Add the candidate data to the playback memory area; and When the amount of data in the playback memory exceeds the upper limit, the data that is least important to the continuous learning model is deleted from the playback memory.

4. The training method for the continuous learning model as described in claim 1, characterized in that, Training the continuous learning model based on the replay memory region and the essential memory region includes: Initialize the first model and the second model according to the architecture of the continuous learning model; The first model is trained based on the essential memory region and the playback memory region; The second model is trained based on the essential memory region; and The linear combination of the first model and the second model is calculated as the continuous learning model.

5. A non-transitory computer-readable medium, characterized in that, This is used to store multiple instructions, which, when executed by a computing device, cause multiple operations, including: Initialize a playback memory area, an essential memory area, and a continuous learning model; When the current training process is the first of multiple tasks in a continuous learning process, train the encoder and self-attention layer in an essential generation program based on the original data of one of the multiple tasks; otherwise, freeze the parameters of the encoder and the self-attention layer. Execute the essence generation procedure to convert the original data into a data essence, and add the data essence to the essence memory area; and Repeat a training procedure until the continuous learning model converges, the training procedure comprising: Obtain a training batch from the original data; The replay memory is updated based on the training batch, wherein the replay memory before the update includes multiple data from the old training batch; The continuous learning model is trained based on the playback memory region and the essential memory region; and When the current training process is the first of the multiple tasks in continuous learning, update the data essence in the essence memory area.

6. The non-transitory computer-readable medium as claimed in claim 5, characterized in that, The essence generation process mentioned above includes: The encoder is used to reduce the dimensionality of the original data to generate a first feature map, wherein the first feature map includes multiple locations; The self-attention layer generates a second feature map based on multiple similarities between any two of the multiple locations; A noise generation module generates multiple noises based on the dimensions and size of the first feature map; and The multiple noises are added to the second feature map to generate the data essence.

7. The non-transitory computer-readable medium as claimed in claim 5, characterized in that, The updating of the replay memory region based on the training batch includes: Select a candidate data from the multiple data in the training batch; Add the candidate data to the playback memory area; and When the amount of data in the playback memory exceeds the upper limit, the data that is least important to the continuous learning model is deleted from the playback memory.

8. The non-transitory computer-readable medium as claimed in claim 5, characterized in that, Training the continuous learning model based on the replay memory region and the essential memory region includes: Initialize the first model and the second model according to the architecture of the continuous learning model; The first model is trained based on the essential memory region and the playback memory region; The second model is trained based on the essential memory region; and The linear combination of the first model and the second model is calculated as the continuous learning model.

Images