Continual learning method and continual learning device based on self-paced mechanism

Abstract

The present invention relates to a self-paced continual learning device comprising: a training data construction unit which calculates a loss value for each of the samples of a new task, measures complexity of the samples on the basis of the calculated loss value, and constructs training data by performing self-paced sampling in order from an easy sample with relatively low complexity to a difficult sample with relatively high complexity; a forgetting occurrence identification unit which, during a training process of the new task, identifies whether forgetting has occurred for each of the samples of a pre-learned previous task, and stores a sample, in which forgetting has occurred, in a replay memory; and a training unit for continually training a previously provided model on the basis of the training data of the new task and the sample of the previous task stored in the replay memory.

Description

Continuous learning method and continuous learning device based on self-regulation mechanism

[0001] The present invention relates to a continuous learning method and a continuous learning device based on a self-paced mechanism.

[0002] Continuous learning algorithms are designed to solve non-independent and non-identically distributed (Non-IID) scenarios. Unlike conventional deep learning algorithms that assume data is IID, continuous learning approaches must consider that data distributions can change over time. In such cases, models trained to minimize empirical risk are vulnerable to catastrophic forgetting, where they lose previously acquired knowledge and adapt to new data.

[0003] Furthermore, artificial intelligence models must be regularly updated to respond to real-time changing data distributions, during which the models are retrained using all accumulated data. However, this retraining is unsuitable for real-time data processing due to the exponential increase in computational costs and training time, and places a significant burden on system resources as more memory is required to store and process continuously growing datasets.

[0004] Meanwhile, self-regulated learning is a method in which an artificial intelligence model adjusts the difficulty and order of the data it learns during the learning process. Similar to the human learning process, it starts with easy tasks and gradually learns more difficult tasks as learning progresses.

[0005] Thus, since self-regulated learning learns gradually starting from easy samples, the AI ​​model can proceed with learning quickly without feeling burdened by complex problems from the beginning, and can improve learning performance by gradually learning difficult samples.

[0006] Therefore, research is needed on ways to apply these principles of self-regulated learning to continuous learning problems.

[0007] [Prior Art Literature]

[0008] [Patent Literature]

[0009] Korean Patent Publication No. 10-2015-0055323

[0010] The present invention has been devised to solve the above-mentioned problems, and the objective of the present invention is to provide a method and a continuous learning device based on a self-regulating mechanism.

[0011] A self-regulating continuous learning device according to an embodiment of the present invention for achieving the above objective comprises: a learning data construction unit that calculates a loss value for each sample of a new task and, based on the calculated loss value, measures the complexity of the samples and constructs learning data by self-regulating sampling in order from easy samples with relatively low complexity to difficult samples with relatively high complexity; a forgetting occurrence confirmation unit that checks whether forgetting has occurred for each sample of a previously learned task during the learning process of the new task and stores the sample in which forgetting has occurred in a repetitive memory; and a learning unit that continuously trains a pre-prepared model based on the learning data of the new task and the sample of the previous task stored in the repetitive memory.

[0012] A self-regulated continuous learning method according to an embodiment of the present invention for achieving the above objective comprises: a step of constructing training data by self-regulating sampling in order from easy samples with relatively low complexity to difficult samples with relatively high complexity, by calculating a loss value for each sample of a new task and measuring the complexity of the samples based on the calculated loss value; a step of checking whether forgetting has occurred for each sample of a previously learned task during the training process of the new task and storing the sample in which forgetting has occurred in a repetitive memory; and a step of continuously training a pre-prepared model based on the training data of the new task and the sample of the previous task stored in the repetitive memory.

[0013] According to one aspect of the present invention described above, by providing a continuous learning method and a continuous learning device based on a self-regulating mechanism, sample complexity can be dynamically adjusted according to the current learning ability of the model, thereby mitigating destructive forgetting and improving the ability to acquire new knowledge.

[0014] Furthermore, it demonstrates excellent performance not only in disjoint task scenarios but also in more realistic and practical blurry tasks, and can be effectively utilized in various real-world applications, such as computer vision, natural language processing, and recommendation systems.

[0015] FIG. 1 is a diagram illustrating the internal blocks of a self-regulated continuous learning device according to an embodiment of the present invention,

[0016] FIG. 2 is a diagram illustrating the detailed operation of a self-regulated continuous learning device according to an embodiment of the present invention.

[0017] FIG. 3 is a diagram for explaining the operation of the forgetting occurrence verification unit of FIG. 1 verifying whether forgetting has occurred.

[0018] FIG. 4 is a diagram for explaining the operation of the forgetting occurrence confirmation unit of FIG. 1 updating the repetitive memory.

[0019] And, FIG. 5 is a flowchart showing the continuous learning operation of a self-regulating continuous learning device according to an embodiment of the present invention.

[0020] The following detailed description of the invention refers to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that various embodiments of the invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in relation to one embodiment. It should also be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the invention is limited only by the appended claims, including all equivalents to those claimed therein, provided appropriately described. Similar reference numerals in the drawings refer to the same or similar functions across various aspects.

[0021] The components according to the present invention are defined by functional distinction rather than physical distinction, and can be defined by the functions each performs. Each component may be implemented as hardware or as program code and processing units that perform each function, and the functions of two or more components may be included and implemented in a single component. Therefore, it should be noted that the names assigned to the components in the following embodiments are not intended to physically distinguish each component but are assigned to imply the representative function performed by each component, and that the technical concept of the present invention is not limited by the names of the components.

[0022] Preferred embodiments of the present invention will be described in more detail below with reference to the drawings.

[0023] FIG. 1 is a diagram illustrating the internal blocks of a self-regulated continuous learning device according to an embodiment of the present invention, FIG. 2 is a diagram for explaining the detailed operation of a self-regulated continuous learning device according to an embodiment of the present invention, FIG. 3 is a diagram for explaining the operation of the forgetting occurrence confirmation unit of FIG. 1 checking whether forgetting has occurred, and FIG. 4 is a diagram for explaining the operation of the forgetting occurrence confirmation unit of FIG. 1 updating a repetitive memory.

[0024] The illustrated self-regulating continuous learning device includes a learning data construction unit (110), a forgetting occurrence verification unit (120), and a learning unit (130).

[0025] The training data construction unit (110) calculates a loss value for each sample of a new task, measures the complexity of the samples based on the calculated loss value, and constructs training data by self-paced sampling in order from easy samples with relatively low complexity to difficult samples with relatively high complexity.

[0026] To explain in more detail with reference to FIG. 2, the training data construction unit (110) has loss values ​​for each of the samples of the new task, for example, x1, x2, x3, x4, and x5. The complexity of the samples is measured by comparing them with a predefined dynamic learning threshold λ. Then, the training data construction unit (110) determines that samples with a loss value smaller than the dynamic learning threshold are easy samples, and gradually increases the dynamic learning threshold to determine that samples are increasingly difficult, thereby constructing training data such that easy samples are included in epoch 1 and more difficult samples are included as the process progresses to epoch k+1.

[0027] At this time, the loss value for each sample of the above-mentioned new task can be calculated using the self-regulating loss function shown in Equation 1 below. The self-regulating continuous learning device proposed in the present invention uses the model parameter θ and the latent weight variable v of Equation 1. i = The model is trained by alternately optimizing, and at this time, the above v i is the current task dataset D t It indicates the importance of the samples.

[0028] [Mathematical Formula 1]

[0029]

[0030] Here, λ represents the dynamic learning threshold controlling the learning pace, and x i wa y iand represent the sample and class label, respectively, and f(θ) represents the neural network model. Additionally, Equation 1 consists of three terms in order: 1) weighted loss for the samples of the current task, 2) regularization term for the weights of the samples of the current task, and 3) unweighted loss for the samples of the replay memory.

[0031] Normalization term v i is the current task dataset D to prioritize samples stored in the iteration memory of the previous task. t It applies only to the samples, and Equation 1 represents the loss value of the samples for each new task with weight v i Indicates that it is discounted by.

[0032] In addition, parameters θ and v are interleaved and optimized at each epoch, and the global optimum v * = can be calculated by the following mathematical formula 2. That is, of the new task Only these single samples are considered as training data samples.

[0033] [Mathematical Formula 2]

[0034]

[0035] The forgetting occurrence verification unit (120) checks whether forgetting has occurred for each of the previously learned samples of the new task during the training process of the new task, and stores the samples that have been forgotten in the iteration memory. That is, the forgetting occurrence verification unit (120) checks whether forgetting has occurred for each of the samples of the previous task during the entire epoch of the training process of the new task, and determines the samples in which the model’s prediction accuracy switches from True to False as the samples that have been forgotten and stores them in the iteration memory.

[0036] As an example of the process of determining a sample that has been forgotten through FIG. 3, the forgetting occurrence confirmation unit (120) inputs a sample from a previous task, for example, an image of a 'dog', into the model to predict a class label, and determines a sample that has been forgotten by checking whether the model's prediction accuracy switches from '1' (true) to '0' (false) over the entire epoch. Here, the 'dog' image is determined to be a sample that has been forgotten because the model's prediction accuracy switches from '1' to '0' over the entire epoch.

[0037] The learning unit (130) continuously trains the model based on the learning data of a new task that is self-regulated and sampled through the learning data building unit (110) and the samples of a previous task stored in the repetitive memory through the forgetting occurrence verification unit (120).

[0038] Subsequently, the forgetting occurrence verification unit (120) inputs the learning data of the new task and the sample of the previous task stored in the iteration memory into the model in which continuous learning is completed through the learning unit (130) to predict the class label, and checks whether forgetting has occurred according to the prediction accuracy of the model and updates the iteration memory.

[0039] That is, as shown in FIG. 4, the forgetting occurrence confirmation unit (120) checks for samples in which the model’s prediction accuracy switches from True to False during the entire epoch, for example, epoch 1 to epoch 4, determines the samples in which forgetting has occurred, and stores the samples in which forgetting has occurred in the iterative memory and updates them.

[0040] In Figures 1 to 4 described above, the operation of the self-regulated continuous learning device proposed in the present invention is described in detail.

[0041] Hereinafter, the self-regulated continuous learning method proposed in the present invention will be described in detail through the algorithms of Tables 1 to 4. Algorithm 1 of Table 1 below is a task sequence D={D1,… ,D T This represents the self-regulated continuous learning method. Here, T represents the total number of tasks.

[0042] [Table 1]

[0043]

[0044] D in Table 1 t = is the dataset for task t, and N t Includes samples with specified class labels, and y i is sample x i Represents the class label of. Also, iteration memory M= It maintains and manages, where K represents the memory size.

[0045] More specifically, Algorithm 1 uses the current dataset D for each task t. t Merge with playback memory sample M to D u Create, (line 2) D u AutoAugment and CutMix data augmentation strategies for each sample i It is applied to all data samples with a probability of (lines 5-8). Here, AutoAugment refers to an automated data augmentation strategy that automatically applies various image transformation techniques according to a pre-set policy, and CutMix refers to a data augmentation strategy that induces the model to learn various partial patterns through the process of cutting and mixing two images.

[0046] D u To learn the model f(θ) and curriculum from, the self-regulation loss function of Equation 1 described earlier is used as described in Algorithm 2 (line 10). Additionally, using the predictions of the updated model, whether forgetting occurs in F[i] for each sample is calculated as described in Algorithm 3 (line 11).

[0047] At the end of each epoch, the dynamic learning threshold λ is adjusted by a scaling factor μ (line 13), and the higher the value of μ, the more accelerated the increase of λ becomes, allowing more samples to be included in the training dataset in earlier epochs.

[0048] Once training for the current task is complete, the replay memory M is updated according to the measured F as described in Algorithm 4. (line 15)

[0049] Algorithm 2 in Table 2 below shows a method for calculating the self-regulation loss function.

[0050] [Table 2]

[0051]

[0052] In Table 2, the model parameter θ is updated by minimizing the self-regulation loss function that distinguishes samples from the current task dataset Dt and samples from the iteration memory M.

[0053] More specifically, Algorithm 2 is D tFor each sample i belonging to, if the loss value is less than the dynamic learning threshold λ, the latent weight variable v i g is set to 1, so the total loss value is integrated into the optimization. (lines 1-3)

[0054] Conversely, if the loss value exceeds λ, v i α is set to 0 to effectively exclude the corresponding sample so that it does not contribute to the training process of the current epoch. (line 4)

[0055] For samples of iterative memory M, the loss value is the latent weight variable v i It is directly integrated into model updates without the application of. (lines 7-8)

[0056] Algorithm 3 in Table 3 below represents a method for determining the occurrence of forgetting.

[0057] [Table 3]

[0058]

[0059] More specifically, Algorithm 3 uses prev-acc for each sample i Initialize to false, then repeat the operations of lines 2-4 from Epoch 1 to Epoch E. (line 1)

[0060] Data set D u For each sample i of, sample x i Model's predicted labels for Extracts (line 3) the above The ground truth label corresponding to is y i am.

[0061] Then, a boolean variable acc that indicates whether sample i was correctly predicted. i =( =y i Calculate ) (line 4) and prev-acc as in line 5 i is true and acc iIf is false, set F[i], which indicates the occurrence of forgetting, to true. (line 6)

[0062] Finally, prev-acc i acc is the current prediction accuracy i Update to (line 8)

[0063] Algorithm 4 in Table 4 below shows a method for updating iterative memory.

[0064] [Table 4]

[0065]

[0066] More specifically, Algorithm 4 is D where F is set to false. u Samples of which no forgetting has occurred, i.e., samples that have not occurred, are excluded to select samples that have been forgotten (line 1), and Reservoir sampling is applied to the selected samples to update the iterative memory.

[0067] Reservoir sampling, which has proven effective for managing iterative memory in scenarios where task identity is not provided during inference time, is a method that samples uniformly from an input data stream while maintaining a fixed memory size K.

[0068] If the number of forgotten samples S is less than or equal to K, all received samples are stored in the repetitive memory until the memory capacity is reached (lines 3-4), and when the memory capacity is reached, a K / S sampling probability is assigned to each sample (lines 6-8).

[0069] FIG. 5 is a flowchart illustrating the continuous learning operation of a self-regulating continuous learning device according to an embodiment of the present invention.

[0070] The self-regulating continuous learning device constructs training data by self-regulating sampling samples of a new task. That is, it calculates a loss value for each sample of the new task, measures the complexity of the samples based on the calculated loss value, and constructs training data by self-regulating sampling in order from easy samples with relatively low complexity to difficult samples with relatively high complexity. (S501)

[0071] Then, the self-regulated continuous learning device stores samples from the previous task that have been forgotten in the repetitive memory. That is, during the learning process of the new task, it checks whether forgetting has occurred for each of the samples from the previous task and stores the samples that have been forgotten in the repetitive memory. (S503)

[0072] Subsequently, the self-regulating continuous learning device continuously trains the model based on the training data of the new task built through S501 and the samples of the previous task stored in the iteration memory through S503. (S505)

[0073] The self-regulated continuous learning method of the present invention can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination.

[0074] The program instructions recorded on the above-mentioned computer-readable recording medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software.

[0075] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, and flash memory.

[0076] Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0077] Although various embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the invention as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present invention.

[0078] [Explanation of the symbol]

[0079] 110: Training Data Construction Section

[0080] 120: Forgetfulness Occurrence Confirmation Unit

[0081] 130: Learning Department

Claims

1. A training data construction unit that calculates a loss value for each sample of a new task, measures the complexity of the samples based on the calculated loss value, and constructs training data by self-paced sampling in order from easy samples with relatively low complexity to difficult samples with relatively high complexity; A forgetting occurrence checking unit that, during the training process of the above-mentioned new task, checks whether forgetting has occurred for each of the previously trained samples of the previous task and stores the forgotten samples in a replay memory; and A self-regulating continuous learning device comprising: a learning unit that continuously trains a pre-prepared model based on the training data of the new task and samples of the previous task stored in the iteration memory.

2. In Paragraph 1, The above-mentioned learning data construction unit is, A self-regulating continuous learning device that measures the complexity by comparing the loss value for each sample of the new task with a predefined dynamic learning threshold, determines a sample with a loss value smaller than the dynamic learning threshold as an easy sample, and determines a difficult sample by increasing the dynamic learning threshold.

3. In Paragraph 1, The above-mentioned forgetfulness occurrence confirmation unit is, A self-regulating continuous learning device that checks for each of the samples of the previous task whether forgetting has occurred during the entire epoch of the learning process of the new task, and determines the sample in which the prediction accuracy of the model switches from True to False as the sample in which forgetting has occurred.

4. In Paragraph 3, The above-mentioned forgetfulness occurrence confirmation unit is, A self-regulating continuous learning device that predicts class labels by inputting the training data of the new task and the samples of the previous task stored in the iteration memory into the model in which the continuous learning is completed.

5. In Paragraph 4, The above-mentioned forgetfulness occurrence confirmation unit is, A self-regulating continuous learning device that updates the repetitive memory by checking whether forgetting occurs according to the prediction accuracy of the above model.

6. A step in which a training data construction unit calculates a loss value for each sample of a new task, measures the complexity of the samples based on the calculated loss value, and constructs training data by self-paced sampling in order from easy samples with relatively low complexity to difficult samples with relatively high complexity; A step in which a forgetting occurrence verification unit checks whether forgetting has occurred for each of the previously learned samples of the previous task during the training process of the new task, and stores the samples for which forgetting has occurred in a replay memory; and A self-regulated continuous learning method comprising: a step in which a learning unit continuously trains a pre-prepared model based on the training data of the new task and samples of the previous task stored in the iteration memory.

7. In Paragraph 6, The above-mentioned learning data construction unit is, A self-regulating continuous learning method that measures complexity by comparing the loss value for each sample of the new task with a predefined dynamic learning threshold, determines the sample with the loss value smaller than the dynamic learning threshold as the easy sample, and determines the difficult sample by increasing the dynamic learning threshold.

8. In Paragraph 6, The above-mentioned forgetfulness occurrence confirmation unit is, A self-regulated continuous learning method that checks whether forgetting occurs for each of the samples of the previous task during the entire epoch of the learning process of the new task, and determines the sample in which the prediction accuracy of the model switches from True to False as the sample in which forgetting has occurred.

9. In Paragraph 8, The above-mentioned forgetfulness occurrence confirmation unit is, A self-regulating continuous learning method for predicting class labels by inputting the training data of the new task and the samples of the previous task stored in the iteration memory into the model in which the continuous learning is completed.

10. In Paragraph 9, The above-mentioned forgetfulness occurrence confirmation unit is, A self-regulating continuous learning method that updates the iterative memory by checking whether forgetting occurs based on the prediction accuracy of the above model.

Images