Lifetime learning method for solving catastrophic forgetting problem

Abstract

The invention relates to a lifetime learning method for solving a catastrophic forgetting problem. The method comprises the steps of (1) obtaining feature information before and after updating of a generation model by means of a coding network of an old solver; (2) whitening the features before updating, and obtaining robust features through linear combination of new and old features for constructing a transformation matrix to whiten the new features so as to obtain orthogonal feature information before and after updating; (3) according to a style migration algorithm, obtaining orthogonal style information before and after updating of the generation model by utilizing the Gram matrix, and updating the generation model by fusing loss items of orthogonal style consistency; and (4) based on aknowledge distillation algorithm, training a solver to be paired with the generated pseudo data to represent an old task, and shuffling the old task with new data for updating the solver. According to the method, the problems that the generation model is forgotten along with increase of tasks in the generation model and classification information is lost due to independent training of a solver can be relieved, and therefore the catastrophic forgetting problem in lifetime learning existing in neural network and artificial intelligence (AI) system development is solved.

Description

technical field [0001] The invention relates to the technical field of artificial intelligence, in particular to a lifelong learning method for solving the problem of catastrophic forgetting. Background technique [0002] Computing systems operating in the real world often encounter continuous information flow, or scenarios where data can only be used temporarily due to storage constraints or privacy concerns. At this time, multiple tasks need to be learned and remembered from dynamic data distributions, and constantly updated models to adapt to new tasks. However, what machine learning builds is a static model that cannot adapt or expand its behavior over time. When dealing with new tasks, the entire model needs to be retrained. At this time, the updated model is no longer suitable for the processing of old tasks. Lifelong learning (lifelong learning) tries to simulate human behavior, so that for sequential tasks, the model can not only handle the current task well, but al...

AI Technical Summary

Problems solved by technology

[0009] The purpose of the present invention is to provide a lifelong learning method for solving the catastrophic forgetting problem, which can alleviate the forgetting of the generative model caused by the increase of tasks in the generative model and the loss of classification information due to the independent training of the solver. problem, so as to solve the problem of catastrophic forgetting in the lifelong learning of neural network and artificial intelligence (AI) system development

Images