A continuous learning method based on Bayesian variational inference
Through a Bayesian variational inference method, combined with Gaussian mixed distribution to generate pseudo-sample images, the catastrophic forgetting problem of deep neural networks in dynamic environments is solved, the stability and plasticity balance between tasks is achieved, and the learning efficiency and model adaptability is improved.
Patent Information
- Application Number
- CN202211050124.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-28
- Publication Date
- 2025-08-22
- Estimated Expiration
- 2042-11-28
AI Technical Summary
Existing continuous learning methods are difficult to achieve the trade-off between task stability and plasticity in dynamic environments, resulting in the inability to effectively alleviate the problem of catastrophic forgetting, especially in deep neural networks, especially those experiencing backpropagation.
Using Bayesian variational inference method, by obtaining the pseudo-sample images of the current task and the previous task, using Bayesian variational inference method for feature extraction and parameter estimation, combining Gaussian mixed distribution to generate pseudo-sample images, optimize the model classifier, and achieve uncertainty estimation and balance between tasks.
It effectively alleviates the problem of catastrophic forgetting, realizes the trade-off between task stability and plasticity on complex data sets, improves the learning efficiency and computing efficiency of the model, and reduces the consumption of network training resources.
Smart Images
Figure CN115731396B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision, and in particular to a continuous learning method based on Bayesian variational inference in a dynamic learning environment. Background Art
[0002] Deep neural networks are widely used in image processing and have developed to a level of recognition close to that of humans in image classification, object detection, face recognition, and semantic segmentation. Although deep learning has made great progress in these areas, some problems still exist. The ability to acquire continuous experience in dynamic and open environments is severely challenged. Any learning system should adapt to a changing environment. In most practical application scenarios, the training data is a dynamic data stream, that is, a single network cross-trains multiple tasks. When training a new task, the information collected previously is forgotten, resulting in a sharp drop in the performance of the earlier training tasks. This phenomenon is called catastrophic forgetting. Especially for neural networks that have undergone backpropagation, catastrophic forgetting is widely considered to be a key obstacle to achieving artificial general intelligence with neural networks. Recently, continuous learning (CL), also known as lifelong learning or incremental learning, has received widespread attention and can enable deep neural networks to continuously preserve and expand knowledge.
[0003] Continuous learning is proposed on the basis of deep learning in order to quickly learn new knowledge and effectively alleviate the problem of catastrophic forgetting in dynamically distributed data streams. Its main task is to obtain a model that can adapt to changes in the input data distribution (plasticity) while retaining past knowledge (stability). Currently, continuous learning mainly includes regularization, network structure expansion, parameter isolation, memory replay and other methods.
[0004] Among the above-mentioned continuous learning methods, the regularization method is to impose constraints on the weight updates in the neural network and penalize the changes of important weights to reduce catastrophic forgetting. Usually, an additional regularization term is introduced in the loss function to limit the model's ability to adapt to new knowledge; network structure expansion is to achieve non-forgetting by expanding the network structure, but the subsequent network structure is large and requires a large memory occupancy rate; parameter isolation mainly adopts fixed task parameters to reduce forgetting, which easily causes data redundancy and large amount of calculation; memory replay methods mainly include experience replay and generation sample replay. The experience replay method stores old samples, requires a large amount of memory and damages the plasticity of the model. Generative replay transfers the catastrophic forgetting of continuous learning to the generative network; however, these methods cannot achieve a trade-off between task stability and plasticity, and effectively prevent catastrophic forgetting.
[0005] In response to the limitations of the above continuous learning methods, some research has begun to avoid catastrophic forgetting from the perspective of Bayesian variational inference. For example, the well-known VCL and DGR methods retain the distribution of model parameters through Bayesian inference and use the posterior of the previous task as the prior of the current task. However, these inference methods often cannot maintain the uncertainty estimation of the parameters. Compared with other continuous learning methods, Bayesian variational inference-based continuous learning still needs to solve the problem of how to provide better uncertainty estimation on a single network model. Summary of the Invention
[0006] To address some of the aforementioned issues with current continuous learning in computer vision, an embodiment of the present invention provides a continuous learning method based on Bayesian variational inference. This method can provide better uncertainty estimation for the model, effectively alleviate catastrophic forgetting of old tasks during continuous learning training, and achieve a trade-off between task stability and plasticity. The technical solutions of this application are as follows:
[0007] According to an embodiment of the present application, a continuous learning method based on Bayesian variational inference is provided, including:
[0008] S1: Obtain a task to be trained, where the task to be trained is a current task image and a pseudo sample image generated by a previous task;
[0009] S2: Input the task to be trained into the model for training, perform feature extraction through the encoder, and obtain the mean and covariance of the task;
[0010] S3: Based on the Bayesian variational inference method, we obtain the posterior distribution of the representation in the network, calculate the uncertainty of the parameter estimation, and the loss of the model variational inference;
[0011] S4: Through maximum likelihood estimation, the model classifier is trained to obtain the optimal classification result;
[0012] S5: Substitute the potential feature prior distribution obtained in S2 for a Gaussian mixture distribution, generate pseudo sample images through the decoder, and calculate the loss of the deep generative model;
[0013] Optionally, the encoder network performs feature extraction, Where z represents the feature vector of the data in the latent space.
[0014] Optionally, the loss function used in the Bayesian variational inference method is as follows:
[0015]
[0016] Among them, t, N t are the t-th task and the number of samples it contains; represents the nth sample and its true label; q t (ω),q t-1 (ω) represents the posterior distribution of the latent variable z in the hidden space of the t-th and t-1-th tasks; ω represents the parameters obtained in its network; in Bayesian variational inference, in order to obtain the parameter distribution of its task, the KL divergence (Kullback-Leibler divergence) is used for measurement, then KL(q t (ω)||q t-1 (ω)), in order to obtain the posterior distribution of the current task, the posterior distribution of the previous task is used as the prior distribution of the current model training.
[0017] is the maximum likelihood estimation function.
[0018] During the training process, it is necessary to minimize the Bayesian variational inference loss function
[0019] Optionally, the Gaussian mixture model (GMM) is a ubiquitous statistical model used for density estimation, pattern recognition, and function approximation. It has analytical tractability, asymptotic properties, and universal approximation capabilities for continuous density functions. When generating pseudo samples, a Gaussian mixture probability distribution is used instead of a priori distribution, so that each category has a separate model. The distribution of the latent variable z is more flexible, while increasing the plasticity of the model. By limiting the sampling of z to generate a specified category, the features of the original latent space are converted into:
[0020]
[0021]
[0022] in p Ω (z) are the posterior distribution and prior distribution of the deep generative model GMM, Ω is the set of mean and standard deviation of each Gaussian model, α k =y represents the input category, K is α k Total number of inputs, corresponding class prior coefficient ω k =ω' k =p(y=α k )=1 / k,
[0023] The KL divergence loss of the latent variables obtained based on the Gaussian mixture prior distribution is further:
[0024]
[0025] in and is the mean and standard deviation of the k-th target value.
[0026] Optionally, the decoder network p φ According to the posterior distribution of the latent variable z Reconstruct pseudo sample images Let the pseudo samples generated by the latent variable feature z To reduce the error of generating pseudo samples as similar as possible to the original task input data, the reconstruction loss function expression should be:
[0027]
[0028] in, φ are the distribution parameters of the encoder and decoder respectively
[0029] The loss function of the further deep generative model should be
[0030]
[0031] According to the technical solutions of the embodiments of the present application, compared with the prior art, the advantages and beneficial effects of the present invention are as follows:
[0032] (1) The continuous learning method based on Bayesian variational inference provided by the example of the present invention performs Bayesian variational inference on the features in the latent space. The posterior distribution of the current task can be obtained by simply multiplying the posterior distribution of the previous task by the probability of the data set belonging to the current task, thereby saving network model training resources; the parameter uncertainty is estimated to measure the prediction difference between the new and old network models for the old task, thereby evaluating the prediction uncertainty of the new model for the old knowledge, so that the network classification model of the new task can automatically adjust to learn new knowledge and retain old knowledge during the training phase; it is more in line with continuous learning in complex real-world scenarios and can effectively avoid the problem of catastrophic forgetting.
[0033] (2) The image sample prior distribution provided by the present invention obeys the Gaussian mixture distribution, and a separate model is provided for each class, so that the replay pseudo samples are generated according to the context memory reproduction control network, making the network training more flexible and improving its computational efficiency.
[0034] (3) The present invention will achieve better overall results on complex datasets such as CIFAR10, CIFAR100 and ImageNet100. The model achieves a trade-off between stability and plasticity in the task, effectively preventing catastrophic forgetting. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0036] Figure 1 A flowchart of a continuous learning method based on Bayesian variational inference provided by an embodiment of the present invention;
[0037] Figure 2 A schematic diagram of a model flow based on Bayesian variational inference according to an embodiment of the present invention;
[0038] Figure 3 A schematic diagram of the network structure of a continuous learning method based on Bayesian variational inference of the present invention; DETAILED DESCRIPTION
[0039] In order to enable ordinary persons in the art to better understand the purpose, technical solutions and advantages of the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.
[0040] like Figure 1 As shown, this embodiment provides a continuous learning method for Bayesian variational inference, and the specific steps are as follows:
[0041] S1: Obtain a task to be trained, where the task to be trained is a current task image and a pseudo sample image generated by a previous task;
[0042] Specifically, define the dataset Given a picture The corresponding label is Belong to class and the corresponding distribution probability is p i (x,y); Divide into N tasks, the task flow is For each task T i There are n classes in satisfy and Relationship between tasks i When training, the model cannot access the data from the previous T1,T2,...,T i-1 data, and cannot access future tasks T i+1 ,T i+2 ,...,T N Specific reference Figure 2 As shown, the pseudo sample dataset is defined as If the current task is T1, then its training data does not need to replay the pseudo sample image; if the current task is T i , then its training data is the current task image and tasks T1, T2...T i-1 Generate pseudo sample images; when the training data preparation is completed, the network model training is performed.
[0043] Furthermore, the present invention evaluates the incremental learning scenario in continuous learning. Incremental learning refers to a task flow and the corresponding input and output pairs sequentially reaching the incremental learning model. The training is performed in the ,and no task boundary information can be provided to the model during the ,evaluation phase; the goal of the class of incremental learning ,models is to train a classifier that can train and ,accurately classify the newly arrived task T i , while maintaining the performance of the previously learned classes, minimizing the target classification function:
[0044]
[0045] Where M represents all categories currently seen, L CE (f θ (x), y) is the classification cross entropy loss function between the predicted value and the true label:
[0046]
[0047] Where σ is the softmax activation function.
[0048] S2: Input the task to be trained into the model for training, perform feature extraction through the encoder, and obtain the mean and covariance of the task;
[0049] Specifically, such as Figure 3 As shown, for the task flow The input vector x is sequentially mapped to the encoder network As the feature extraction of the task, the encoder learns the distribution of category data from each task, obtains the mean μ and covariance σ of the task, and then obtains the probability distribution of the hidden latent variable z. The prior distribution of the latent variable z obeys the standard normal distribution p prior (z)=N(0,I), is the distribution parameter on the encoder.
[0050] S3: Based on the Bayesian variational inference method, we obtain the posterior distribution of the representation in the network, calculate the uncertainty of the parameter estimation, and the loss of the model variational inference;
[0051] To overcome the catastrophic forgetting problem in neural networks, Bayesian approaches to neural networks were found to potentially avoid some of the drawbacks of explicitly parameterized importances in conventional neural networks.
[0052] Bayesian variational inference techniques naturally account for uncertainty in parameter estimates, and network models represent each parameter using a distribution defined by the mean and covariance of possible values from a shared underlying probability distribution.
[0053] Furthermore, the predicted mean μ and covariance σ of the latent space distribution are used to describe the importance of each parameter, and the Bayesian neural network controls the learning rate of each parameter as a function of its uncertainty.
[0054] In Bayesian variational inference, the Kullback-Leibler divergence (KL divergence) is used to measure the posterior probability distribution of the task parameters. In the Bayesian variational inference model, the posterior distribution of the previous task is used as the prior distribution for the current model training to obtain the posterior distribution of the current task. The loss function used by the Bayesian variational inference method is as follows:
[0055]
[0056] S4: Through maximum likelihood estimation, the model classifier is trained to obtain the optimal classification result;
[0057] The maximum likelihood estimation is further introduced into the network classifier, and the posterior distribution q of the latent variable is derived according to S3. t (ω), get the sample The maximum probability appears. It not only considers the center distance of the sample to be classified to the known category, but also considers the feature distribution of the known category, making its classification result better. Its maximum likelihood estimation loss is:
[0058]
[0059] S5: Substitute the potential feature prior distribution obtained in S2 for a Gaussian mixture distribution, generate pseudo sample images through the decoder, and calculate the loss of the deep generative model;
[0060] Furthermore, in order to generate pseudo sample images of the replayed data, the prior distribution of the latent variable z in the hidden space obtained by S2 is made to obey the Gaussian mixture probability distribution, so that each category has a separate model, and the prior characteristics of its latent space are converted to Through the posterior distribution Approximate prior distribution p Ω (z), measured using the KL divergence (Kullback-Leibler divergence), the KL divergence loss of the latent variable obtained based on the Gaussian mixture prior distribution is:
[0061]
[0062] Furthermore, in the decoder network p φ The pseudo sample image is reconstructed in , and the decoder network is based on the posterior distribution just obtained by the latent variable z Reconstruct pseudo sample images to prevent the low quality of generated pseudo samples, which in turn affects the continuous learning network model training. Replaying pseudo sample images produces catastrophic forgetting. By letting the pseudo samples generated by the latent variable feature z To reduce the error of generating pseudo samples as similar as possible to the original task input data, the reconstruction loss function expression should be:
[0063]
[0064] The loss function of the further deep generative model in generating pseudo sample images should be:
[0065]
[0066] The present invention proposes a Bayesian variational inference method combined with a generative replay method. The Bayesian variational inference method is used to train a continuous learning network model and optimize the model weights, and a deep generative network model is used to generate pseudo sample images. This reduces the catastrophic forgetting problem caused by the imbalance of new and old data in the Bayesian network model during training, and achieves a trade-off between stability and plasticity between tasks.
Claims
1. A continuous learning method based on Bayesian variational inference, characterized in that: The following steps are involved: 1) Obtaining a task to be trained, wherein the task to be trained is a current task image and a pseudo sample image generated by a previous task; 2) Inputting the task to be trained into the model for training, performing feature extraction through the encoder, and obtaining the mean and covariance of the task; 3) Based on the Bayesian variational inference method, the posterior distribution of the representation in the network is obtained, the uncertainty of the parameter estimation and the loss of the model variational inference are calculated; the loss function used by the Bayesian variational inference method is as follows: Among them, t, N t are the t-th task and the number of samples it contains; represents the nth sample and its true label; q t (ω), represents the posterior distribution of the latent variable z in the hidden space of the t-th and t-1-th tasks; ω represents the parameters obtained in its network; in Bayesian variational inference, in order to obtain the parameter distribution of its task, KL divergence (Kullback-Leibler divergence) is used for measurement, then KL(q t (ω)||q t-1 (ω)), in order to obtain the posterior distribution of the current task, the posterior distribution of the previous task is used as the prior distribution of the current model training; is the maximum likelihood estimation function; during the training process, it is necessary to minimize the Bayesian variational inference loss function 4) Through maximum likelihood estimation, the model classifier is trained to obtain the optimal classification result; 5) Substitute the potential feature prior distribution obtained in step 2 above for a Gaussian mixture distribution, generate pseudo sample images through the decoder, and calculate the loss of the deep generative model; The Gaussian mixture model (GMM) is a ubiquitous statistical model used for density estimation, pattern recognition, and function approximation. It has analytical tractability, asymptotic properties, and the ability to approximate continuous density functions. When generating pseudo samples, the Gaussian mixture probability distribution is used instead of the prior distribution, so that each category has a separate model. The distribution of the latent variable z is more flexible, while increasing the model's plasticity. By limiting the sampling of z, the model generates a specified category. The features of the original latent space are converted into: in p Ω (z) are the posterior distribution and prior distribution of the Gaussian mixture model GMM, Ω is the set of mean and standard deviation of each Gaussian model, α k =y represents the input category, K is α k Total number of inputs, corresponding class prior coefficient ω k =ω' k =p(y=α k )=1 / k, The KL divergence loss of the latent variables obtained based on the Gaussian mixture prior distribution is further: in and is the mean and standard deviation of the k-th target value; Decoder network p φ According to the posterior distribution of the latent variable z Reconstruct pseudo sample images Let the pseudo samples generated by the latent variable feature z Similar to the original task input data, to reduce the error of generating pseudo samples, the reconstruction loss function expression should become: in, φ are the distribution parameters of the encoder and decoder respectively; the loss function of the further deep generation model should be: 。 2. The continuous learning method based on Bayesian variational inference according to claim 1, characterized in that: Perform encoder network feature extraction, Where z represents the feature vector of the data in the latent space.