Self-adaptive continuous learning method and device, equipment, storage medium and product

By obtaining optimized hyperparameters and building objective functions, and combining time ensemble algorithms to correct the stability gap, the stability and reliability problems of continuous learning in highly dynamic and uncertain scenarios are solved, reducing catastrophic forgetting and improving Model adaptability and performance.

CN119067203APending Publication Date: 2024-12-03TSINGHUA UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410999983.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing continuous learning technologies have poor stability and reliability in highly dynamic and uncertain scenarios, making them difficult to adapt to changes in data distribution, resulting in serious catastrophic forgetting problems.

Method used

By obtaining optimized hyperparameters, building objective functions, and applying time-integration algorithms to correct the stability gap, dynamic adjustment of hyperparameters during continuous learning training, reducing the degree of catastrophic forgetting, and improving stability and reliability.

Benefits of technology

Effectively reduce the degree of catastrophic forgetting, improve the stability and reliability of continuous learning, and enable the model to better adapt to new tasks and maintain the performance of old tasks under dynamic data distribution.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067203A_ABST
    Figure CN119067203A_ABST
Patent Text Reader

Abstract

The invention provides an adaptive continuous learning method and device, equipment, a storage medium and a product, and the method comprises the steps: obtaining optimizable hyper-parameters which comprise a first uncertain parameter corresponding to a new task, a second uncertain parameter corresponding to a number space and a third uncertain parameter corresponding to an output space; constructing a current objective function based on the optimizable hyper-parameters; based on the current objective function, training the current continuous learning model to learn a new task; a time integration algorithm is applied to correct a stability gap; wherein the stability difference represents the difference between the optimizable hyper-parameter and the current training progress. According to the scheme of the invention, the dynamic adjustment of the hyper-parameters in the continuous learning training process is realized, and the stability and reliability of continuous learning are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of continuous learning technology, and in particular to an adaptive continuous learning method, device, equipment, storage medium and product. Background Art

[0002] Continual Learning (CL), also known as incremental learning or lifelong learning, is a learning paradigm for machine learning of multiple tasks. Multiple task data are delivered to the continuous learning algorithm in sequence (only the current task data is delivered each time), so that the final learned model can handle all tasks. The catastrophic forgetting of old tasks when deep models learn new tasks is one of the difficulties of continuous learning.

[0003] In the existing technology, the degree of catastrophic forgetting is reduced through hyperparameter optimization (HPO). The current hyperparameter optimization is suitable for relatively static data distribution and is difficult to adapt to continuous learning scenarios with high dynamics and uncertainty.

[0004] Therefore, the stability and reliability of existing continuous learning are poor. Summary of the invention

[0005] The present invention provides an adaptive continuous learning method, device, equipment, storage medium and product to solve the defects of poor stability and reliability of existing continuous learning in the prior art, realize dynamic adjustment of hyperparameters during continuous learning training, and improve the stability and reliability of continuous learning.

[0006] The present invention provides an adaptive continuous learning method, comprising: obtaining optimizable hyperparameters, the optimizable hyperparameters including a first uncertain parameter corresponding to a new task, a second uncertain parameter corresponding to a parameter space, and a third uncertain parameter corresponding to an output space; constructing a current objective function based on the optimizable hyperparameters; training a current continuous learning model based on the current objective function to learn new tasks; applying a time integration algorithm to correct a stability gap; wherein the stability gap represents a gap between the optimizable hyperparameters and the current training progress.

[0007] According to an adaptive continuous learning method provided by the present invention, the obtaining of optimizable hyperparameters includes: applying a reservoir sampling algorithm to sample network parameters of old tasks to obtain sampling data corresponding to the old tasks; obtaining network parameters of the new task; and determining the optimizable hyperparameters based on the network parameters of the old tasks and the network parameters of the new tasks.

[0008] According to an adaptive continuous learning method provided by the present invention, the current objective function is constructed based on the optimizable hyperparameters, including: for a regression problem, based on the optimizable hyperparameters, determining a first log-likelihood expression corresponding to a predicted distribution; for a classification problem, based on the optimizable hyperparameters, determining a second log-likelihood expression corresponding to the predicted distribution; and constructing the current objective function based on the first log-likelihood expression and the second log-likelihood expression.

[0009] According to an adaptive continuous learning method provided by the present invention, for the regression problem, based on the optimizable hyperparameters, determining the first log-likelihood expression corresponding to the predicted distribution, includes: for the regression problem, based on the optimizable hyperparameters, determining the Gaussian approximate expression of the continuous probability density corresponding to the predicted distribution; according to the Gaussian approximate expression of the continuous probability density corresponding to the predicted distribution, determining the first log-likelihood expression corresponding to the predicted distribution.

[0010] According to an adaptive continuous learning method provided by the present invention, for the classification problem, based on the optimizable hyperparameters, the second log-likelihood expression corresponding to the predicted distribution is determined, including: for the classification problem, based on the optimizable hyperparameters, determining the expression of the discrete probability density corresponding to the predicted distribution; according to the expression of the discrete probability density corresponding to the predicted distribution, determining the second log-likelihood expression corresponding to the predicted distribution.

[0011] According to an adaptive continuous learning method provided by the present invention, before training the current continuous learning model based on the current objective function to learn a new task, the method further includes: initializing the optimizable hyperparameters.

[0012] The present invention also provides an adaptive continuous learning device, comprising: an acquisition module for acquiring optimizable hyperparameters, the optimizable hyperparameters including a first uncertain parameter corresponding to a new task, a second uncertain parameter corresponding to a parameter space, and a third uncertain parameter corresponding to an output space; a construction module for constructing a current objective function based on the optimizable hyperparameters; a training module for training a current continuous learning model based on the current objective function to learn new tasks; a correction module for applying a time integration algorithm to correct a stability gap; wherein the stability gap characterizes the gap between the optimizable hyperparameters and the current training progress.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the adaptive continuous learning method as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the adaptive continuous learning method as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the above-mentioned adaptive continuous learning methods.

[0016] The adaptive continuous learning method, device, equipment, storage medium and product provided by the present invention obtain optimizable hyperparameters; construct the current objective function based on the optimizable hyperparameters; train the current continuous learning model based on the current objective function to learn new tasks; apply the time integration algorithm to correct the stability gap; wherein the stability gap characterizes the gap between the optimizable hyperparameters and the current training progress. The scheme of the present application constructs the current objective function based on the optimizable hyperparameters, that is, incorporates the optimizable hyperparameters into the optimization goal of continuous learning, and further applies the time integration algorithm to correct the stability gap, which can realize the dynamic adjustment of the hyperparameters during the continuous learning training process, can reduce the degree of catastrophic forgetting, and improve the stability and reliability of continuous learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the flow charts of the adaptive continuous learning method provided by the present invention.

[0019] Figure 2 This is the second flow chart of the adaptive continuous learning method provided by the present invention.

[0020] Figure 3 It is a structural schematic diagram of the adaptive continuous learning device provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0024] The terms "first", "second", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise indicated. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented in an order other than those given in the diagrams or descriptions of the embodiments of this application.

[0025] In addition, the terms "including" and "having" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a series of components is not necessarily limited to those components explicitly listed, but may include other components not explicitly listed or inherent to these products or devices. The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that can perform the functions associated with the element.

[0026] The term "hyperparameter" used in each embodiment of the present application refers to a parameter manually set before the algorithm runs to control the behavior and performance of the model. In practical applications, the choice of hyperparameters will affect the training speed, convergence, capacity and generalization ability of the model. For example, hyperparameters include learning rate, number of iterations, regularization parameters, etc.

[0027] The term "Continual Learning (CL)" used in each embodiment of the present application, also known as incremental learning or lifelong learning, is a learning paradigm for machine learning of multiple tasks. Multiple task data are delivered to the continuous learning algorithm in sequence (only the current task data each time), so that the final learned model can be competent for all tasks.

[0028] In practical applications, the performance of continuous learning can be evaluated from three aspects: the overall performance of the tasks learned so far, the memory stability of old tasks, and the learning plasticity of new tasks. For example, the overall performance of the tasks learned so far can be evaluated by the average accuracy (AA) and the average incremental accuracy (AIA). The memory stability of old tasks can be evaluated by the forgetting measure (FM) and the backward transfer (BWT). The learning plasticity of new tasks can be evaluated by the invariance measure (IM) and the forward transfer (FWT).

[0029] Catastrophic forgetting of old tasks when deep models learn new tasks is one of the difficulties of continuous learning. In some exemplary techniques, continuous learning methods attempt to determine the balance between task contributions in the same parameter space. For example, regularization-based methods use explicit regularization terms to selectively stabilize network parameters and simulate old models. For another example, replay-based methods approximate and restore old data distributions by retaining small memory buffers or generating models. In other exemplary techniques, continuous learning methods optimize network parameters in different parameter spaces, explicitly avoiding the problem of balancing task contributions. Other methods for optimizing network parameters in different parameter spaces are generally referred to as architecture-based methods.

[0030] Specifically, according to the availability of training samples, the widely used continuous learning settings can be divided into online continuous learning and offline continuous learning, among which online continuous learning methods are more challenging. The objective function of continuous learning can be classified into a unified framework, and the expression of the objective function L is as follows.

[0031]

[0032] in, represents the loss function for learning new tasks, Used to limit the update rate on the output space, Used to limit the update rate in parameter space. , and for , and The corresponding hyperparameters.

[0033] In practical applications, an important issue in continuous learning is task balance, which ensures that the model maintains the performance of old tasks while learning new tasks. For representative continuous learning methods, it is necessary to accurately set hyperparameters (such as learning rate, regularization strength, memory buffer size, , and ) is crucial to achieve excellent task balance on dynamic data distribution.

[0034] In some exemplary techniques, multi-task learning attempts to solve the task balance problem through various weighting strategies. Continuous learning and multi-task learning aim to achieve the same goal, which is to find a solution that performs well for all observed tasks. The objective function of multi-task learning can be defined as follows.

[0035]

[0036] in, represents the loss function of each task, represents the hyperparameters that adjust the task weights. Currently, multi-task learning methods optimize hyperparameters by selecting deterministic hyperparameters from web searches or simplifying them to unweighted forms. In addition, multi-task learning methods model hyperparameters as optimizable functions, but the hyperparameters are related to the relative confidence between tasks. However, due to the dynamic and unpredictable nature of data distribution, these adaptive weighting strategies have not been fully explored.

[0037] It can be seen that the current hyperparameter optimization is suitable for relatively static data distribution and is difficult to adapt to continuous learning scenarios with high dynamics and uncertainty. Therefore, the stability and reliability of existing continuous learning are poor.

[0038] In response to the above technical problems, the present invention proposes the following technical concept: obtaining optimizable hyperparameters, incorporating the optimizable hyperparameters into the optimization objectives of continuous learning, and further applying a time integration algorithm to correct the stability gap to achieve dynamic adjustment of the hyperparameters during the continuous learning training process, so that the continuous learning method can adapt to scenarios with high dynamics and uncertainty, thereby reducing the degree of catastrophic forgetting, and further improving the stability and reliability of continuous learning.

[0039] The following specific embodiments are used to describe in detail the technical solution of the present application and how the technical solution of the present application solves the above technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Figure 1 and Figure 2 The improved adaptive continuous learning method of the present invention is described.

[0040] In practical applications, the executor of the adaptive continuous learning method can be an adaptive continuous learning device, which can be implemented in many ways, for example, through a computer program, such as application software, or a chip, etc. It can also be implemented as a medium storing relevant computer programs, such as a USB disk, a cloud disk, etc. Or, it can also be implemented through a physical device integrated or installed with relevant computer programs, such as a server, etc.

[0041] Figure 1 is one of the flow charts of the adaptive continuous learning method provided by the present invention, such as Figure 1 As shown, the method includes the following: step 101 to step 104.

[0042] Step 101: Obtain optimizable hyperparameters, where the optimizable hyperparameters include a first uncertain parameter corresponding to the new task, a second uncertain parameter corresponding to the parameter space, and a third uncertain parameter corresponding to the output space.

[0043] Among them, the hyperparameter that can be optimized is the hyperparameter selected to adaptively balance the contribution of the task. The adaptive continuous learning method provided in this embodiment is committed to solving the catastrophic forgetting problem by limiting the model update in the output space and / or parameter space, thereby retaining the knowledge previously learned by the deep model (i.e., the knowledge learned by the deep model for the old task).

[0044] Combined with the above description, the objective function of continuous learning can be classified into a unified framework, and the expression of the objective function L is as follows.

[0045]

[0046] in, represents the loss function for learning new tasks, Used to limit the update rate on the output space, Used to limit the update rate in parameter space. , and for , and The corresponding hyperparameters.

[0047] Specifically, the optimizable hyperparameters can be expressed as , corresponding to the objective function of continuous learning, can optimize the hyperparameters Including the first uncertain parameter corresponding to the new task , the second uncertain parameter corresponding to the parameter space And the third uncertain parameter corresponding to the output space .

[0048] As an example of a method for obtaining optimizable hyperparameters, in one possible implementation, Figure 2 This is the second flow chart of the adaptive continuous learning method provided by the present invention, such as Figure 2 As shown, the above step 101 includes: step 201 to step 203.

[0049] Step 201: Apply the reservoir sampling algorithm to sample the network parameters of the old task to obtain the sampling data corresponding to the old task.

[0050] Step 202: Obtain network parameters for the new task.

[0051] Step 203: Determine optimizable hyperparameters based on the network parameters of the old task and the network parameters of the new task.

[0052] It can be understood that the network parameters of the old task represent the knowledge that the deep network has currently learned. Specifically, the network parameters of the old task are sampled to obtain the sampled data corresponding to the old task, and the sampled data corresponding to the old task is stored in the buffer. Furthermore, when it is necessary to learn the new task, the network parameters of the new task are obtained. Furthermore, the network parameters of the old task and the network parameters of the new task are comprehensively considered to determine the hyperparameters that can be optimized.

[0053] Step 102: construct the current objective function based on the optimizable hyperparameters.

[0054] Specifically, in the learning process of each task of the continuous learning deep model, the predicted distribution The confidence level gradually increases, and the corresponding hyperparameters can be optimized It gradually decreases and finally stabilizes to a certain value.

[0055] It is understandable that the loss Corresponding to solving the regression problem, the loss term and loss items Corresponding to solving classification problems. Classification problems are used to predict discrete values, and regression problems are used to predict continuous values. This application focuses on homoscedastic arbitrary uncertainty, that is, optimizing hyperparameters It does not depend on specific input data, but remains constant for all input data and only varies between different tasks.

[0056] In this embodiment, by analyzing the regression problem and the classification problem, The corresponding hyperparameters , expressed as the first uncertain parameter Function ; In the objective function The corresponding hyperparameters , expressed as the second uncertain parameter Function ; In the objective function The corresponding hyperparameters , expressed as the third uncertain parameter Function .

[0057] Specifically, by analyzing the regression problem and the classification problem, the current objective function is constructed. In a possible implementation, the above step 102 includes: for the regression problem, based on the optimizable hyperparameters, determining the first log-likelihood expression corresponding to the predicted distribution; for the classification problem, based on the optimizable hyperparameters, determining the second log-likelihood expression corresponding to the predicted distribution; and constructing the current objective function based on the first log-likelihood expression and the second log-likelihood expression.

[0058] Specifically, the regression problem is analyzed. In one example, for the regression problem, based on the optimizable hyperparameters, a first log-likelihood expression corresponding to the predicted distribution is determined, including: for the regression problem, based on the optimizable hyperparameters, a Gaussian approximate expression of the continuous probability density corresponding to the predicted distribution is determined; based on the Gaussian approximate expression of the continuous probability density corresponding to the predicted distribution, a first log-likelihood expression corresponding to the predicted distribution is determined.

[0059] Specifically, for regression problems, the predicted distribution It can be defined as a Gaussian distribution under the Laplace approximation, predicting the distribution The corresponding Gaussian approximate expression of the continuous probability density is as follows:

[0060] in, Characterizing homoscedastic aleatoric uncertainty.

[0061] Furthermore, the first log-likelihood expression corresponding to the predicted distribution is determined based on the Gaussian approximate expression of the continuous probability density corresponding to the predicted distribution. In practical applications, when the average score cannot be used as the loss function, the first log-likelihood expression is as follows:

[0062] Correspondingly, the classification problem is analyzed. In one example, for the classification problem, based on the optimizable hyperparameters, a second log-likelihood expression corresponding to the predicted distribution is determined, including: for the classification problem, based on the optimizable hyperparameters, an expression of discrete probability density corresponding to the predicted distribution is determined; according to the expression of discrete probability density corresponding to the predicted distribution, a second log-likelihood expression corresponding to the predicted distribution is determined.

[0063] Specifically, for classification problems, applying Boltzmann distribution modeling can learn the predictive distribution , predicting the distribution The corresponding discrete probability density expression is as follows:

[0064] in, Characterizing the degree of entropy divergence in deterministic discrete distribution, the corresponding log-likelihood becomes:

[0065] Furthermore, for discrete classification problems, the second log-likelihood expression corresponding to the predicted distribution is determined according to the expression of the discrete probability density corresponding to the predicted distribution. The second log-likelihood expression is as follows:

[0066] Furthermore, the current objective function is constructed, and the current objective function expression is as follows.

[0067]

[0068] in, is an additional term used to avoid optimizable hyperparameters The rapid changes of maintain the numerical stability of relative weights. represents the loss function for learning new tasks, Used to limit the update rate on the output space, Used to limit the update rate in parameter space.

[0069] Step 103: Based on the current objective function, the current continuous learning model is trained to learn new tasks.

[0070] Step 104: Apply a time integration algorithm to correct the stability gap; wherein the stability gap characterizes the gap between the optimizeable hyperparameters and the current training progress.

[0071] In practical applications, continuous learning encounters the challenge of stability gap during training, that is, it suffers a lot of forgetting when starting to learn a new task, and then gradually recovers performance. In order to ensure that the hyperparameters can be optimized In order to match the current training progress and adapt to the continuous learning scenario with high dynamics and uncertainty, the stability gap needs to be corrected to achieve dynamic adjustment of hyperparameters during continuous learning training, thereby reducing the degree of catastrophic forgetting and improving the stability and reliability of continuous learning.

[0072] Specifically, along the training trajectory, the current model is associated with the historical model according to the Exponential Moving Average (EMA) algorithm to correct the stability gap. For example, the historical model is recorded as , the current model is recorded as , in the present invention, the timing integration algorithm is defined as follows:

[0073] in, is the sliding average control parameter, which is controlled by The size of can control the impact of the sliding average, so as to more effectively utilize the influence of different historical models in the training trajectory.

[0074] In practice, temporal integration algorithms can be applied to different models to increase the stability to perturbations or reinitializations of hyperparameters, thus enabling correction of the stability gap.

[0075] The solution of this embodiment constructs the current objective function based on the optimizable hyperparameters, that is, incorporates the optimizable hyperparameters into the optimization goal of continuous learning. Further, the time integration algorithm is applied to correct the stability gap, which can realize the dynamic adjustment of the hyperparameters during the continuous learning training process, reduce the degree of catastrophic forgetting, and improve the stability and reliability of continuous learning.

[0076] In addition, in order to further improve the stability and reliability of continuous learning, in a possible implementation manner, before the above step 103, the adaptive continuous learning method further includes: initializing the above optimizable hyperparameters.

[0077] In the prior art, the hyperparameter values ​​obtained are usually only for the model performance after the training is completed. When the data distribution changes significantly during the training process, the use of deterministic hyperparameter values ​​may not match the training progress. That is, when starting to learn a new task, the performance of the previous task will drop rapidly. As the new task continues to be trained, the model performance gradually recovers, thereby aggravating the catastrophic forgetting phenomenon.

[0078] In this implementation, when the data distribution changes significantly, the optimizable hyperparameter is initialized to a higher value to adapt to the changes in different data distributions, that is, to meet the learning rule that the uncertainty increases again in the new data distribution and then gradually decreases. The solution of this implementation improves the stability and reliability of continuous learning.

[0079] In the adaptive continuous learning method provided in this embodiment, by obtaining optimizable hyperparameters; constructing the current objective function based on the optimizable hyperparameters; training the current continuous learning model based on the current objective function to learn new tasks; applying the time integration algorithm to correct the stability gap; wherein the stability gap characterizes the gap between the optimizable hyperparameters and the current training progress. The scheme of the present application constructs the current objective function based on the optimizable hyperparameters, that is, the optimizable hyperparameters are incorporated into the optimization goal of continuous learning, and further, the time integration algorithm is applied to correct the stability gap, which can realize the dynamic adjustment of the hyperparameters during the continuous learning training process, can reduce the degree of catastrophic forgetting, and improve the stability and reliability of continuous learning.

[0080] The adaptive continuous learning device provided by the present invention is described below. The adaptive continuous learning device described below and the adaptive continuous learning method described above can be referenced to each other.

[0081] Figure 3 is a schematic diagram of the structure of the adaptive continuous learning device provided by the present invention, such as Figure 3 As shown, the above-mentioned adaptive continuous learning device includes: an acquisition module 31, a construction module 32, a training module 33 and a correction module 34.

[0082] The acquisition module 31 is used to acquire optimizable hyperparameters, which include a first uncertain parameter corresponding to the new task, a second uncertain parameter corresponding to the parameter space, and a third uncertain parameter corresponding to the output space.

[0083] The hyperparameters that can be optimized are the hyperparameters selected to adaptively balance the contribution of the task. The adaptive continuous learning device provided in this embodiment is dedicated to solving the catastrophic forgetting problem by limiting the model update in the output space and / or parameter space, thereby retaining the knowledge previously learned by the deep model (i.e., the knowledge learned by the deep model for the old task).

[0084] Combined with the above description, the objective function of continuous learning can be classified into a unified framework, and the expression of the objective function L is as follows.

[0085]

[0086] in, represents the loss function for learning new tasks, Used to limit the update rate on the output space, Used to limit the update rate in parameter space. , and for , and The corresponding hyperparameters.

[0087] Specifically, the optimizable hyperparameters can be expressed as , corresponding to the objective function of continuous learning, can optimize the hyperparameters Including the first uncertain parameter corresponding to the new task , the second uncertain parameter corresponding to the parameter space And the third uncertain parameter corresponding to the output space .

[0088] As an example, in one possible implementation, the acquisition module 31 is specifically used to: apply a reservoir sampling algorithm to sample the network parameters of the old task to obtain the sampled data corresponding to the old task. Obtain the network parameters of the new task. Determine the optimizable hyperparameters based on the network parameters of the old task and the network parameters of the new task.

[0089] It can be understood that the network parameters of the old task represent the knowledge that the deep network has currently learned. Specifically, the network parameters of the old task are sampled to obtain the sampled data corresponding to the old task, and the sampled data corresponding to the old task is stored in the buffer. Furthermore, when it is necessary to learn the new task, the network parameters of the new task are obtained. Furthermore, the network parameters of the old task and the network parameters of the new task are comprehensively considered to determine the hyperparameters that can be optimized.

[0090] The above-mentioned construction module 32 is used to construct the current objective function based on the optimizable hyperparameters.

[0091] Specifically, in the learning process of each task of the continuous learning deep model, the predicted distribution The confidence level gradually increases, and the corresponding hyperparameters can be optimized It gradually decreases and finally stabilizes to a certain value.

[0092] It is understandable that the loss Corresponding to solving the regression problem, the loss term and loss items Corresponding to solving classification problems. Classification problems are used to predict discrete values, and regression problems are used to predict continuous values. This application focuses on homoscedastic arbitrary uncertainty, that is, optimizing hyperparameters It does not depend on specific input data, but remains constant for all input data and only varies between different tasks.

[0093] In this embodiment, by analyzing the regression problem and the classification problem, The corresponding hyperparameters , expressed as the first uncertain parameter Function ; In the objective function The corresponding hyperparameters , expressed as the second uncertain parameter Function ; In the objective function The corresponding hyperparameters , expressed as about the third uncertain parameter Function .

[0094] Specifically, by analyzing the regression problem and the classification problem, the current objective function is constructed. In a possible implementation, the above-mentioned construction module 32 includes: a first analysis unit, which is used to determine, for the regression problem, a first log-likelihood expression corresponding to the predicted distribution based on an optimizable hyperparameter; a second analysis unit, which is used to determine, for the classification problem, a second log-likelihood expression corresponding to the predicted distribution based on an optimizable hyperparameter; and a construction unit, which is used to construct the current objective function based on the first log-likelihood expression and the second log-likelihood expression.

[0095] Specifically, the regression problem is analyzed. In one example, the above-mentioned first analysis unit is specifically used to: for the regression problem, based on the optimizable hyperparameters, determine the Gaussian approximate expression of the continuous probability density corresponding to the predicted distribution; according to the Gaussian approximate expression of the continuous probability density corresponding to the predicted distribution, determine the first log-likelihood expression corresponding to the predicted distribution.

[0096] Specifically, for regression problems, the predicted distribution It can be defined as a Gaussian distribution under the Laplace approximation, predicting the distribution The corresponding Gaussian approximate expression of the continuous probability density is as follows:

[0097] in, Characterizing homoscedastic aleatoric uncertainty.

[0098] Furthermore, the first log-likelihood expression corresponding to the predicted distribution is determined based on the Gaussian approximate expression of the continuous probability density corresponding to the predicted distribution. In practical applications, when the average score cannot be used as the loss function, the first log-likelihood expression is as follows:

[0099] Correspondingly, the classification problem is analyzed. In one example, the above-mentioned second analysis unit is specifically used to: for the classification problem, determine the expression of the discrete probability density corresponding to the predicted distribution based on the optimizable hyperparameters; according to the expression of the discrete probability density corresponding to the predicted distribution, determine the second log-likelihood expression corresponding to the predicted distribution.

[0100] Specifically, for classification problems, applying Boltzmann distribution modeling can learn the predictive distribution , predicting the distribution The corresponding discrete probability density expression is as follows:

[0101] in, Characterizing the degree of entropy divergence in deterministic discrete distribution, the corresponding log-likelihood becomes:

[0102] Furthermore, for discrete classification problems, the second log-likelihood expression corresponding to the predicted distribution is determined according to the expression of the discrete probability density corresponding to the predicted distribution. The second log-likelihood expression is as follows:

[0103] Furthermore, the construction module 32 constructs the current objective function, and the current objective function expression is as follows.

[0104]

[0105] in, is an additional term used to avoid optimizable hyperparameters The rapid changes of maintain the numerical stability of relative weights. represents the loss function for learning new tasks, Used to limit the update rate on the output space, Used to limit the update rate in parameter space.

[0106] The training module 33 is used to train the current continuous learning model based on the current objective function to learn new tasks.

[0107] The correction module 34 is used to apply a time integration algorithm to correct the stability gap, wherein the stability gap represents the gap between the optimizable hyperparameter and the current training progress.

[0108] In practical applications, continuous learning encounters the challenge of stability gap during training, that is, it suffers a lot of forgetting when starting to learn a new task, and then gradually recovers performance. In order to ensure that the hyperparameters can be optimized In order to match the current training progress and adapt to the continuous learning scenario with high dynamics and uncertainty, the stability gap needs to be corrected to achieve dynamic adjustment of hyperparameters during continuous learning training, thereby reducing the degree of catastrophic forgetting and improving the stability and reliability of continuous learning.

[0109] Specifically, along the training trajectory, the current model is associated with the historical model according to the Exponential Moving Average (EMA) algorithm to correct the stability gap. For example, the historical model is recorded as , the current model is recorded as , in the present invention, the timing integration algorithm is defined as follows:

[0110] in, is the sliding average control parameter, which is controlled by The size of can control the impact of the sliding average, so as to more effectively utilize the influence of different historical models in the training trajectory.

[0111] In practice, temporal integration algorithms can be applied to different models to increase the stability to perturbations or reinitializations of hyperparameters, thus enabling correction of the stability gap.

[0112] In the solution of this embodiment, the construction module 32 constructs the current objective function based on the optimizable hyperparameters, that is, the optimizable hyperparameters are incorporated into the optimization target of continuous learning. Furthermore, the correction module 34 applies a time integration algorithm to correct the stability gap, which can realize dynamic adjustment of the hyperparameters during continuous learning training, reduce the degree of catastrophic forgetting, and improve the stability and reliability of continuous learning.

[0113] In addition, in order to further improve the stability and reliability of continuous learning, in a possible implementation manner, the above-mentioned adaptive continuous learning device also includes: an initialization module, which is used to initialize the above-mentioned optimizable hyperparameters.

[0114] In this implementation, when the data distribution changes significantly, the optimizable hyperparameter is initialized to a higher value to adapt to the changes in different data distributions, that is, to meet the learning rule that the uncertainty increases again in the new data distribution and then gradually decreases. The solution of this implementation improves the stability and reliability of continuous learning.

[0115] In the adaptive continuous learning device provided by this embodiment, the acquisition module acquires the optimizable hyperparameters; the construction module constructs the current objective function based on the optimizable hyperparameters; the training module trains the current continuous learning model based on the current objective function to learn new tasks; the correction module applies the time integration algorithm to correct the stability gap; wherein the stability gap characterizes the gap between the optimizable hyperparameters and the current training progress. The scheme of this embodiment constructs the current objective function based on the optimizable hyperparameters, that is, the optimizable hyperparameters are incorporated into the optimization goal of continuous learning. Further, the time integration algorithm is applied to correct the stability gap, which can realize the dynamic adjustment of the hyperparameters during the continuous learning training process, can reduce the degree of catastrophic forgetting, and improve the stability and reliability of continuous learning.

[0116] Figure 4 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the adaptive continuous learning method, which includes: obtaining an optimizable hyperparameter, the optimizable hyperparameter includes a first uncertain parameter corresponding to the new task, a second uncertain parameter corresponding to the parameter space and a third uncertain parameter corresponding to the output space; constructing the current objective function based on the optimizable hyperparameter; training the current continuous learning model based on the current objective function to learn the new task; applying the time integration algorithm to correct the stability gap; wherein the stability gap represents the gap between the optimizable hyperparameter and the current training progress.

[0117] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0118] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the adaptive continuous learning method provided by the above methods, and the method includes: obtaining optimizable hyperparameters, the optimizable hyperparameters include a first uncertain parameter corresponding to the new task, a second uncertain parameter corresponding to the parameter space, and a third uncertain parameter corresponding to the output space; based on the optimizable hyperparameters, constructing the current objective function; based on the current objective function, training the current continuous learning model to learn new tasks; applying a time integration algorithm to correct the stability gap; wherein the stability gap characterizes the gap between the optimizable hyperparameters and the current training progress.

[0119] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the adaptive continuous learning method provided by the above-mentioned methods, the method comprising: obtaining optimizable hyperparameters, the optimizable hyperparameters comprising a first uncertain parameter corresponding to the new task, a second uncertain parameter corresponding to the parameter space, and a third uncertain parameter corresponding to the output space; based on the optimizable hyperparameters, constructing a current objective function; based on the current objective function, training the current continuous learning model to learn new tasks; applying a time integration algorithm to correct the stability gap; wherein the stability gap characterizes the gap between the optimizable hyperparameters and the current training progress.

[0120] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0121] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive continuous learning method, It is characterized in that include: Obtaining optimizable hyperparameters, where the optimizable hyperparameters include a first uncertain parameter corresponding to the new task, a second uncertain parameter corresponding to the parameter space, and a third uncertain parameter corresponding to the output space; Based on the optimizable hyperparameters, construct a current objective function; Based on the current objective function, training the current continuous learning model to learn new tasks; A time integration algorithm is applied to correct a stability gap, wherein the stability gap represents a gap between the optimizable hyperparameter and the current training progress.

2. The adaptive continuous learning method according to claim 1, It is characterized in that The obtaining of the optimizable hyper parameters includes: Applying a reservoir sampling algorithm to sample network parameters of old tasks, and obtaining sampling data corresponding to the old tasks; Obtaining network parameters of the new task; The optimizable hyperparameters are determined according to the network parameters of the old task and the network parameters of the new task.

3. The adaptive continuous learning method according to claim 1, It is characterized in that The constructing of the current objective function based on the optimizable hyperparameters includes: For the regression problem, determining a first log-likelihood expression corresponding to the predicted distribution based on the optimizable hyperparameters; For the classification problem, based on the optimizable hyperparameter, determining a second log-likelihood expression corresponding to the predicted distribution; A current objective function is constructed according to the first log-likelihood expression and the second log-likelihood expression.

4. The adaptive continuous learning method according to claim 3, It is characterized in that For the regression problem, determining a first log-likelihood expression corresponding to the predicted distribution based on the optimizable hyperparameters includes: For the regression problem, determining a Gaussian approximate expression of the continuous probability density corresponding to the predicted distribution based on the optimizable hyperparameters; According to the Gaussian approximate expression of the continuous probability density corresponding to the predicted distribution, a first log-likelihood expression corresponding to the predicted distribution is determined.

5. The adaptive continuous learning method according to claim 3, It is characterized in that The step of determining, for the classification problem, a second log-likelihood expression corresponding to the predicted distribution based on the optimizable hyperparameters includes: For a classification problem, determining an expression for a discrete probability density corresponding to the predicted distribution based on the optimizable hyperparameters; According to the expression of discrete probability density corresponding to the predicted distribution, a second log-likelihood expression corresponding to the predicted distribution is determined.

6. The adaptive continuous learning method according to any one of claims 1 to 5, It is characterized in that Before training the current continuous learning model based on the current objective function to learn a new task, the method further includes: The optimizable hyperparameters are initialized.

7. An adaptive continuous learning device, It is characterized in that include: An acquisition module, used to acquire optimizable hyperparameters, wherein the optimizable hyperparameters include a first uncertain parameter corresponding to the new task, a second uncertain parameter corresponding to the parameter space, and a third uncertain parameter corresponding to the output space; A construction module, used to construct a current objective function based on the optimizable hyperparameters; A training module, used for training the current continuous learning model based on the current objective function to learn new tasks; A correction module is used to apply a time integration algorithm to correct a stability gap; wherein the stability gap represents a gap between the optimizable hyperparameter and the current training progress.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the adaptive continuous learning method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the adaptive continuous learning method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the adaptive continuous learning method according to any one of claims 1 to 6 is implemented.