A neural network continual learning method and related devices

Abstract

The application discloses a neural network continuous learning method and related equipment, including: receiving a new input sample writing a short-term memory area, evaluating the importance of parameters to learned tasks to adjust their training plasticity, associatively retrieving relevant historical knowledge from a long-time / external memory area, mixing and playing back training with new samples, and migrating knowledge among multiple memory areas to network convergence according to rules. The application uses dual protection of parameter dynamic adjustment and historical knowledge playback to overcome catastrophic forgetting, and avoids performance degradation of old tasks. By storing knowledge representations instead of original samples, combining knowledge migration rules suitable for tasks, reducing storage and computing costs, and improving scalability, the association and integration of new and old knowledge are realized through associative retrieval, and the generalization ability is enhanced. A multi-memory area knowledge circulation mechanism is constructed, the external memory is deeply involved in training, and the interaction efficiency is improved. The overall balance of the efficiency, stability and generalization of continuous learning significantly optimizes the performance of the neural network continuous learning.

Description

Technical Field

[0001] This application relates to the field of neural network technology, and more specifically, to a method and related device for continuous learning of neural networks. Background Technology

[0002] With the rapid development of deep learning technology, large-scale neural network models have made breakthroughs in fields such as natural language processing and computer vision, driving the widespread application of artificial intelligence in real-world scenarios. In practical applications, neural network models often need to iteratively adapt to new tasks through continuous learning or incremental learning based on new business needs or data increments.

[0003] However, existing neural network models generally suffer from a core technical bottleneck when facing such continuous learning scenarios: catastrophic forgetting. This means that during training and optimization for new tasks, the model rapidly forgets previously mastered knowledge of older tasks, leading to a significant drop in performance for those older tasks. This severely impacts the long-term reusability and practical application value of the model, becoming a key obstacle restricting the further promotion of deep learning technology in scenarios requiring continuous iterative updates.

[0004] Currently, the industry has conducted research on the above-mentioned problems and proposed some continuous learning optimization methods. However, these methods still have obvious shortcomings: First, the catastrophic forgetting problem has not been effectively solved. Even with mainstream strategies such as parameter regularization and sample replay, the performance degradation of old tasks is still significant when the model learns new tasks, making it difficult to meet the performance stability requirements of practical applications. Second, the efficiency and scalability of existing methods are insufficient. Some solutions require additional storage of a large number of historical task samples or frequent replay of old samples during the training of new tasks, which leads to a significant increase in the consumption of computational resources for model training and a significant increase in storage costs. Especially when dealing with large-scale, multi-batch tasks, it is difficult to achieve efficient adaptation. Third, existing methods mostly rely on fixed sample replay or parameter constraint logic, resulting in poor generalization performance in scenarios such as cross-task knowledge transfer and rapid adaptation to new tasks.

[0005] In summary, the industry urgently needs a deep neural network continuous learning solution that can efficiently achieve continuous learning, effectively avoid catastrophic forgetting, and has strong knowledge generalization capabilities, in order to break through existing technological bottlenecks. Summary of the Invention

[0006] This application provides a method and related device for continuous learning of neural networks. It overcomes catastrophic forgetting by using dynamic parameter adjustment and historical knowledge playback as dual safeguards to avoid performance degradation of old tasks. It reduces storage and computing costs and improves scalability by combining stored knowledge representation with knowledge transfer rules adapted to the task. It enhances generalization ability by using associative retrieval to integrate new and old knowledge. Furthermore, it constructs a multi-memory area knowledge circulation mechanism to allow external memory to deeply participate in training to improve interaction efficiency. Overall, it balances the efficiency, stability and generalization of continuous learning, and significantly optimizes the performance of continuous learning of neural networks.

[0007] A method for continuous learning of neural networks, comprising:

[0008] Receive new input samples and write them into the short-term memory area;

[0009] Evaluate the importance of each parameter in the neural network to the learned task and obtain the importance index of each parameter;

[0010] The plasticity of each parameter in training the new input sample is adjusted based on the importance index;

[0011] Retrieve historical knowledge related to the new input sample from long-term memory and / or external memory.

[0012] The retrieved historical knowledge is mixed with the new input sample for replay training;

[0013] According to the preset transfer rules, knowledge transfer is performed between the short-term memory region, the long-term memory region, and the external memory region until the neural network converges.

[0014] Optionally, adjusting the plasticity of each parameter during training of the new input sample based on the importance index includes:

[0015] Based on the importance index, the loss function for training the new input samples is weighted and regularized. The weighted and regularized loss function is as follows:

[0016]

[0017] The actual update step size of each parameter is controlled by a plasticity gating factor, wherein the plasticity gating factor is negatively correlated with the importance index, and the update formula for the parameter is:

[0018]

[0019]

[0020] In the formula, The loss function after weighted regularization. The loss function for the new input sample. The regularization coefficient is . For the i-th parameter, For the value of the i-th parameter, The value of the i-th parameter after training on the old task. For learning rate, Let be the plasticity gating factor of the i-th parameter. Let be the partial derivative of the weighted regularized loss function with respect to the i-th parameter. ← represents the gating adjustment coefficient, and ← is the assignment symbol that assigns the calculated value on the right to the value on the left.

[0021] Optionally, the importance index of the parameter is calculated using the Fisher information matrix method or the gradient accumulation method;

[0022] The formula for calculating the importance index of the parameter using the Fisher information matrix method is as follows:

[0023]

[0024] In the formula, For the i-th parameter, To compute the expectation operator for each sample x in the learned task set, For the value of the i-th parameter, For an input sample of x and an overall model parameter of x, Given the premise that the predicted output is the logarithmic probability of the true label y.

[0025] Optionally, the system can retrieve historical knowledge related to the new input sample, including:

[0026] Extract the feature vector of the new input sample;

[0027] Calculate the similarity between the feature vector and the sample feature vectors stored in the long-term memory region and the external memory region;

[0028] Based on similarity ranking, the most similar historical samples and their corresponding labels are selected and retrieved as historical knowledge related to the new input sample.

[0029] Optionally, the replay training process includes:

[0030] The retrieved historical samples and their corresponding labels are mixed with the new input samples to form a mixed training dataset;

[0031] Construct a total loss function, which is composed of a weighted sum of the loss function of the new input sample and the replay loss term, wherein the replay loss term represents the prediction loss of the neural network for the historical samples, and the weight of the replay loss term in the total loss is adjusted by the replay loss weight.

[0032] The neural network is trained based on the hybrid training dataset and the total loss function.

[0033] Optionally, the preset migration rules include:

[0034] For the first migration sample in the short-term memory region whose access frequency exceeds a preset frequency threshold or whose importance score exceeds a preset importance threshold, the first migration sample is migrated to the long-term memory region, wherein the importance score of the sample is calculated based on the prediction confidence of the sample by the neural network and gradient information.

[0035] For a second migration sample in the long-term memory region that has not been accessed within a set time window, the second migration sample is migrated to the external memory region.

[0036] A neural network continuous learning device, comprising:

[0037] The sample receiving module is used to receive new input samples and write them into the short-term memory area;

[0038] The parameter metrics module is used to evaluate the importance of each parameter in the neural network to the learned task and obtain the importance metrics of each parameter.

[0039] A plasticity adjustment module is used to adjust the plasticity of each parameter when training the new input sample based on the importance index;

[0040] The associative retrieval module is used to retrieve historical knowledge related to the new input sample from long-term memory and / or external memory.

[0041] The replay training module is used to mix the retrieved historical knowledge with the new input sample for replay training.

[0042] The knowledge transfer module is used to perform knowledge transfer between the short-term memory region, the long-term memory region, and the external memory region according to preset transfer rules until the neural network converges.

[0043] A neural network continuous learning device, including a memory and a processor;

[0044] The memory is used to store programs;

[0045] The processor is configured to execute the program to implement the various steps of the neural network continuous learning method as described in any of the preceding claims.

[0046] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the neural network continuous learning method as described in any of the preceding claims.

[0047] A computer program product includes a computer program that, when executed by a processor, performs the steps of the neural network continuous learning method as described in any of the preceding claims.

[0048] As can be seen from the above technical solutions, the neural network continuous learning method and related device provided in this application embodiment receive new input samples and write them into the short-term memory area; evaluate the importance of each parameter of the neural network to the learned task to obtain an importance index; adjust the plasticity of each parameter in the training of new samples based on the index; retrieve historical knowledge related to the new samples from the long-term memory area and / or the external memory area; mix the retrieved historical knowledge with the new samples for replay training; and perform knowledge transfer between the short-term, long-term, and external memory areas according to a preset transfer rule until the neural network converges, so as to achieve continuous learning of the neural network.

[0049] This application addresses the shortcomings of existing technologies in several ways: First, to address the problem of catastrophic forgetting, it dynamically adjusts plasticity through parameter importance assessment, reducing the update rate of key parameters for old tasks and combining this with historical knowledge replay to form a double guarantee, preventing old knowledge from being overwritten and resolving the performance degradation of old tasks. Second, to address insufficient efficiency and scalability, it stores only knowledge representations rather than a large number of original samples, significantly reducing storage overhead. Knowledge transfer rules adapt to task changes, preventing model parameter expansion and reducing computational costs. Third, to address the lack of associative generalization ability, it retrieves relevant historical knowledge through associative retrieval, achieving the integration of new and old knowledge, simulating human brain associative memory, and improving the generalization effect of knowledge transfer. Fourth, through dynamic knowledge transfer across multiple memory intervals, it constructs a complete knowledge flow mechanism, allowing external memory to deeply participate in training and improving interaction efficiency. Overall, it achieves a balance between efficiency, stability, and generalization in continuous learning, significantly optimizing the continuous learning performance of neural networks. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0051] Figure 1This is a flowchart of a neural network continuous learning method disclosed in an embodiment of this application;

[0052] Figure 2 This is a schematic diagram of a neural network continuous learning method disclosed in an embodiment of this application;

[0053] Figure 3 This is a schematic diagram of a neural network continuous learning device disclosed in an embodiment of this application;

[0054] Figure 4 This is a hardware structure block diagram of a neural network continuous learning device disclosed in an embodiment of this application. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] This application can be used in a wide variety of general-purpose or special-purpose computing device environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above devices, etc.

[0057] The following section introduces the solution proposed in this application. The technical solution is as follows, and details are provided below.

[0058] Figure 1 This is a flowchart of a neural network continuous learning method disclosed in an embodiment of this application.

[0059] Figure 2 This is a schematic diagram of a neural network continuous learning method disclosed in an embodiment of this application.

[0060] like Figure 1 and Figure 2 As shown, the method may include:

[0061] Step S1: Receive new input samples and write them into the short-term memory area.

[0062] Specifically, new input samples are acquired for the neural network to learn from. These new input samples can include common data types in the field of artificial intelligence, such as text data in natural language processing, image data in computer vision, and audio data in speech processing, as long as they meet the input format requirements of the neural network. The Short-Term Memory (STM) is a temporary storage unit in a pre-defined multi-level memory buffer. Its core characteristics are limited capacity and support for fast write and read operations. It is mainly used to temporarily store samples and related feature information of the current or recent task. Writing new input samples into the STM allows them to be directly provided as initial data in subsequent steps such as parameter importance evaluation, historical knowledge association retrieval, and replay training, ensuring that the new sample information can be called and processed in a timely manner.

[0063] Step S2: Evaluate the importance of each parameter in the neural network to the learned task and obtain the importance index of each parameter.

[0064] Specifically, this method quantifies the role of each parameter in the neural network (such as model weights and biases) in previously completed learning tasks to determine the criticality of each parameter in retaining old knowledge, ultimately outputting parameter importance indices that can be used for subsequent adjustments. These parameter importance indices can be calculated using the Fisher information matrix method or the gradient accumulation method.

[0065] When using the Fisher information matrix method, the core logic is to evaluate the importance of parameters by analyzing the magnitude of the impact of small changes in parameters on the probability distribution of the model output. Among them, the parameters that have a greater impact on the output results of old tasks have higher importance indices.

[0066] When using the gradient accumulation method, the core logic is to accumulate the gradient changes of the parameter during the training process of all learned tasks. The gradient changes reflect the degree of contribution of the parameter to the learning effect of the old tasks. The more significant the contribution of the parameter, the higher its importance index.

[0067] The importance indicators obtained by the two methods described above can clearly distinguish between old knowledge-related parameters that need to be protected and non-critical parameters that can be flexibly updated in neural networks.

[0068] The formula for calculating the importance index of the parameter using the Fisher information matrix method is as follows:

[0069]

[0070] In the formula, For the i-th parameter, To compute the expectation operator for each sample x in the learned task set, For the value of the i-th parameter, For an input sample of x and an overall model parameter of x, Given the premise that the predicted output is the logarithmic probability of the true label y.

[0071] Step S3: Adjust the plasticity of each parameter when training the new input sample based on the importance index.

[0072] Specifically, plasticity refers to the ease with which neural network parameters are updated during training. Adjusting the parameter importance index obtained in step S2 achieves a dynamic balance between old and new knowledge, thereby mitigating the catastrophic forgetting problem. For parameters with high importance indices (i.e., parameters strongly correlated with old knowledge of the learned task), their plasticity is reduced, specifically by decreasing the update magnitude of these parameters during training on new samples, to avoid old knowledge being overwritten or forgotten due to large parameter adjustments. For parameters with low importance indices (i.e., parameters weakly correlated with old knowledge of the learned task), their plasticity is increased, specifically by allowing these parameters to be updated relatively flexibly during training on new samples, in order to better adapt to new sample features and absorb new knowledge.

[0073] In actual adjustment, the above logic can be achieved by weighting the training loss function, that is, applying stronger constraints to the updates of high-importance parameters and relatively looser constraints to the updates of low-importance parameters, so as to ensure that the model can retain old knowledge and efficiently absorb new information when learning new samples.

[0074] Adjusting the plasticity of each parameter during training of the new input sample based on the importance index includes:

[0075] Based on the importance index, the loss function for training the new input samples is weighted and regularized. The weighted and regularized loss function is as follows:

[0076]

[0077] The actual update step size of each parameter is controlled by a plasticity gating factor, wherein the plasticity gating factor is negatively correlated with the importance index, and the update formula for the parameter is:

[0078]

[0079]

[0080] In the formula, The loss function after weighted regularization. The loss function for the new input sample. The regularization coefficient is . For the i-th parameter, For the value of the i-th parameter, The value of the i-th parameter after training on the old task. For learning rate, Let be the plasticity gating factor of the i-th parameter. Let be the partial derivative of the weighted regularized loss function with respect to the i-th parameter. ← represents the gating adjustment coefficient, and ← is the assignment symbol that assigns the calculated value on the right to the value on the left.

[0081] Weighted regularization of the loss function: To constrain the magnitude of parameter updates during training on new samples, the loss function for training new input samples needs to be weighted and regularized based on parameter importance metrics, constructing a comprehensive loss function that balances learning new tasks with retaining old knowledge. This weighted regularized loss function consists of two parts: one is the base loss function for new input samples, used to measure the model's fit to the features of the new samples, ensuring the model can effectively learn new task knowledge; the other is a regularization term designed based on parameter importance metrics, used to impose differentiated constraints on parameter updates.

[0082] The core logic of the regularization term is as follows: The current value of each parameter is compared with its historical best value after training on the old task. The difference between the two is calculated, and the difference is weighted using the parameter's importance index. The higher the parameter's importance index (i.e., the more critical it is to the old task), the larger its corresponding weighting coefficient, meaning the difference between the parameter and the optimal value of the old task will be more strictly constrained. Simultaneously, the influence strength of the regularization term can be adjusted overall by a preset regularization coefficient, avoiding excessively strong constraints that prevent the absorption of new sample knowledge, or excessively weak constraints that lead to the loss of old knowledge. Ultimately, this achieves differentiated update control with strong constraints on key parameters and weak constraints on non-key parameters.

[0083] Plasticity gating factor controls the actual update step size of parameters: Based on the weighted regularized loss function, a plasticity gating factor is further introduced. This factor directly controls the actual update step size of each parameter, enabling fine-grained plasticity adjustment at the parameter level. The plasticity gating factor is negatively correlated with the parameter importance index: the higher the parameter importance index, the smaller the corresponding plasticity gating factor value; conversely, the lower the parameter importance index, the larger the corresponding plasticity gating factor value.

[0084] In the specific update process, the partial derivative of the weighted regularized loss function with respect to each parameter is first calculated. This partial derivative reflects the degree of influence of small changes in the parameter on the overall loss. Then, this partial derivative is multiplied by a preset learning rate (controlling the overall update magnitude) and the plasticity gating factor of the corresponding parameter to obtain the actual update step size for that parameter. Parameters with high importance have smaller gating factors, resulting in a compressed actual update step size, thus reducing interference with old task knowledge. Parameters with low importance have larger gating factors, allowing for more flexible actual update step sizes that better adapt to new sample features. Finally, the current parameter value is subtracted from the calculated actual update step size to complete one parameter update, achieving the goal of stable key parameters and flexible plasticity adjustment for non-key parameters.

[0085] Step S4: Retrieve historical knowledge related to the new input sample from long-term memory and / or external memory.

[0086] Specifically, the Long Memory Region (LTM) is a consolidation storage unit within a multi-level memory buffer. It primarily stores selected consolidation knowledge crucial to older tasks, such as core sample features of older tasks and parameter correlation information demonstrating the model's superior performance in those tasks. It features large capacity and stable storage. The External Memory Region (EM) is an extended storage unit that can be linked to external databases, vector storage, and other devices. It primarily stores large-scale, less frequently used but potentially useful auxiliary historical knowledge, such as a complete sample library of older tasks and feature data from edge scenarios.

[0087] The associative retrieval process follows a feature matching logic. First, features are extracted from the new input sample written to the short-term memory region in step S1. This can be achieved through feature extraction layers in a neural network, such as fully connected layers or convolutional layers, to obtain the feature representation of the new sample. Then, this feature representation is compared with the historical knowledge features stored in the long-term memory region and the external memory region to filter out the historical knowledge with the highest correlation to the new sample's features. For example, if the new input sample is "images of cats," relevant historical knowledge such as "general features of animal images," "feline-specific features," and "sample data of cats in different poses" can be retrieved from the long-term memory region. The retrieval scope can be flexibly selected according to the needs of the new task. It can retrieve core knowledge only from the long-term memory region or retrieve comprehensive knowledge from both the long-term and external memory regions simultaneously to ensure that the retrieved historical knowledge can serve as effective learning material for the new sample.

[0088] Step S5: Mix the retrieved historical knowledge with the new input sample and perform replay training.

[0089] Specifically, historical knowledge refers to the old sample task data and corresponding labels retrieved in step S4 that are related to the new input sample. The mixed operation must follow the preset ratio rules (such as matching the ratio of historical knowledge to new input samples according to the complexity of the new task) to form a comprehensive training dataset containing both new and old knowledge, so as to avoid forgetting old knowledge caused by training new samples alone.

[0090] The core process of the replay training is as follows: The mixed training dataset is input into the neural network, allowing the model to learn both new knowledge from new input samples and retrieved old knowledge simultaneously. During training, the model optimizes both its ability to fit the features of new samples and its ability to retain old knowledge. By alternately learning from new and old data, the model strengthens the association between new and old knowledge. Simultaneously, the training process comprehensively considers the learning loss from new tasks and the loss from retaining old knowledge, ensuring that the model adapts to new samples without losing previously acquired knowledge. Ultimately, this achieves simultaneous progress in absorbing new knowledge and consolidating old knowledge, enhancing the model's knowledge transfer capabilities.

[0091] Step S6: According to the preset transfer rules, perform knowledge transfer between the short-term memory region, the long-term memory region, and the external memory region until the neural network converges.

[0092] Specifically, the preset transfer rules are dynamic scheduling rules based on the functional positioning of multi-level memory buffers to achieve hierarchical management, efficient storage and on-demand retrieval of knowledge. The neural network convergence means that the performance of the model on both the new input sample learning task and the already learned task tends to be stable (such as the prediction accuracy no longer changing significantly and the loss value stabilizing at a low level).

[0093] The preset migration rules may include:

[0094] For the first migration sample in the short-term memory region whose access frequency exceeds a preset frequency threshold or whose importance score exceeds a preset importance threshold, the first migration sample is migrated to the long-term memory region, wherein the importance score of the sample is calculated based on the prediction confidence of the sample by the neural network and gradient information.

[0095] For a second migration sample in the long-term memory region that has not been accessed within a set time window, the second migration sample is migrated to the external memory region.

[0096] The specific migration logic is as follows:

[0097] Migration from Short-Term Memory (STM) to Long-Term Memory (LTM): If a new input sample or related feature stored in the short-term memory is accessed multiple times during training (access frequency exceeds a preset threshold), or is determined to have important learning value in the neural network learning process (importance score exceeds a preset threshold), then this part of the sample is migrated from the short-term memory to the long-term memory to consolidate the storage of important knowledge and avoid the loss of important information due to the capacity limitation of the short-term memory.

[0098] Migration from Long Memory (LTM) to External Memory (EM): If the historical knowledge stored in the long memory is not retrieved or called within a preset time window (i.e. it has not been used for training / reasoning for a long time), then this part of the knowledge is migrated from the long memory to the external memory, releasing the storage resources of the long memory and ensuring that the long memory always stores high-value, high-frequency core knowledge.

[0099] The transfer process is executed cyclically: The above knowledge transfer operation is carried out cyclically along with the training process of the neural network. After each transfer, the model performance is re-evaluated until the model converges. At this time, the neural network has not only fully learned the knowledge of the new input samples, but also achieved efficient management of new and old knowledge through memory transfer, and finally completed a complete continuous learning process, ensuring that the neural network has the ability to learn continuously.

[0100] As can be seen from the above technical solutions, the neural network continuous learning method and related device provided in this application embodiment receive new input samples and write them into the short-term memory area; evaluate the importance of each parameter of the neural network to the learned task to obtain an importance index; adjust the plasticity of each parameter in the training of new samples based on the index; retrieve historical knowledge related to the new samples from the long-term memory area and / or the external memory area; mix the retrieved historical knowledge with the new samples for replay training; and perform knowledge transfer between the short-term, long-term, and external memory areas according to a preset transfer rule until the neural network converges, so as to achieve continuous learning of the neural network.

[0101] This application addresses the shortcomings of existing technologies in several ways: First, to address the problem of catastrophic forgetting, it dynamically adjusts plasticity through parameter importance assessment, reducing the update rate of key parameters for old tasks and combining this with historical knowledge replay to form a double guarantee, preventing old knowledge from being overwritten and resolving the performance degradation of old tasks. Second, to address insufficient efficiency and scalability, it stores only knowledge representations rather than a large number of original samples, significantly reducing storage overhead. Knowledge transfer rules adapt to task changes, preventing model parameter expansion and reducing computational costs. Third, to address the lack of associative generalization ability, it retrieves relevant historical knowledge through associative retrieval, achieving the integration of new and old knowledge, simulating human brain associative memory, and improving the generalization effect of knowledge transfer. Fourth, through dynamic knowledge transfer across multiple memory intervals, it constructs a complete knowledge flow mechanism, allowing external memory to deeply participate in training and improving interaction efficiency. Overall, it achieves a balance between efficiency, stability, and generalization in continuous learning, significantly optimizing the continuous learning performance of neural networks.

[0102] In some embodiments of this application, the process of associative retrieval of historical knowledge related to the new input sample in step S4 is described, which may specifically include:

[0103] Step S41: Extract the feature vector of the new input sample.

[0104] Specifically, new input samples are transformed into standardized feature representations that can be used for subsequent similarity calculations, providing a unified basis for comparison in associative retrieval. The extraction process relies on a pre-defined feature extraction function or feature extraction layer in the neural network. The final output feature vector must have uniform dimensionality and condensed information, accurately reflecting the core attributes of the new input sample, ensuring the effectiveness and reliability of subsequent similarity calculations with historical sample feature vectors.

[0105] Step S42: Calculate the similarity between the feature vector and the sample feature vectors stored in the long-term memory region and the external memory region.

[0106] Specifically, using the extracted feature vector of the new input sample as a benchmark, a correlation metric is calculated between it and the feature vectors of historical samples pre-stored in the Long Memory (LTM) and External Memory (EM) regions. The LTM stores core historical sample feature vectors strongly correlated with the old task, while the EM stores larger, less frequently used but potentially valuable auxiliary historical sample feature vectors. The calculation scope can be flexibly selected according to the needs of the new task; it can calculate similarity only with the feature vectors in the LTM region (suitable for scenarios requiring priority access to core old knowledge), or it can calculate similarity with the feature vectors in both the LTM and EM regions simultaneously (suitable for scenarios requiring comprehensive retrieval of related old knowledge). Similarity calculation measures the degree of correlation between two sets of feature vectors in the feature space, which can be achieved through methods such as distance or vector correlation. The final output is the similarity value between the new input sample feature vector and each historical sample feature vector.

[0107] Step S43: Sort by similarity, filter and retrieve the most similar historical samples and their corresponding labels as historical knowledge related to the new input sample.

[0108] Specifically, all calculated similarity values ​​are sorted from highest to lowest, meaning the feature vectors of historical samples with higher similarity values ​​are ranked higher, indicating a stronger association with the new input sample. Then, according to preset filtering rules (such as a preset fixed number threshold or a preset minimum similarity threshold), the most similar historical samples at the top of the ranking are selected, or all historical samples with similarity values ​​greater than a preset value are selected. Specific rules can be flexibly adjusted based on the complexity of the new task and the storage capacity of the memory area. Simultaneously, the label information corresponding to these selected historical samples needs to be extracted (labels are the category, attribute, or output result of the historical sample in the old task, such as the "cat / dog" category label for image samples, the "sentiment tendency" label for text samples, etc.). Finally, the most similar historical samples and their corresponding labels are used together as historical knowledge related to the new input sample, providing old knowledge material for the mixed playback training of new and old knowledge in subsequent step S5. This ensures that the model can assist in learning new samples by associating with old knowledge, improving knowledge transfer and generalization capabilities.

[0109] Based on this, the replay training process in step S5 will be described, which may include:

[0110] Step S51: Mix the retrieved historical samples and their corresponding labels with the new input samples to form a mixed training dataset.

[0111] Specifically, the retrieved historical samples and their corresponding labels are mixed with the new input samples. The mixing follows the principle of balancing the proportion of old and new knowledge, avoiding situations where an excessive proportion of new samples leads to the forgetting of old knowledge, or an excessive proportion of historical samples hinders the absorption of new knowledge. Simultaneously, the mixed samples undergo format standardization to ensure that the feature dimensions, data types, and label formats of the historical samples and the new input samples are completely consistent. This ultimately forms a mixed training dataset that can be directly input into the neural network for training, providing data support for subsequent synchronous learning of new and old knowledge.

[0112] Step S52: Construct a total loss function, which is composed of a weighted sum of the loss function of the new input sample and the replay loss term, wherein the replay loss term represents the prediction loss of the neural network for the historical samples, and the weight of the replay loss term in the total loss is adjusted by the replay loss weight.

[0113] Specifically, constructing a total loss function allows the neural network to simultaneously focus on learning from new input samples and remembering historical samples during training, avoiding the loss of old knowledge caused by solely optimizing the new task.

[0114] The loss function for new input samples measures how well the model fits the new task knowledge. Its type needs to be selected based on the new task scenario. The smaller the value of this loss function, the more fully the model has learned the features of the new input samples. The replay loss term measures the model's retention of historical samples. By comparing the model's predicted output for historical samples with the true labels of those samples, it quantifies the degree to which the model has forgotten old knowledge. The smaller the value of the replay loss term, the stronger the model's retention of old knowledge.

[0115] The replay loss weight is a hyperparameter used to balance the importance of both old and new knowledge. It needs to be dynamically adjusted according to task requirements: if the old task knowledge is crucial for subsequent learning (such as historical case knowledge in medical imaging diagnosis), the replay loss weight can be appropriately increased to strengthen the constraint of memorizing old knowledge; if the new task is the core learning objective (such as learning new data in emerging fields), the replay loss weight can be appropriately decreased to prioritize the absorption of new knowledge. The final total loss function achieves a dynamic balance between the learning of old and new knowledge through weighted fusion, providing a clear optimization objective for subsequent training.

[0116] Step S53: Train the neural network based on the hybrid training dataset and the total loss function.

[0117] Specifically, the training process adopts an end-to-end optimization mode, inputting the mixed training dataset into the neural network in preset batches, aiming to minimize the total loss function, and iteratively updating the model parameters through gradient descent and its variant optimizers. At the same time, combined with the dynamic synaptic plasticity adjustment mechanism of this application (i.e., the parameter importance and plasticity constraints determined in steps S2-S3), continuous learning effect is achieved.

[0118] The core operations during the training process include:

[0119] Batch Iterative Training: The samples in the mixed training dataset are divided into batches. Each iteration inputs a batch of samples (including new input samples and historical samples). The model performs forward propagation on the batch of samples, calculates the loss of the new input samples and the replay loss respectively, and then obtains the total loss value of the batch according to the total loss function.

[0120] Gradient calculation and parameter update: The gradient of each parameter is calculated by backpropagation based on the total loss value. Combined with the plasticity gating factor determined in step S3 (the gradient of highly important parameters is appropriately suppressed, and the gradient of less important parameters is updated normally), the model parameters are adjusted by the optimizer to ensure that key old knowledge parameters are not over-modified, and non-key parameters can be flexibly adapted to new knowledge.

[0121] Training convergence monitoring: After each iteration, the changes in the total loss value, the loss value of the new input sample, and the replay loss value are monitored in real time. At the same time, it can help monitor the performance indicators of the model on the new task validation set and the old task validation set. When the total loss value stabilizes below the preset threshold and the performance indicators of the new task and the old task no longer improve significantly, the model training is determined to be converged and the replay training ends.

[0122] Through the above training process, the neural network can absorb knowledge from new input samples while continuously consolidating old knowledge corresponding to historical samples, effectively alleviating the problem of catastrophic forgetting and significantly improving the model's continuous learning ability and knowledge transfer ability.

[0123] The following describes a neural network continuous learning device provided in the embodiments of this application. The neural network continuous learning device described below can be referred to in correspondence with the neural network continuous learning method described above.

[0124] See Figure 3 , Figure 3 This is a schematic diagram of a neural network continuous learning device disclosed in an embodiment of this application.

[0125] like Figure 3 As shown, the neural network continuous learning device may include:

[0126] The sample receiving module is used to receive new input samples and write them into the short-term memory area;

[0127] The parameter metrics module is used to evaluate the importance of each parameter in the neural network to the learned task and obtain the importance metrics of each parameter.

[0128] A plasticity adjustment module is used to adjust the plasticity of each parameter when training the new input sample based on the importance index;

[0129] The associative retrieval module is used to retrieve historical knowledge related to the new input sample from long-term memory and / or external memory.

[0130] The replay training module is used to mix the retrieved historical knowledge with the new input sample for replay training.

[0131] The knowledge transfer module is used to perform knowledge transfer between the short-term memory region, the long-term memory region, and the external memory region according to preset transfer rules until the neural network converges.

[0132] As can be seen from the above technical solutions, the neural network continuous learning method and related device provided in this application embodiment receive new input samples and write them into the short-term memory area; evaluate the importance of each parameter of the neural network to the learned task to obtain an importance index; adjust the plasticity of each parameter in the training of new samples based on the index; retrieve historical knowledge related to the new samples from the long-term memory area and / or the external memory area; mix the retrieved historical knowledge with the new samples for replay training; and perform knowledge transfer between the short-term, long-term, and external memory areas according to a preset transfer rule until the neural network converges, so as to achieve continuous learning of the neural network.

[0133] This application addresses the shortcomings of existing technologies in several ways: First, to address the problem of catastrophic forgetting, it dynamically adjusts plasticity through parameter importance assessment, reducing the update rate of key parameters for old tasks and combining this with historical knowledge replay to form a double guarantee, preventing old knowledge from being overwritten and resolving the performance degradation of old tasks. Second, to address insufficient efficiency and scalability, it stores only knowledge representations rather than a large number of original samples, significantly reducing storage overhead. Knowledge transfer rules adapt to task changes, preventing model parameter expansion and reducing computational costs. Third, to address the lack of associative generalization ability, it retrieves relevant historical knowledge through associative retrieval, achieving the integration of new and old knowledge, simulating human brain associative memory, and improving the generalization effect of knowledge transfer. Fourth, through dynamic knowledge transfer across multiple memory intervals, it constructs a complete knowledge flow mechanism, allowing external memory to deeply participate in training and improving interaction efficiency. Overall, it achieves a balance between efficiency, stability, and generalization in continuous learning, significantly optimizing the continuous learning performance of neural networks.

[0134] Optionally, adjusting the plasticity of each parameter during training of the new input sample based on the importance index includes:

[0135] Based on the importance index, the loss function for training the new input samples is weighted and regularized. The weighted and regularized loss function is as follows:

[0136]

[0137] The actual update step size of each parameter is controlled by a plasticity gating factor, wherein the plasticity gating factor is negatively correlated with the importance index, and the update formula for the parameter is:

[0138]

[0139]

[0140] In the formula, The loss function after weighted regularization. The loss function for the new input sample. The regularization coefficient is . For the i-th parameter, For the value of the i-th parameter, The value of the i-th parameter after training on the old task. For learning rate, Let be the plasticity gating factor of the i-th parameter. Let be the partial derivative of the weighted regularized loss function with respect to the i-th parameter. ← represents the gating adjustment coefficient, and ← is the assignment symbol that assigns the calculated value on the right to the value on the left.

[0141] Optionally, the importance index of the parameter is calculated using the Fisher information matrix method or the gradient accumulation method;

[0142] The formula for calculating the importance index of the parameter using the Fisher information matrix method is as follows:

[0143]

[0144] In the formula, For the i-th parameter, To compute the expectation operator for each sample x in the learned task set, For the value of the i-th parameter, For an input sample of x and an overall model parameter of x, Given the premise that the predicted output is the logarithmic probability of the true label y.

[0145] Optionally, the system can retrieve historical knowledge related to the new input sample, including:

[0146] Extract the feature vector of the new input sample;

[0147] Calculate the similarity between the feature vector and the sample feature vectors stored in the long-term memory region and the external memory region;

[0148] Based on similarity ranking, the most similar historical samples and their corresponding labels are selected and retrieved as historical knowledge related to the new input sample.

[0149] Optionally, the replay training process includes:

[0150] The retrieved historical samples and their corresponding labels are mixed with the new input samples to form a mixed training dataset;

[0151] Construct a total loss function, which is composed of a weighted sum of the loss function of the new input sample and the replay loss term, wherein the replay loss term represents the prediction loss of the neural network for the historical samples, and the weight of the replay loss term in the total loss is adjusted by the replay loss weight.

[0152] The neural network is trained based on the hybrid training dataset and the total loss function.

[0153] Optionally, the preset migration rules include:

[0154] For the first migration sample in the short-term memory region whose access frequency exceeds a preset frequency threshold or whose importance score exceeds a preset importance threshold, the first migration sample is migrated to the long-term memory region, wherein the importance score of the sample is calculated based on the prediction confidence of the sample by the neural network and gradient information.

[0155] For a second migration sample in the long-term memory region that has not been accessed within a set time window, the second migration sample is migrated to the external memory region.

[0156] The neural network continuous learning device provided in this application embodiment can be applied to neural network continuous learning equipment. Figure 4 The hardware structure block diagram of the neural network continuous learning device is shown, with reference to... Figure 4 The hardware structure of a neural network continuous learning device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4.

[0157] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0158] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0159] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0160] The memory stores a program, which the processor can call. The program is used for:

[0161] Receive new input samples and write them into the short-term memory area;

[0162] Evaluate the importance of each parameter in the neural network to the learned task and obtain the importance index of each parameter;

[0163] The plasticity of each parameter in training the new input sample is adjusted based on the importance index;

[0164] Retrieve historical knowledge related to the new input sample from long-term memory and / or external memory.

[0165] The retrieved historical knowledge is mixed with the new input sample for replay training;

[0166] According to the preset transfer rules, knowledge transfer is performed between the short-term memory region, the long-term memory region, and the external memory region until the neural network converges.

[0167] Optionally, the refined and extended functions of the program can be referred to the above description.

[0168] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:

[0169] Receive new input samples and write them into the short-term memory area;

[0170] Evaluate the importance of each parameter in the neural network to the learned task and obtain the importance index of each parameter;

[0171] The plasticity of each parameter in training the new input sample is adjusted based on the importance index;

[0172] Retrieve historical knowledge related to the new input sample from long-term memory and / or external memory.

[0173] The retrieved historical knowledge is mixed with the new input sample for replay training;

[0174] According to the preset transfer rules, knowledge transfer is performed between the short-term memory region, the long-term memory region, and the external memory region until the neural network converges.

[0175] Optionally, the refined and extended functions of the program can be referred to the above description.

[0176] This application also provides a computer program product, including a computer program, wherein the computer program is executed by a processor using the following method:

[0177] Receive new input samples and write them into the short-term memory area;

[0178] Evaluate the importance of each parameter in the neural network to the learned task and obtain the importance index of each parameter;

[0179] The plasticity of each parameter in training the new input sample is adjusted based on the importance index;

[0180] Retrieve historical knowledge related to the new input sample from long-term memory and / or external memory.

[0181] The retrieved historical knowledge is mixed with the new input sample for replay training;

[0182] According to the preset transfer rules, knowledge transfer is performed between the short-term memory region, the long-term memory region, and the external memory region until the neural network converges.

[0183] Optionally, the refined and extended functions of the program can be referred to the above description.

[0184] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0185] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0186] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for continuous learning of neural networks, characterized in that, include: Receive new input samples and write them into the short-term memory area, wherein the new input samples include text data in a natural language processing scenario, image data in a computer vision scenario, or audio data in a speech processing scenario; Evaluate the importance of each parameter in the neural network to the learned task and obtain the importance index of each parameter; The plasticity of each parameter in training the new input sample is adjusted based on the importance index; Retrieve historical knowledge related to the new input sample from long-term memory and / or external memory. The retrieved historical knowledge is mixed with the new input sample for replay training; According to the preset transfer rules, knowledge transfer is performed between the short-term memory region, the long-term memory region, and the external memory region until the neural network converges; The preset migration rules include: For the first migration sample in the short-term memory region whose access frequency exceeds a preset frequency threshold or whose importance score exceeds a preset importance threshold, the first migration sample is migrated to the long-term memory region, wherein the importance score of the sample is calculated based on the prediction confidence of the sample by the neural network and gradient information. For a second migration sample in the long-term memory region that has not been accessed within a set time window, the second migration sample is migrated to the external memory region.

2. The method according to claim 1, characterized in that, Adjusting the plasticity of each parameter during training of the new input sample based on the importance index includes: Based on the importance index, the loss function for training the new input samples is weighted and regularized. The weighted and regularized loss function is as follows: The actual update step size of each parameter is controlled by a plasticity gating factor, wherein the plasticity gating factor is negatively correlated with the importance index, and the update formula for the parameter is: In the formula, The loss function after weighted regularization. The loss function for the new input sample. The regularization coefficient is . For the first i The importance of each parameter, For the first i Parameter value, The first training after the old task i Parameter value, For learning rate, For the first i The plasticity gating factor of each parameter, For the weighted regularized loss function on the th i Partial derivatives of each parameter, ← represents the gating adjustment coefficient, and ← is the assignment symbol that assigns the calculated value on the right to the value on the left.

3. The method according to claim 1, characterized in that, The importance index of the parameter is obtained by Fisher's information matrix method or gradient accumulation method; The formula for calculating the importance index of the parameter using the Fisher information matrix method is as follows: In the formula, For the first i The importance of each parameter, To traverse each sample in the learned task set x The expected value operator is calculated. For the first i Parameter value, For the input sample is x The overall parameters of the model are Under the premise that the predicted output is the true label y The logarithmic probability.

4. The method according to claim 1, characterized in that, The system retrieves historical knowledge related to the new input sample, including: Extract the feature vector of the new input sample; Calculate the similarity between the feature vector and the sample feature vectors stored in the long-term memory region and the external memory region; Based on similarity ranking, the most similar historical samples and their corresponding labels are selected and retrieved as historical knowledge related to the new input sample.

5. The method according to claim 4, characterized in that, The replay training process includes: The retrieved historical samples and their corresponding labels are mixed with the new input samples to form a mixed training dataset; Construct a total loss function, which is composed of a weighted sum of the loss function of the new input sample and the replay loss term, wherein the replay loss term represents the prediction loss of the neural network for the historical samples, and the weight of the replay loss term in the total loss is adjusted by the replay loss weight. The neural network is trained based on the hybrid training dataset and the total loss function.

6. A neural network continuous learning device, characterized in that, include: The sample receiving module is used to receive new input samples and write them into the short-term memory area, wherein the new input samples include text data in natural language processing scenarios, image data in computer vision scenarios, or audio data in speech processing scenarios. The parameter metrics module is used to evaluate the importance of each parameter in the neural network to the learned task and obtain the importance metrics of each parameter. A plasticity adjustment module is used to adjust the plasticity of each parameter when training the new input sample based on the importance index; The associative retrieval module is used to retrieve historical knowledge related to the new input sample from long-term memory and / or external memory. The replay training module is used to mix the retrieved historical knowledge with the new input sample for replay training. The knowledge transfer module is used to perform knowledge transfer between the short-term memory region, the long-term memory region, and the external memory region according to preset transfer rules until the neural network converges. The preset migration rules include: For the first migration sample in the short-term memory region whose access frequency exceeds a preset frequency threshold or whose importance score exceeds a preset importance threshold, the first migration sample is migrated to the long-term memory region, wherein the importance score of the sample is calculated based on the prediction confidence of the sample by the neural network and gradient information. For a second migration sample in the long-term memory region that has not been accessed within a set time window, the second migration sample is migrated to the external memory region.

7. A neural network continuous learning device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the various steps of the neural network continuous learning method as described in any one of claims 1-5.

8. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the various steps of the neural network continuous learning method as described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor to perform the steps of the neural network continuous learning method as described in any one of claims 1-5.

Images