Decoupling playback and memory wakeup method for online continuous learning
By decoupling the replay and memory wake-up methods, initializing the model parameters and replay buffer, configuring an independent optimizer, combining cross-entropy and mean square error loss to optimize model parameters, and dynamically adjusting the number of review samples, we solve the problems of recent bias and gradient conflict in online continuous learning, achieve efficient learning and memory accumulation, and improve the adaptability and stability of the model.
Patent Information
- Application Number
- CN202510887912.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-30
- Publication Date
- 2025-10-17
AI Technical Summary
Existing replay continuous learning methods suffer from recent bias and gradient conflict problems in online continuous learning. They are unable to reasonably allocate computing resources, find it difficult to maintain memory of old tasks and adapt to changes in different task characteristics and data distribution, resulting in unstable performance.
A decoupled replay and memory recall method is adopted to initialize model parameters and replay buffer. Independent cognitive flow optimizer and review flow optimizer are configured. Model parameters are jointly optimized through cross entropy loss and mean square error loss. Combined with the task-adaptive incremental review strategy and plasticity sampling strategy, the number of review samples and resource allocation are dynamically adjusted.
The model achieves decoupling optimization between new task learning and old task review, alleviates catastrophic forgetting, improves the adaptability and stability of the model, has good task transition capabilities and knowledge retention potential, and enhances the scalability and generalization capabilities of the model.
Smart Images
Figure CN120803767A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of replay-based continual learning, and in particular to a decoupled replay and memory wake-up method for online continual learning. BACKGROUND
[0002] Replay is a technique specially designed for online continual learning scenarios, effectively managing and utilizing historical data to support long-term learning processes. Therefore, how to improve the intelligence level and security of the replay-based method using advanced technical means has become one of the current problems to be solved.
[0003] In the field of online continual learning, traditional replay-based methods can better solve the problem of forgetting previously learned knowledge when the model learns new tasks in a continuous learning scenario, enabling machine learning models to perform in dynamic environments.
[0004] However, there are inherent limitations in solving the core challenges of incremental learning, including recent bias and gradient conflict problems. Moreover, when dealing with multiple tasks, it is difficult to allocate computational resources reasonably to ensure effective learning of new tasks while maintaining memory of old tasks. At the same time, existing replay continual learning methods lack sufficient flexibility and adaptability to ensure consistent and stable performance in the face of different task characteristics and changing data distribution. SUMMARY
[0005] In view of the above existing problems, the present application is proposed.
[0006] Therefore, the present application provides a decoupled replay and memory wake-up method for online continual learning to solve the problem that traditional replay-based methods still have inherent limitations, including recent bias and gradient conflict problems, when used in continuous learning scenarios. Moreover, when dealing with multiple tasks, it is difficult to allocate computational resources reasonably to ensure effective learning of new tasks while maintaining memory of old tasks. At the same time, existing replay continual learning methods lack sufficient flexibility and adaptability to ensure consistent and stable performance in the face of different task characteristics and changing data distribution.
[0007] To solve the above technical problems, the present application provides the following technical solutions:
[0008] In a first aspect, the present application provides a decoupled replay and memory wake-up method for online continual learning, comprising:
[0009] S1, initializing model parameters and replay buffer, loading a pre-trained backbone network, and configuring independent cognitive flow optimizer and review flow optimizer;
[0010] S2, in the cognitive stream, the current new task data is used to train the model, the model parameters are optimized through the cross entropy loss function, and the current task samples and their output logits are stored in the replay buffer;
[0011] S3, based on the task adaptive incremental review strategy, the total amount of review samples required by the current task is dynamically calculated, and the total amount of review samples increases linearly with the increase of the number of tasks;
[0012] S4, in the review stream, sample historical samples and their logits from the replay buffer, and jointly optimize the model parameters using cross entropy loss and mean square error loss to reactivate the passivated classifier weights;
[0013] Repeat steps S2 to S4 until all tasks of online continuous learning are completed.
[0014] As a preferred scheme of the decoupled replay and memory awakening method for online continuous learning, wherein: the learning rate of the cognitive stream optimizer in step S1 is set to 0.01, the learning rate of the review stream optimizer is set to 0.005, and the optimizer type is stochastic gradient descent (SGD).
[0015] As a preferred scheme of the decoupled replay and memory awakening method for online continuous learning, wherein: the specific implementation of the task adaptive incremental review strategy in step S3 includes:
[0016] Initialize the review ratio r init And the incremental review ratio r incre ;
[0017] For the first task T1, the initial review sample number
[0018] For the i-th task T i , update the total amount of review samples according to its data amount N1
[0019] Wherein, And The total amount of review samples of the first task and the i-th task.
[0020] As a preferred scheme of the decoupled replay and memory awakening method for online continuous learning, wherein: the sampling process in step S4 adopts a plasticity sampling strategy, which specifically includes:
[0021] Define the plasticity sampling probability a;
[0022] For the t-th sampling, calculate the number of new task samples St , the expression is:
[0023]
[0024] where B M is the total sample size of single review stream sampling, S t is the number of samples allocated to new tasks at the t-th sampling;
[0025] When t M , calculate according to the formula ;
[0026] When t M , it is fixed to 1 to avoid new task samples being completely ignored.
[0027] As a preferred solution of the decoupled playback and memory awakening method for online continuous learning described in the application, wherein: the calculation formula of the mean square error loss in step S4 is:
[0028] L mse = γ·MSE(f θ (x old ), l old )
[0029] Where γ is the knowledge distillation loss weight, l old = f θ (x new ) is the historical sample output logits saved by the cognitive stream.
[0030] As a preferred solution of the decoupled playback and memory awakening method for online continuous learning described in the application, wherein: the playback buffer in step S2 uses Reservoir Sampling strategy to manage samples, specifically including:
[0031] Set the maximum capacity of the buffer M;
[0032] For each new task sample (x new , y new ), replace a random sample in the buffer with a probability , where N is the total number of samples currently processed.
[0033] As a preferred solution of the decoupled playback and memory awakening method for online continuous learning described in the application, wherein: the loss function of the cognitive stream in step S2 is:
[0034]
[0035] Where L CE is the cross-entropy loss function, T iThe data distribution of the current task.
[0036] As a preferred solution of the decoupled playback and memory awakening method for online continuous learning described in the present invention, the joint optimization of the review flow in step S4 includes:
[0037] The cross entropy loss and mean square error loss functions are expressed as:
[0038]
[0039] L mse =γ·MSE(f θ (x old ),l old );
[0040] Where (x old ,y old ,l old ) are historical samples sampled in the playback buffer and their corresponding logits.
[0041] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the decoupled playback and memory wake-up method for online continuous learning as described in the first aspect of the present invention is implemented.
[0042] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the decoupled playback and memory wake-up method for online continuous learning as described in the first aspect of the present invention.
[0043] The beneficial effects of the present invention are as follows: by initializing model parameters and a replay buffer, loading a pre-trained backbone network, and configuring two independent optimizers, the cognitive stream and the review stream, a decoupling optimization mechanism is implemented between the model's new task learning and the review of old tasks, thereby alleviating catastrophic forgetting and improving the model's adaptability, and having stronger learning stability and generalization ability; by using current task data for training and saving key samples and logits to the replay buffer, the model can quickly acquire and retain new knowledge, achieve the effect of efficient learning and memory accumulation, and have good task transition capabilities and knowledge retention potential; by dynamically adjusting the number of review samples through a task-adaptive incremental review strategy, the intelligent allocation of review resources is achieved, and the effect of balancing new and old task learning resources and maintaining overall performance stability is achieved; the present invention has good scalability and adjustability; by jointly using cross entropy loss and mean square error loss to retrain historical samples, the effective awakening and integration of old knowledge is achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0045] Figure 1 This is a flowchart of the decoupled playback and memory awakening method for online continuous learning in Example 1. DETAILED DESCRIPTION
[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0047] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0048] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.
[0049] Example 1, reference Figure 1 , which is the first embodiment of the present invention, provides a decoupled playback and memory awakening method for online continuous learning, comprising the following steps:
[0050] S1. Initialize model parameters and playback buffer, load the pre-trained backbone network, and configure independent cognitive flow optimizers and retrospective flow optimizers;
[0051] Furthermore, the learning rate of the cognitive flow optimizer is set to 0.01, the learning rate of the retrospective flow optimizer is set to 0.005, and the optimizer type is stochastic gradient descent SGD;
[0052] It should be noted that S1 is configured with two independent optimizers: the cognitive flow optimizer and the retrospective flow optimizer, which are used for parameter updates during the new task learning phase and the historical knowledge review phase, respectively. By setting the learning rate of the cognitive flow optimizer to 0.01 and the learning rate of the retrospective flow optimizer to 0.005, and using the SGD optimizer, we can achieve fine-grained control of the learning rate at different stages. While ensuring that the model quickly adapts to new tasks, it avoids excessive perturbations to existing knowledge during the retrospective process, thereby improving overall training stability.
[0053] S2. In the cognitive stream, the model is trained using the current new task data, the model parameters are optimized using the cross-entropy loss function, and the samples of the current task and their output logits are stored in the playback buffer;
[0054] Furthermore, the playback buffer uses the Reservoir Sampling strategy to manage samples, including:
[0055] Set the maximum capacity of the buffer M;
[0056] For each new task sample (x new ,y new ), with probability Replace the random sample in the buffer, where N is the total number of samples currently processed;
[0057] The loss function of cognitive flow is:
[0058]
[0059] Among them, L CE is the cross entropy loss function, T i The data distribution of the current task;
[0060] It should be noted that the playback buffer is managed using the Reservoir Sampling strategy, the core of which is to dynamically maintain a representative historical sample set under the premise of limited buffer capacity. Specifically, when processing a new sample, a certain probability is used to determine whether to retain it in the buffer and replace an old sample, thereby ensuring that the buffer always contains data distribution covering multiple tasks. This strategy effectively solves the problem of severe loss of early task information in traditional fixed window methods, providing high-quality historical data support for memory consolidation in subsequent continuous learning; the cross-entropy loss function used by cognitive flow is one of the most commonly used supervised learning objective functions in current classification tasks, which has the advantages of efficient gradient calculation and certain robustness to class imbalance problems. In the formula, represents the real data distribution of the current task, by minimizing the loss function, the model can be guided to accurately fit the characteristics of the current task, while outputting the corresponding logits for the subsequent review flow, forming a complete knowledge distillation mechanism, and enhancing the memory consistency of the model for historical tasks.
[0061] S3, based on the task adaptive incremental review strategy, dynamically calculating the total amount of review samples required for the current task, the total amount of review samples linearly increases with the increase of the number of tasks;
[0062] Further, the specific implementation of the task adaptive incremental review strategy includes:
[0063] Initialize the review ratio r init and the incremental review ratio r incre ;
[0064] For the first task T1, the initial review sample number
[0065] For the i-th task T i , update the total amount of review samples according to its data amount N1
[0066] wherein, and are the total amounts of review samples of the first task and the i-th task.
[0067] The sampling process uses a plasticity sampling strategy, which includes:
[0068] Define the plasticity sampling probability a;
[0069] For the t-th sampling, calculate the number of new task samples S t , the expression is:
[0070]
[0071] where B M is the total sample size of the single review stream, S t is the number of samples allocated to new tasks at the t-th sampling;
[0072] When t < B M , it is calculated according to the formula
[0073] When t ≥ B M , it is fixed to 1 to avoid new task samples being completely ignored;
[0074] It should be noted that the task-adaptive incremental review strategy aims to dynamically adjust the amount of historical samples used for each review according to the change in the number of tasks. The initial review ratio is used to set the base review sample number for the first task, and the incremental review ratio determines the growth rate of the review sample for subsequent tasks. Through this linear growth method, the system can maintain effective review of old knowledge while preventing the occurrence of forgetting phenomenon, while avoiding resource waste and improving the scalability and practicality of the entire continuous learning framework.
[0075] S4, in the review stream, sample historical samples and their logits from the replay buffer, and jointly optimize the model parameters using cross-entropy loss and mean square error loss to reactivate the passivated classifier weights;
[0076] Further, the formula for calculating the mean square error loss is:
[0077] L mse = γ·MSE(f θ (x old ), l old )
[0078] Where γ is the knowledge distillation loss weight, l old = f θ (x new ) is the output logits of the historical samples saved by the cognitive stream;
[0079] The joint optimization of the review stream includes:
[0080] Cross-entropy loss and mean square error loss function, expression is:
[0081]
[0082] L mse = γ·MSE(f θ (x old ), l old );
[0083] Where (x old , yold , l old ) are historical samples of the replay buffer and their corresponding logits;
[0084] It should be noted that the introduced plasticity sampling strategy is a method of dynamically adjusting the review sample distribution based on task importance. The strategy defines a sampling probability, so that in the review stream, both historical tasks and appropriate attention to current tasks can be focused on. When the proportion of new task samples is too small, it is dynamically adjusted according to the formula; when it approaches or exceeds the threshold, it is fixed to 1 to prevent the new task from being completely ignored. This design enhances the adaptability of the method in the face of frequent task switching or task difficulty changes, and improves the overall generalization performance of the model. In the review stream, all tasks are sampled with the same probability, which leads to the fact that after multiple reviews, the front tasks are learned every time, and the total learning times are greater than the rear tasks. Plasticity sampling controls the learning distribution of samples in the replay, reduces excessive attention to old tasks, and increases review of new tasks to improve the model's new task learning ability. The review stream uses mean square error loss as the knowledge distillation term, which aims to guide the model to update the parameters as much as possible to maintain the consistency of the historical sample logits. In the formula, is the knowledge distillation loss weight, which is used to balance the relationship between the classification performance of the new task and the memory retention of the old task. By recording the historical logits in the cognitive stream as the target, the overall effect of the review stream can effectively suppress the classifier weight blunting problem caused by new task learning, thereby realizing the smooth migration and stable retention of knowledge in the multi-task continuous learning process. The joint optimization process in the review stream, and the former and the latter jointly act to improve the classification accuracy on all old tasks (after the cognitive stream, the current task also becomes an old task and can be sampled in the buffer), under the joint action of the two, not only can effectively alleviate the catastrophic forgetting problem, but also can enhance the model's adaptability to new tasks. The plasticity sampling strategy enhances the model's adaptability to new tasks, and under the joint action of the two, the catastrophic forgetting and recent bias are effectively alleviated. Task-adaptive incremental review + plasticity sampling significantly improves the robustness and generalization ability of the model in long-term task sequences.
[0085] The embodiment also provides a computer device suitable for the decoupled replay and memory awakening method for online continuous learning, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the decoupled replay and memory awakening method for online continuous learning proposed in the above embodiment.
[0086] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, a carrier network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, a trackball or a touchpad arranged on the shell of the computer device, or an external keyboard, a touchpad or a mouse, etc.
[0087] The embodiment also provides a storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the method for decoupling replay and memory wake-up for online continual learning. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.
[0088] In summary, by initializing the model parameters and the replay buffer, loading the pre-trained backbone network, and configuring the two independent optimizers of the cognitive stream and the review stream, the application realizes the decoupling optimization mechanism of the model between new task learning and old task review, alleviates the catastrophic forgetting, solves the problem of recent bias, and improves the model plasticity. By training using the current task data and saving the key samples and logits to the replay buffer, the application realizes the isolation of new and old knowledge to eliminate the gradient conflict caused by the old task to the new task, achieves the effect of efficient learning and memory accumulation, has good task transition ability and knowledge retention potential, dynamically adjusts the number of review samples through the task adaptive incremental review strategy, realizes the intelligent allocation of review resources, achieves the effect of balancing the learning resources of new and old tasks and maintaining the stability of the overall performance, has good scalability and adjustability, and re-trains the historical samples by jointly using the cross-entropy loss and the mean square error loss, realizes the effective awakening and integration of old knowledge.
[0089] Embodiment 2, referring to Table 1 and Table 2, is a second embodiment of the application, and experimental simulation data of the decoupling replay and memory awakening method for online continuous learning are given to further verify the technical solution of the application.
[0090] The experiment adopts an online incremental learning setting, that is, each task arrives in sequence, and the model can only access the data stream of the current task and cannot repeatedly access the historical task data. All methods use the same backbone network structure and are compared under the same buffer size (such as 500, 2000, 5000). The application participates in the comparison as "the application" and "the application P version" respectively, wherein "the application" contains the decoupling mechanism and the task adaptive incremental review, and the plasticity sampling can be introduced on this basis to obtain "the application P version".
[0091] Specifically, as shown in Table 1 below:
[0092]
[0093]
[0094] Table 1: Comparison table of key performance indicators
[0095] Extension performance under long task sequence:
[0096] Specifically, as shown in Table 2 below, T represents the number of tasks into which the data set is divided, each task contains the same number of categories, and the median value is A B , which represents the final accuracy:
[0097]
[0098]
[0099] Table 2: Extensibility performance table under long task sequence
[0100] It can be found that the present application and its P version show significant advantages in multiple key performance indicators:
[0101] On the CIFAR-100 dataset (buffer size 5000), the P version of the present application significantly outperforms the optimal baseline DualNet (64.4%) and ERACE (63.5%) with an accuracy of 70.4%, with an improvement of more than 60%.
[0102] On the CORe50 and MiniImageNet datasets, the P version of the present application achieves an accuracy of 86.0% and 81.3% respectively under a buffer size of 2000, both of which are better than the comparative methods (such as DER++ on CORe50 with 84.0% and SCR on MiniImageNet with 74.5%).
[0103] Under the same buffer capacity, the present application can achieve higher accuracy, indicating its efficient utilization of historical data.
[0104] In the extreme long task sequence of T = 100, the P version of the present application far surpasses ERACE (54.5%) and ER (58.9%) with an accuracy of 67.1%, and does not appear serious performance degradation like DER++ (44.4%).
[0105] In terms of time efficiency, the P version of the present application (1824s) is close to ER (1768s), but significantly better than DualNet (4543s).
[0106] In summary, the present application effectively alleviates the problem of catastrophic forgetting by decoupling the replay mechanism and task-adaptive incremental review, especially in the long task sequence and small buffer scenarios. The introduced plasticity sampling (P version) further improves the model's adaptability to dynamic data streams, providing an online continuous learning solution with high performance and scalability.
[0107] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.
Claims
1. A decoupled playback and memory awakening method for online continuous learning, characterized by: include: S1. Initialize model parameters and playback buffer, load the pre-trained backbone network, and configure independent cognitive flow optimizers and retrospective flow optimizers; S2. In the cognitive stream, the model is trained using the current new task data, the model parameters are optimized using the cross-entropy loss function, and the samples of the current task and their output logits are stored in the playback buffer; S3. Based on the task-adaptive incremental review strategy, dynamically calculate the total number of review samples required for the current task, where the total number of review samples increases linearly with the number of tasks; S4. In the review stream, historical samples and their logits are sampled from the replay buffer, and the model parameters are jointly optimized using cross entropy loss and mean squared error loss to reactivate the passivated classifier weights; Repeat steps S2 to S4 until the online continuous learning of all tasks is completed.
2. The decoupled playback and memory awakening method for online continuous learning according to claim 1, characterized in that: In step S1, the learning rate of the cognitive flow optimizer is set to 0.01, the learning rate of the retrospective flow optimizer is set to 0.005, and the optimizer type is stochastic gradient descent SGD.
3. The decoupled playback and memory awakening method for online continuous learning according to claim 2, characterized in that: The specific implementation of the task adaptive incremental review strategy in step S3 includes: Initialize the review ratio r init and incremental review ratio r incre ; For the first task T1, calculate the number of initial review samples based on its data volume N1 For the i-th task T i , based on its data volume N1, the total amount of review samples is updated to in, and is the total number of review samples for the 1st task and the i-th task.
4. The decoupled playback and memory awakening method for online continuous learning according to claim 3, characterized in that: The sampling process in step S4 adopts a plastic sampling strategy, which specifically includes: Define the plasticity sampling probability α; For the tth sampling, calculate the number of new task samples S t , the expression is: Among them, B M is the total sample size of a single review flow sampling, S t is the number of samples assigned to the new task at the tth sampling; When t M When, according to the formula calculate; When t≥B M When , it is fixed to 1 to avoid new task samples being completely ignored.
5. The decoupled playback and memory awakening method for online continuous learning according to claim 4, characterized in that: The calculation formula of the mean square error loss in step S4 is: L mse =γ·MSE(f θ (x old ),l old ) Among them, γ is the knowledge distillation loss weight, l old =f θ (x new ) outputs logits for historical samples saved in the cognitive stream.
6. The decoupled playback and memory awakening method for online continuous learning according to claim 5, characterized in that: In step S2, the playback buffer adopts the Reservoir Sampling strategy to manage samples, specifically including: Set the maximum capacity of the buffer M; For each new task sample (x new ,y new ), with probability Replaces a random sample in the buffer, where N is the total number of samples currently processed.
7. The decoupled playback and memory awakening method for online continuous learning according to claim 6, characterized in that: The loss function of the cognitive flow in step S2 is: Among them, L CE is the cross entropy loss function, T i The data distribution of the current task.
8. The decoupled playback and memory awakening method for online continuous learning according to claim 7, characterized in that: The joint optimization of the review flow in step S4 includes: The cross entropy loss and mean square error loss functions are expressed as: L mse =γ·MSE(f θ (x old ),l old ); Where (x old ,y old ,l old ) are historical samples sampled in the playback buffer and their corresponding logits.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the decoupled playback and memory awakening method for online continuous learning according to any one of claims 1 to 8 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the decoupled playback and memory awakening method for online continuous learning according to any one of claims 1 to 8 are implemented.