A federated class incremental learning method, system, device and medium for streaming data

Abstract

The application discloses a kind of federal class incremental learning method, device and medium for streaming data, wherein the method comprises: server initializes global feature extractor and is issued to client;Client receives new data containing new class and global feature extractor;According to the fusion ratio, fuse global feature extractor and local feature extractor;Current new data and saved old class representative sample are divided into several meta-tasks according to arrival time;Incremental meta-learning method is used to train on new and old tasks, update fusion ratio and local model, and global feature extractor is not updated;Client sends the parameter of fused feature extractor to server for aggregation, as the global feature extractor of next round.The application dynamically extracts the knowledge in global feature extractor that is beneficial to client through fusion ratio, prevents the interference of heterogeneous data on client local model, reduces the problem of catastrophic forgetting, and can be widely applied in the field of artificial intelligence technology.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a federated incremental learning method, apparatus and medium for streaming data. Background Technology

[0002] With the rapid development of artificial intelligence, deep learning is widely used in edge devices. These edge devices constantly generate new data, and due to privacy concerns, this data cannot be shared with other devices. Federated learning enables multiple clients to collaboratively learn a global model without sharing private data. This global model aggregates knowledge from different clients to help clients in the network achieve better data processing performance. However, real-world distributed data is heterogeneous, meaning that the feature or label distributions among clients can be very uneven. Existing work has shown that data heterogeneity degrades the classification accuracy and convergence performance of the global model. To address this issue, recent research has proposed various methods, one of which is personalized federated learning. This method leverages the knowledge aggregated globally to learn a model adapted to the local data distribution for each client, thus avoiding interference caused by data heterogeneity.

[0003] Furthermore, data on individual clients is often fluid, meaning its distribution changes over time, causing the model to forget what it has learned during training. Incremental learning allows models to learn new knowledge from constantly arriving new data while retaining as much of their past knowledge as possible; incremental learning is specifically designed for scenarios where new categories emerge over time. In federated learning scenarios, traditional aggregation methods may mask past knowledge learned by the client, thus exacerbating the local forgetting problem. Summary of the Invention

[0004] In order to at least partially solve one of the technical problems existing in the prior art, the present invention aims to provide a federated incremental learning method, apparatus and medium for streaming data.

[0005] The technical solution adopted in this invention is:

[0006] A federated incremental learning method for streaming data includes the following steps:

[0007] The server initializes the global feature extractor and sends it to the client.

[0008] The client receives new data containing the new classes and a global feature extractor, and saves the received information locally;

[0009] Initialize the fusion ratio, and then fuse the global feature extractor and the local feature extractor according to the fusion ratio;

[0010] The current new data and the saved old representative samples are divided into several meta-tasks according to their arrival time;

[0011] Incremental meta-learning is used to train on old and new tasks, updating the fusion ratio and the local model, while the global feature extractor is not updated.

[0012] After training, the client sends the fused feature extractor parameters to the server for aggregation, which will then be used as the global feature extractor for the next round.

[0013] Further, the step of fusing the global feature extractor and the local feature extractor according to the fusion ratio includes:

[0014] At the beginning of the t-th incremental round, each client receives the global feature extractor aggregated by the server in the (t-1)-th round. The global feature extractor is kept locally and participates in the subsequent training together with the local feature extractor.

[0015] Set a scaling vector Used to fuse global and local feature extractors; this scaling vector maintains a fusion weight for each layer of the feature extractor, denoted as... Where L is the number of layers in the feature extractor;

[0016] The feature extractor for fusion is defined as follows:

[0017]

[0018] in, For global feature extractor, This is the local feature extractor obtained in round t-1. The fused feature extractor is used to extract sample features, which are then fed into the local classifier for final classification.

[0019] Furthermore, the feature extractor is trained using the following method:

[0020] Before each round of training, the scaling vector is randomly initialized.

[0021] During training, the global feature extractor With a fixed value, train for E rounds using a meta-learning method to update the scale vector. Local Feature Extractor and local classifier To minimize the following loss function:

[0022]

[0023] Where l is the cross-entropy loss, f is the mapping function of the feature extractor, and g is the classification function. For the current data on client n, For sample features, The true labels for the samples.

[0024] Furthermore, the server aggregates the fused feature extractors collected from each client using the following formula:

[0025]

[0026] in Let n be the number of training samples for client n. This represents the total number of training samples across all clients.

[0027] Furthermore, the step of dividing the current new data and the saved representative samples of the old class into several meta-tasks according to their arrival time includes:

[0028] Receive new tasks on the client Then, the model on the client classifies the data into t×C categories, where C is the number of new categories at each time step;

[0029] For each training batch, the data is divided into several meta-tasks, with a maximum of t meta-tasks, where samples belonging to the same meta-task come from the same time point.

[0030] Furthermore, the incremental meta-learning method is used to train on both new and old tasks, updating the fusion ratio and the local model, including:

[0031] Basic Model Θ base Initialize to the current local model, train a copy of the base model for each meta-task, and obtain a task-specific meta-model Θ. k ;

[0032] During the training of each meta-task, the entire classifier is updated to obtain a classifier that can distinguish all observed classes. Finally, the meta-model parameters are averaged and combined with the base model to obtain a new model, which will be used as the base model for the next batch.

[0033] Furthermore, the expression for the new model is:

[0034]

[0035] in β is a constant coefficient, set to 1.0, and T is the total number of incremental learning rounds for the federated class.

[0036] Another technical solution adopted in this invention is:

[0037] A federated incremental learning system for streaming data includes:

[0038] The data delivery module is used by the server to initialize the global feature extractor and send it to the client;

[0039] The data receiving module is used by the client to receive new data containing new classes and the global feature extractor, and to save the received information locally;

[0040] The feature fusion module is used to initialize the fusion ratio and fuse the global feature extractor and the local feature extractor according to the fusion ratio;

[0041] The task partitioning module is used to divide the current new data and the saved old representative samples into several meta-tasks according to their arrival time.

[0042] The model training module is used to train on new and old tasks using incremental meta-learning methods, update the fusion ratio and the local model, while the global feature extractor is not updated.

[0043] The data aggregation module is used after training, where the client sends the fused feature extractor parameters to the server for aggregation, which serves as the global feature extractor for the next round.

[0044] Another technical solution adopted in this invention is:

[0045] A federated incremental learning device for streaming data includes:

[0046] At least one processor;

[0047] At least one memory for storing at least one program;

[0048] When the at least one program is executed by the at least one processor, the at least one processor implements the method described above.

[0049] Another technical solution adopted in this invention is:

[0050] A computer-readable storage medium storing a processor-executable program, which, when executed by a processor, performs the method described above.

[0051] The beneficial effects of this invention are: by dynamically extracting knowledge that is beneficial to the client from the global feature extractor through fusion ratio, this invention prevents heterogeneous data from interfering with the client's local model, and by using meta-learning, it enables both new and old class data on the client to participate in personalized training, thereby mitigating the catastrophic forgetting problem. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating the steps of a federated incremental learning method for streaming data in an embodiment of the present invention. Detailed Implementation

[0054] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0055] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0056] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0057] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0058] In incremental learning, the model learns incrementally on continuously arriving data. This requires a learning strategy that can learn new knowledge from new data while preventing the forgetting of old knowledge. Incremental learning methods are mainly divided into two categories: task-based incremental learning and class-based incremental learning. Task-based incremental learning deals with data streams containing different classification tasks, distinguished by task IDs, and each sample needs to be classified into the class corresponding to its task. Class-based incremental learning focuses on data streams from the same classification task, where new classes emerge over time. All samples need to be classified using a classifier that can distinguish between currently observed classes.

[0059] There are two main approaches to incremental learning: regularization-based and replay-based methods. Regularization-based methods add a regularization term to the loss function to prevent the model from forgetting previously learned knowledge. Knowledge distillation loss is widely used for regularization, using the output of the old model as a soft label to transfer old knowledge to the new model. Replay-based methods store a certain amount of old data and replay it during subsequent training, allowing for review of old knowledge while learning new knowledge. Recently, meta-learning has also been used to address the catastrophic forgetting problem in incremental learning. Meta-learning can extract general knowledge across multiple tasks and quickly apply it to new tasks, so incremental meta-learning can learn knowledge common to both new and old tasks.

[0060] Due to the distributed and fluid nature of real-world data, federated incremental learning has been proposed to address this challenging scenario. DCIL proposed a distributed class-incremental learning framework that uses matrix fusion, keeping the parameters of the old classes unchanged while only updating the parameters of the new classes, which are then directly added to the current global model. However, this approach has only proven effective on linear classifiers and is not suitable for multi-layer neural networks. DCIGAN trains a global generator for each observed class, which can generate data that approximates the global data distribution within the distributed class-incremental learning framework, thus preserving data distribution information for all classes. FedWeIT proposed a distributed continuous learning framework where each client's model is decomposed into a globally shared model and several task-specific models, all of which are sent to each client for knowledge transfer. The storage costs of the above two methods increase significantly because the number of models on each client grows with the arrival of new data. GLFC learns a global class-incremental model to simultaneously address both global and local forgetting problems, using gradients calculated from samples to help the server select the best old model for distillation. GLFC also proposes a novel loss function to balance the learning progress of each class, thus addressing the problem of data imbalance. However, a single global model may be more susceptible to interference from non-IiD environments.

[0061] like Figure 1 As shown, this embodiment provides a federated incremental learning method for streaming data, aiming to simultaneously address the spatial data heterogeneity problem and the temporal catastrophic forgetting problem. The method specifically includes the following steps:

[0062] S1. The server initializes the global feature extractor and sends it to the client;

[0063] S2. The client receives new data containing the new class and the global feature extractor, and saves the received information locally;

[0064] S3. Initialize the fusion ratio, and fuse the global feature extractor and the local feature extractor according to the fusion ratio;

[0065] S4. Divide the current new data and the saved old representative samples into several meta-tasks according to their arrival time;

[0066] S5. Use incremental meta-learning to train on new and old tasks, update the fusion ratio and local model, and do not update the global feature extractor.

[0067] S6. After training, the client sends the fused feature extractor parameters to the server for aggregation, which will serve as the global feature extractor for the next round.

[0068] In this embodiment, there are N clients in the network, and all clients will perform T rounds of incremental class learning together. During each round of incremental class learning, each client receives a new task containing data on categories that the client has not yet learned. Simultaneously, the client retains a fixed number of representative samples from the old classes. Both new and old data participate in local training. The client's model consists of two parts: a feature extractor and a classifier. The classifier is the output layer of the model, and the rest of the model is the feature extractor. Since each client learns different categories, the classifiers between clients are heterogeneous and cannot be directly aggregated. Therefore, this embodiment dynamically extracts knowledge beneficial to the client from the global feature extractor by fusion ratio, preventing heterogeneous data from interfering with the client's local model. Furthermore, through meta-learning, both new and old class data on the client participate in personalized training, thereby mitigating the catastrophic forgetting problem.

[0069] The following section provides a detailed explanation of dynamic feature extractor fusion and incremental meta-learning.

[0070] (1) Fusion of dynamic feature extractors

[0071] Traditional federated learning methods aggregate local models into a global model and send it to the client. The client then uses the global model to replace the local model for further training. However, this replacement strategy can lead to catastrophic forgetting in federated incremental learning scenarios. Because the client's data exhibits non-independent and identically distributed characteristics, the aggregated model integrates information irrelevant to the specific client. Therefore, if the local model is directly replaced by the global model, the client's past knowledge may be contaminated by irrelevant information, resulting in catastrophic forgetting. To address this, this embodiment attempts to retain both the local and global models locally and proposes an effective method: dynamic feature extractor fusion, to simultaneously mitigate catastrophic forgetting and learn new knowledge.

[0072] like Figure 1 As shown, due to the heterogeneity among the classifiers on the client side, the only information exchanged between the server and the client is the parameter of the feature extractor. At the beginning of the t-th incremental round, each client receives the global feature extractor (hereinafter referred to as the global FE) aggregated by the server in the (t-1)-th round. The global FE is retained locally and participates in subsequent training along with the local feature extractor (hereinafter referred to as the local FE). Our goal is to utilize the global knowledge in the global FE and prevent overwriting local knowledge. To achieve this goal, this embodiment uses a scaling vector. The global feature extraction (FE) and local feature extraction (FE) are fused. This vector maintains a fusion weight for each layer of the feature extractor, so it can be represented as... Where L represents the number of layers in the feature extractor. The fused feature extractor is defined as follows:

[0073]

[0074] in For global FE, This is the local FE obtained in round t-1. The fused feature extractor is used to extract sample features, which are then fed into the local classifier for final classification.

[0075] Before each round of training, It is randomly initialized. During training, we will use the global FE. With a fixed value, train for E rounds using a meta-learning method to update the scale vector. Local FE and local classifier To minimize the following loss function:

[0076]

[0077] Where l is the cross-entropy loss, f is the mapping function of the feature extractor, and g is the classification function. This represents the current data on client n.

[0078] By learning these parameters, the client can dynamically adjust the utilization of global knowledge based on the local data distribution, allowing the model to absorb only client-relevant information. Furthermore, the local model can learn new knowledge while maintaining past knowledge. At the end of the meta-learning process, the fused feature extractor is transformed into a single feature extractor, which will serve as the local feature extractor. Additionally, we update the representative samples stored on the client, selecting the most representative samples from each class. Specifically, for each category in both the new and old classes, we extract the sample features for that class using the current feature extractor, and then store the samples whose features are closest to the mean of the sample features. Finally, the server collects the fused feature extractors from each client and aggregates them using a weighted average.

[0079]

[0080] in This represents the total number of training samples across all clients.

[0081] (2) Incremental meta-learning

[0082] Feature extractor fusion mitigates catastrophic forgetting by preventing local knowledge from being replaced by noisy global knowledge, but we only have access to new class data and a portion of past data. This leads to class imbalance, causing the model to favor the new class with more samples. A simple solution is to use a regularization term, such as knowledge distillation loss, to prevent the model from forgetting old knowledge. However, this method performs poorly on new data because the model doesn't attempt to update. To better balance the learning of new and old classes, we focus on meta-learning methods, which can extract knowledge shared between different tasks. Meta-learning is suitable for our class increment scenario because we want to learn knowledge shared between incremental tasks.

[0083] In existing literature [1], meta-learning has been applied to task incremental learning to avoid catastrophic forgetting, but this method cannot be directly used to solve the problem of this invention. This is because literature [1] trains a task-specific classifier for each task, while class incremental learning requires training a unified classifier for all tasks to distinguish different categories. Therefore, we modify the method in literature [1] and apply it to our federated class incremental method.

[0084] Receive new tasks on the client Then, its model needs to classify data into t×C categories. For each training batch, we divide the data into several meta-tasks, at most t, with samples belonging to the same meta-task coming from the same incremental round. Basic Model Θ baseInitialize to the current local model, train a copy of the base model for each meta-task to minimize the above loss function, and obtain a task-specific meta-model Θ. k We update the entire classifier during the training of each meta-task because we want to obtain a classifier that can distinguish all observed classes, unlike in reference [1]. Finally, the meta-model parameters are averaged and combined with the base model to obtain a new model:

[0085]

[0086] in The new model will be used as the base model for the next batch.

[0087] We utilize meta-learning to learn the fusion ratio and local parameters, extracting shared knowledge from both new and old tasks for personalized training. Although the old task has fewer samples than the new task, each meta-model is trained independently on its corresponding meta-task, thus ensuring that the new model is not biased towards the new task.

[0088] Literature [1]: J.Rajasegaran, S.Khan, M.Hayat, FSKhan, and M.Shah, "iTAML: Anincremental task-agnostic meta-learning approach," in IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

[0089] This embodiment also provides a federated incremental learning system for streaming data, including:

[0090] The data delivery module is used by the server to initialize the global feature extractor and send it to the client;

[0091] The data receiving module is used by the client to receive new data containing new classes and the global feature extractor, and to save the received information locally;

[0092] The feature fusion module is used to initialize the fusion ratio and fuse the global feature extractor and the local feature extractor according to the fusion ratio;

[0093] The task partitioning module is used to divide the current new data and the saved old representative samples into several meta-tasks according to their arrival time.

[0094] The model training module is used to train on new and old tasks using incremental meta-learning methods, update the fusion ratio and the local model, while the global feature extractor is not updated.

[0095] The data aggregation module is used after training, where the client sends the fused feature extractor parameters to the server for aggregation, which serves as the global feature extractor for the next round.

[0096] This embodiment of the federated incremental learning system for streaming data can execute the federated incremental learning method for streaming data provided in the method embodiment of the present invention. It can execute any combination of implementation steps of the method embodiment and has the corresponding functions and beneficial effects of the method.

[0097] This embodiment also provides a federated incremental learning device for streaming data, including:

[0098] At least one processor;

[0099] At least one memory for storing at least one program;

[0100] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 1 The method shown.

[0101] This embodiment provides a federated incremental learning device for streaming data, which can execute a federated incremental learning method for streaming data provided in the method embodiment of the present invention. It can execute any combination of implementation steps of the method embodiment and has the corresponding functions and beneficial effects of the method.

[0102] This application also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform... Figure 1 The method shown.

[0103] This embodiment also provides a storage medium storing instructions or programs that can execute the federated incremental learning method for streaming data provided in the method embodiment of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the method has the corresponding functions and beneficial effects.

[0104] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0105] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0106] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0108] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0109] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0110] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0111] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0112] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A federated class incremental learning method for streaming data, characterized in that, The method comprises the following steps: The server initializes a global feature extractor and sends it to the client; The client receives new data containing new classes and the global feature extractor, and saves the received information locally; A fusion ratio is initialized, and the global feature extractor and the local feature extractor are fused according to the fusion ratio; The current new data and the saved representative samples of the old classes are divided into a plurality of meta-tasks according to the arrival time; An incremental meta-learning method is used to train on new and old tasks, update the fusion ratio and the local model, and the global feature extractor is not updated; After the training is completed, the client sends the fused feature extractor parameters to the server for aggregation as the global feature extractor of the next round; The global feature extractor and the local feature extractor are fused according to the fusion ratio, comprising: At the beginning of the first incremental round, each client receives the global feature extractor aggregated from the first round of servers, which is kept locally and participates in the following training together with the local feature extractor; Setting a scale vector for fusing the global feature extractor and the local feature extractor; the scale vector maintains a fusion weight for each layer of the feature extractor, denoted as wherein is the number of layers of the feature extractor; The fused feature extractor is defined as follows: wherein, is a global feature extractor, is a first is a local feature extractor The incremental meta-learning method is used to train on new and old tasks, update the fusion ratio and the local model, comprising: base model initialized to the current local model, each meta-task trains a copy of the base model and obtains a task-specific meta-model ; During the training of each meta-task, the entire classifier is updated to obtain a classifier that can distinguish all observed classes. Finally, the meta-model parameters are averaged and combined with the base model to obtain a new model, which will be used as the base model for the next batch.

2. The federated episodic learning method for streaming data according to claim 1, wherein, The feature extractor is trained in the following way: Initialize the proportion vector randomly before each round of training ; During training, the global feature extractor is fixed, trained using a meta-learning method The global feature extractor , the local feature extractor and the local classifier are trained to minimize the following loss function: in, For cross-entropy loss, The mapping function for the feature extractor. For classification functions, For the client Current data on For sample features, The true labels for the samples.

3. The federated episodic learning method for streaming data according to claim 2, wherein, The server aggregates the fused feature extractors collected from each client by the following formula: wherein is the number of training samples for the client is the total number of training samples for all clients is the total number of training samples for all clients 4. The federated episodic learning method for streaming data according to claim 1, wherein, The current new data and the saved representative samples of the old classes are divided into a plurality of meta-tasks according to the arrival time, comprising: Receiving new tasks at a client The model on the client then classifies data into a number of classes, where is the number of new classes for each time instant; For each training batch, the data is partitioned into a number of meta-tasks, up to wherein samples belonging to the same meta-task come from the same time instant.

5. The federated episodic learning method for streaming data according to claim 1, wherein, The expression of the new model is: wherein , is a coefficient constant, is the total number of federal class incremental learning rounds.

6. A federated class incremental learning system for streaming data, the system comprising: Comprising: A data sending module is configured to initialize a global feature extractor by a server and send the global feature extractor to a client; A data receiving module is configured to receive new data containing new classes and the global feature extractor by the client, and save the received information locally; A feature fusion module is configured to initialize a fusion ratio, and fuse the global feature extractor and the local feature extractor according to the fusion ratio; A task division module is configured to divide the current new data and the saved representative samples of the old classes into a plurality of meta-tasks according to the arrival time; A model training module is configured to use an incremental meta-learning method to train on new and old tasks, update the fusion ratio and the local model, and the global feature extractor is not updated; A data aggregation module is configured to send the fused feature extractor parameters to the server for aggregation as the global feature extractor of the next round after the training is completed. The global feature extractor and the local feature extractor are fused according to the fusion ratio, comprising: At the beginning of the first incremental round, each client receives the global feature extractor aggregated from the first round of servers, which is kept locally and participates in the following training together with the local feature extractor; Setting a scale vector for fusing the global feature extractor and the local feature extractor; the scale vector maintains a fusion weight for each layer of the feature extractor, denoted as wherein is the number of layers of the feature extractor; The fused feature extractor is defined as follows: wherein, is a global feature extractor, is a first local feature extractor. The incremental meta-learning method is used to train on new and old tasks, update the fusion ratio and the local model, comprising: base model initialized to the current local model, each meta-task trains a copy of the base model and obtains a task-specific meta-model ; During the training of each meta-task, the entire classifier is updated to obtain a classifier that can distinguish all observed classes. Finally, the meta-model parameters are averaged and combined with the base model to obtain a new model, which will be used as the base model for the next batch.

7. A federated class incremental learning device for streaming data, characterized in that, Comprising: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of claims 1-5.

8. A computer readable storage medium having stored therein a program which is executable by a processor, characterized in that, The program executable by the processor when executed by the processor is for performing the method of any one of claims 1-5.

Images