A federated class incremental learning method based on feature distillation and class prototype

Abstract

The application relates to a federal class incremental learning method based on feature distillation and class prototype, which comprises an initialization training stage and an incremental learning stage. In the initialization training stage, i.e. when a user receives a task for the first time, the system trains a local model by using a cross-entropy loss function; when the user task is updated, the system enters the incremental learning stage, and on the basis of the initialization training, an incremental learning strategy for retaining old knowledge is added. The incremental learning strategy comprises a feature distillation strategy for a local feature extractor, a prototype enhancement strategy for a local classifier, a fine-tuning strategy for an aggregated model and an updating strategy for a class prototype. Without storing old data locally, the application effectively alleviates the catastrophic forgetting of a federal learning system when facing a dynamic data stream by letting a local user and a server jointly participate in training, and improves the recognition performance of a model on new and old data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of federal class incremental learning, and particularly relates to a federal class incremental learning method based on feature distillation and class prototype. BACKGROUND

[0002] With the wide popularity of the Internet of Things, hundreds of millions of devices worldwide are constantly generating massive amounts of data, posing unprecedented demands for data processing and analysis. Due to data security and privacy issues, devices often cannot share data with other devices, and in this context, federal learning has emerged. It is a distributed learning system. Devices participating in federal learning train local models using local data, and a central server collects and aggregates local models to achieve the purpose of collaborative training of shared models by multiple devices. However, federal learning often assumes that training data and test data come from the same and fixed class, but the actual environment is open and dynamic. If training is only performed on new class data, it will seriously affect the classification performance of the model on old class data, i.e., the catastrophic forgetting problem. Therefore, it is necessary to design appropriate federal incremental learning strategies to address the challenges of incremental environments.

[0003] The existing federal class incremental learning technology has the following technical problems: (1) old class data is retained locally, and as the number of classes increases, the computing power and storage space of the local device have higher requirements; (2) existing drift compensation strategies are mostly suitable for centralized systems, and due to the heterogeneity of system structure and data, they are not suitable for distributed systems; (3) the federal learning aggregation strategy can exacerbate class prototype drift. If class prototype correction is performed after each aggregation is completed, on the one hand, due to the complexity and time-consuming of existing drift compensation strategies, it will lead to low efficiency of the entire system; on the other hand, based on neural network prediction drift or based on new class data prediction drift, the results obtained are not accurate. SUMMARY

[0004] To solve the above technical problems, the present application provides a federal class incremental learning method based on feature distillation and class prototype, which improves the existing technology, alleviates the catastrophic forgetting problem, and enhances the adaptability and reliability of the incremental learning strategy.

[0005] To achieve the above purpose, the technical solution adopted by the present application is as follows: a federal class incremental learning method based on feature distillation and class prototype is provided, and the steps include:

[0006] S1 When the user receives the task t for the first time, enter the initialization training stage, including the following steps:

[0007] S1.1 The server initializes the training model and distributes it to the user;

[0008] S1.2 After the user k trains the local data set locally to obtain an initial local model, the initial local model is uploaded to the server;

[0009] S1.3 The server collects the initial local model to obtain an initial global model ω t,r , which is then distributed to the users for local training in the next round r;

[0010] S1.4 The steps S1.2-S1.3 are repeatedly performed to iteratively update the initial global model ω t,r until the performance converges, and the local training of the task t is completed to obtain an initial model ω t .

[0011] S2 The class prototypes of each class c are calculated , and the prototype enhancement amplitude is calculated . The class prototypes are used to select local sample data, and the class prototype set and the local sample data are uploaded to the server . The prototype enhancement amplitude is saved locally.

[0012] S3 The server aggregates the class prototype set to obtain a global class prototype set {e c}; The server collects the local sample data to construct a shared data set S.

[0013] S4 When the user receives a new task, the system enters an incremental learning stage, including the following steps:

[0014] S4.1 The server freezes the model obtained after the iterative update of the previous task until the performance converges as an old model, adds |C new | neuron nodes to the fully connected layer of the old model copy as a new classification model for the new task, and distributes the old model, the new classification model, and the global class prototype set {e c} to the users; The new classification model has |C new |+|C old | output nodes, |C old | is the total number of old classes, and |C new | is the number of new classes.

[0015] S4.2 The user trains the new classification model using the feature distillation and prototype enhancement strategies to obtain a new local model, which is then uploaded to the server;

[0016] S4.3 The server collects the new local model to perform aggregation operation to obtain a new global model;

[0017] ​​The S4.4 server fine-tunes the new global model classifier based on the shared dataset S, and then distributes the fine-tuned new global model to users for the next round of local training.

[0018] S4.5 repeats S4.2-S4.4, iteratively updating the new global model until performance convergence, thus obtaining the new model;

[0019] S5 users calculate the new class prototype and the enhancement of the new class prototype, use the new class prototype to select new class sample data, and upload the new class prototype and new class sample data to the server.

[0020] The S6 server aggregates the new class prototypes to obtain the new class global class prototype, supplementing the global class prototype set {e} mentioned in S3. c The server collects sample data of the new class to supplement the shared dataset S, and at the same time, the server updates the global class prototype of the old class based on the shared dataset.

[0021] Preferably, the initial local model described in step S1.2 utilizes the local loss function L local Local training is performed, and the local loss function L local By cross-entropy loss function L CE constitute:

[0022]

[0023] Where f represents the feature extractor and θ represents the classifier. This represents the local dataset used for the first time to receive the task. For the local dataset Size, p c It is the probability that the model predicts a sample as class c. This is an indicator function; if · is true, it takes the value 1; otherwise, it takes the value 0.

[0024] Preferably, the initial global model ω described in step S1.3 t,r The calculation process for the new global model described in step 4.3 is the same, wherein the initial global model ω t,r :

[0025]

[0026] in, For the local dataset Size,

[0027] Preferably, the class prototype described in step S2 The process is the same as the new class prototype calculation process described in step S5, wherein the class prototype The calculation is as follows:

[0028] Input the data of each category into the initial model ω t Calculate the mean of the local eigenvectors:

[0029]

[0030] wherein, is the set of categories c in the local data set;

[0031] The prototype enhancement amplitude in step S2 The calculation process of the new category prototype enhancement amplitude in step S5 is the same, wherein the prototype enhancement amplitude is obtained by calculating the average category variance of the local eigenvectors:

[0032]

[0033] wherein d is the dimension of the feature space, is the covariance matrix of the feature vectors output by the data of category c, Tr(·) represents the matrix trace operation, is the set of all data categories of task t contacted by user k.

[0034] Preferably, the local sample data in step S2 and the new category sample data in step S5 are selected by the same method, wherein the local sample data selection process is as follows:

[0035] The user uses the L2 norm to measure the similarity between the local eigenvector corresponding to the data belonging to category c and the initial category prototype , and sorts them:

[0036]

[0037] The first M data with the highest similarity are selected as the local sample data, wherein the size of M is much smaller than the total number of data of category c.

[0038] Preferably, the global category prototype e c in step S3 and the new global category prototype in step S6 are calculated by the same process, wherein the global category prototype e c :

[0039]

[0040] Preferably, the loss function used in the new local model training in step S4.1 is:

[0041] L local = L CE + L ft + L orotoAug

[0042] Among them, L CE It is the cross-entropy loss for training new task data, L ft It is the characteristic distillation loss, L protoAug It is the prototype enhancement loss;

[0043] The cosine similarity is used to measure the difference between the features output by the old and new models, and is used as the feature distillation loss L. ft :

[0044]

[0045] The user sets the data volume N for the old category, uses Gaussian noise n ~ N(0,1), and the prototype of the old category to enhance the amplitude. Generate around the global class prototype e c Fluctuating pseudo-eigenvectors that conform to independent and identical distribution

[0046]

[0047] pseudo feature vectors Input the classifier and calculate the prototype enhancement loss:

[0048]

[0049] Preferably, the process of fine-tuning the new global model classifier in step S4.4 is as follows:

[0050] The server uses the shared dataset to fine-tune the new global model classifier in each round:

[0051]

[0052] in This indicates that the feature extractor is frozen, |S| represents the size of the shared dataset, and [:|C old The |] symbol indicates that only the classifier weights related to the old category are updated.

[0053] Preferably, step S6, updating the global class prototype of the old category, is performed as follows:

[0054] First, the server selects data belonging to category c from the shared dataset. o ∈C old The sample data is input into the feature extractor f of the new model. t Feature extractor f of the old model t-1 The eigenvectors z are obtained respectively. i,t and z i,t-1 The relative drift of the eigenvectors is calculated as follows:

[0055]

[0056] computing the feature vector z i,t-1 distance from the old global class prototype with a weight w that measures the relative amount of drift of the feature vector i :

[0057]

[0058] where s is the standard deviation of the Gaussian kernel;

[0059] approximating the semantic drift of the global class prototype with the drift mean of the feature vector is:

[0060]

[0061] updating the global class prototype of the old class:

[0062]

[0063] The above technical solutions have the following advantages or beneficial effects:

[0064] (1) The local does not need to save the old data, saving the calculation cost and storage space; (2) By using the feature distillation loss and the prototype enhancement loss in the local training process in the incremental learning stage, the knowledge learned by the feature extractor of the old model is retained, and the stability of the old knowledge retention and the plasticity of the new knowledge learning are balanced; (3) In the fine-tuning process of the new global model classifier, only the classifier weights of the old class are updated, preventing the training results of the new class data from being destroyed in the fine-tuning; the fine-tuning of the new global model classifier alleviates the drift problem of the class prototype; (4) The global class prototype of the old class is updated, enhancing the adaptability and reliability of the global class prototype of the old class, and in the next task training, the updated global class prototype is used to calculate appropriate pseudo feature vectors.

[0065] The above summary is merely intended to illustrate the description and is not intended to limit in any way. Further aspects, embodiments and features of the present application will be readily apparent to those skilled in the art by reference to the drawings and the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0066] In the drawings, like numerals refer to like elements throughout the various drawings. The drawings are not necessarily to scale, the emphasis instead being placed on the principles of the application. It should be understood that the drawings are merely intended to depict some embodiments of the application and should not be interpreted in limiting sense.

[0067] Figure 1 The overall flowchart of the technical solution of the present application is shown in the figure;

[0068] Figure 2 The local training schematic diagram of the user of the present application is shown in the figure;

[0069] Figure 3 The model fine-tuning strategy schematic diagram of the server of the present application is shown in the figure;

[0070] Figure 4 The class prototype updating strategy schematic diagram of the server of the present application is shown in the figure. DETAILED DESCRIPTION

[0071] In the following, only certain exemplary embodiments are simply described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the present application. Therefore, the drawings and the description are considered to be essentially exemplary rather than limiting.

[0072] The embodiment of the present application proposes a federated class incremental learning method and system based on feature distillation and class prototype, as shown in the figure. Figure 1 The overall flowchart of the present application is shown in the figure.

[0073] The federated learning system includes a server and K users, each user experiences T image classification task sequences, and the data class of the t-th task is Where C is the set of all possible data classes. The data classes between different tasks do not overlap, and each user can only access the data belonging to the task during the training of the task. In the scenario of federated learning, users participating in collaborative training are affected by factors such as personal preferences, geographical distribution, and data collection capabilities of devices. The stored data has the characteristics of non-independent and identically distributed.

[0074] The present application proposes a federated class incremental learning method based on feature distillation and class prototype, and the steps include:

[0075] S1 When the user receives the task t for the first time, enter the initialization training stage, including the following steps:

[0076] S1.1 The server initializes the training model and distributes it to the user; the neural network structure participating in the training is represented as ω t,r ={f t,r ,θ t,r}, wherein f is a feature extractor, and θ is a classifier;

[0077] S1.2 User k processes the local data set Wherein denotes the local dataset used by user k during the t-th task, after local training to obtain the initial local model, it is uploaded to the server;

[0078] The initial local model uses the cross-entropy loss function L CE Local training is performed:

[0079]

[0080] Where f represents the feature extractor, and θ represents the classifier, denotes the local dataset for the first time to accept the task, The size of the local dataset , p c is the probability that the model predicts the sample to be of class c, is an indicator function, which takes the value 1 if • is true, and 0 otherwise.

[0081] S1.3 The server collects the initial local model for aggregation to obtain the initial global model ω t,r is then distributed to the user for local training in the next round r;

[0082] The initial global model ω t,r is calculated as follows:

[0083]

[0084] Where, The size of the local dataset ,

[0085] S1.4 S1.2-S1.3 are executed in a loop to iteratively update the initial global model ω t,r to performance convergence, the local training of task t is completed, and the initial model ω t is obtained.

[0086] S2 Calculate the class prototype and the prototype enhancement amplitude of each class c. Use the class prototype to select local sample data, upload the class prototype set and the local sample data to the server, and save the prototype enhancement amplitude locally on the user's computer.

[0087] The calculation process of the class prototype is as follows:

[0088] Input the data of each class into the initial model ω t , calculate the mean of the local feature vector, and obtain the class prototype

[0089]

[0090] wherein, is the set of classes c in the local data set;

[0091] the prototype enhancement amplitude The average class variance of the local feature vector is calculated as follows:

[0092]

[0093] where d is the dimension of the feature space, is the covariance matrix of the feature vector of the data output of class c, and Tr(·) represents the matrix trace operation, is the set of all data classes of task t contacted by user k.

[0094] The local sample data selection process is as follows:

[0095] The user uses the L2 norm to measure the similarity between the local feature vector corresponding to the data belonging to class c and the initial class prototype , and sorts them in descending order:

[0096]

[0097] The top M data with the highest similarity are selected as the local sample data, where M is much smaller than the total number of data of class c, and the ratio is 0.03, that is, a small amount of local data is uploaded to the server; the local sample data is added with noise and mask for privacy and security protection during uploading to the server.

[0098] S3 Server aggregates class prototype set to obtain the global class prototype set {e c}; the server collects the local sample data and constructs a shared data set S;

[0099] wherein the global class prototype e c is calculated as follows:

[0100]

[0101] S4 When the user receives a new task, the system enters the incremental learning stage, including the following steps:

[0102] S4.1 The server freezes the model obtained after the previous task iteration update to performance convergence as an old model (for example, when receiving a new task for the first time, the initial model ω t is frozen as an old model), and adds |C new| neurons, as a new classification model for the new task, the old model, the new classification model and the global class prototype set {e c} are issued to the user; the new classification model has |C new | output nodes, |C old | output nodes, |C old | is the total amount of old categories, |C new | is the number of new categories;

[0103] S4.2 The user trains the new classification model locally using the feature distillation and prototype enhancement strategy, as shown in the flow chart Figure 2 , and uploads the new local model to the server after obtaining it;

[0104] The loss function used in the training of the new local model is:

[0105] L local = L CE + L ft + L protoAug ,

[0106] Wherein, L CE is the cross-entropy loss of training new task data, L ft is the feature distillation loss, and L protoAug is the prototype enhancement loss;

[0107] The feature distillation loss retains the feature extracted by the old model, alleviates the catastrophic forgetting problem, and uses cosine similarity to measure the difference between the features output by the old and new models as the feature distillation loss L ft ,

[0108]

[0109] The user generates rich pseudo-feature vectors of old categories using the global class prototype e c and the prototype enhancement amplitude of old categories stored locally by the user Specifically, the user sets the data amount N of the old category, uses Gaussian noise n~N(0, 1) and the prototype enhancement amplitude of the old category to generate pseudo-feature vectors that fluctuate around the global class prototype e c and conform to independent and identically distributed

[0110]

[0111] Input the pseudo-feature vector into the classifier to calculate the prototype enhancement loss:

[0112] ​

[0113] S4.3 Server collects new local model, performs aggregation operation, and obtains new global model ω t,r The calculation process is the same as that of the initial global model ω

[0114] S4.4 Server fine-tunes the new global model classifier based on the shared data set S, and the fine-tuning strategy is as shown in Figure 3 The fine-tuned new global model is distributed to the user, and the next round of local training is performed.

[0115] The fine-tuning process of the new global model classifier is as follows:

[0116] The server fine-tunes the new global model classifier of each round based on the shared data set S:

[0117]

[0118] wherein represents a frozen feature extractor, |S| represents the size of the shared data set, [:|C old ] represents that only the classifier weights related to the old categories are updated to prevent the training results of the new category data from being destroyed in the fine-tuning.

[0119] If the pseudo feature vector is reliable, the fine-tuning process will not greatly change the parameters of the classifier; if the pseudo feature vector is unreliable, the fine-tuning process will correct the training results of the new local model.

[0120] S4.5 S4.2-S4.4 are cyclically executed, and the new global model is iteratively updated until the performance converges, to obtain a new model.

[0121] S5 User calculates a new class prototype of the new category and a new class prototype enhancement amplitude, and the class prototype and the prototype enhancement amplitude The calculation process is the same as that of the class prototype

[0122] S6 Server performs aggregation of the new class prototype (and the global class prototype e cThe calculation process is the same, and the prototypes of each type are summed and averaged to obtain a new global class prototype of the type, which supplements the global class prototype set {e c} described in S3. The server collects the new class sample data to supplement the shared data set S. Since the pseudo-feature vector is used in the training process of the new model, its calculation method uses the global class prototype, so the global class prototype needs to be adjusted as the training model changes. Update the global class prototype of the old class based on the shared data set, and the update process is as shown in Figure 4 .

[0123] Update the global class prototype of the old class, as follows:

[0124] First, the server selects sample data {(x o ,y old ) | y i =c i} belonging to class c i ∈C o in the shared data set S, and inputs it into the feature extractor f t of the new model (i.e., the feature extractor of the current task) and the feature extractor f t-1 of the old model (i.e., the feature extractor of the previous task), respectively, to obtain feature vectors z i,t and z i,t-1 , respectively. The relative drift of the feature vector is calculated as:

[0125]

[0126] The semantic drift of the global class prototype o of class c is estimated using the relative drift of the feature vector. Since there is a difference between the feature vector and the global class prototype , each sample participating in the semantic drift estimation should have different measurement capabilities. The distance between the feature vector z i,t-1 and the old global class prototype (i.e., the global class prototype of the previous task) is calculated. When the feature vector z i,t-1 is closer to the old global class prototype , the measurement capability is stronger, and the proportion in the weighted average is larger. The weight w i of the relative drift of the feature vector is:

[0127]

[0128] where σ is the standard deviation of the Gaussian kernel.

[0129] The semantic drift of the global class prototype is approximately estimated using the drift mean of the feature vector as:

[0130]

[0131] updating the global class prototype of the old class:

[0132]

[0133] At the beginning of a new task, the initial classification model is distributed to the user together with the updated global class prototype set, and steps S4-S6 are cycled.

[0134] The above technical solution has the following advantages or beneficial effects: the old data does not need to be saved locally, saving computing cost and storage space; by using feature distillation loss and prototype enhancement loss in the local training process in the incremental learning stage, the knowledge learned by the feature extractor of the old model is retained, and the stability of old knowledge retention and the plasticity of new knowledge learning are balanced; during the fine-tuning process of the global model classifier, only the classifier weights of the old class are updated, preventing the training results of the new class data from being destroyed in the fine-tuning; the fine-tuning of the new global model classifier alleviates the class prototype drift problem; updating the global class prototype of the old class enhances the adaptability and reliability of the global class prototype of the old class, and in the next task training, the updated global class prototype is used to calculate appropriate pseudo-feature vectors.

[0135] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of various changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A federated class incremental learning method based on feature distillation and class prototypes, comprising the following steps: When user S1 receives task t for the first time, it enters the initialization training phase, which includes the following steps: The S1.1 server initializes the trained model and distributes it to the user; S1.2 After user k trains the local dataset locally to obtain the initial local model, it uploads it to the server; The S1.3 server collects and aggregates the initial local models to obtain the initial global model ω. t,r Then it is distributed to users for the next round of local training of r; S1.4 repeats S1.2-S1.3 iteratively, updating the initial global model ω. t,r When performance convergence is achieved, local training for task t ends, and the initial model ω is obtained. t ; S2 calculates the class prototype for each category c. and the extent of prototype enhancement Using class prototypes Select local sample data and upload the class prototype set to the server. The prototype enhancement magnitude and local sample data Stored locally by the user; S3 Server Aggregator Prototype Set Get the global class prototype set {e c The server collects the local sample data and constructs a shared dataset S. When an S4 user receives a new task, the system enters the incremental learning phase, which includes the following steps: The S4.1 server freezes the model obtained from the previous task iteration update after performance convergence as the old model, and adds |C to the fully connected layer of the old model copy. new | 1 neuron node, serving as a new classification model for the new task, combining the old model, the new classification model, and the global class prototype set {e c The new classification model is distributed to users; it has |C new |+|C old | Output nodes, | C old | represents the total amount of the old category, | C new | This represents the number of new categories; S4.2 Users train the new classification model locally using feature distillation and prototype enhancement strategies, and then upload the new local model to the server. The S4.3 server collects the new local model, performs aggregation operations, and obtains the new global model; The S4.4 server fine-tunes the new global model classifier based on the shared dataset S, and then distributes the fine-tuned new global model to users for the next round of local training. S4.5 repeats S4.2-S4.4, iteratively updating the new global model until performance convergence, thus obtaining the new model; S5 users calculate the new class prototype and the enhancement of the new class prototype, use the new class prototype to select new class sample data, and upload the new class prototype and new class sample data to the server. The S6 server aggregates the new class prototypes to obtain the new class global class prototype, supplementing the global class prototype set {e} mentioned in S3. c The server collects sample data of the new class to supplement the shared dataset S, and at the same time, the server updates the global class prototype of the old class based on the shared dataset.

2. The federated class incremental learning method based on feature distillation and class prototype as described in claim 1, characterized in that: Step S1.2 describes the initial local model using the local loss function L. local Local training is performed, and the local loss function L local By cross-entropy loss function L CE constitute: Where f represents the feature extractor and θ represents the classifier. This represents the local dataset used for the first time to receive the task. For the local dataset Size, p c It is the probability that the model predicts a sample as class c. This is an indicator function; if · is true, it takes the value 1; otherwise, it takes the value 0.

3. The federated class incremental learning method based on feature distillation and class prototype as described in claim 1, characterized in that: Step S1.3 describes the initial global model ω t,r The calculation process for the new global model described in step 4.3 is the same, wherein the initial global model ω t,r : in, For the local dataset Size, 4. The federated class incremental learning method based on feature distillation and class prototype as described in claim 1, characterized in that: The class prototype described in step S2 The process is the same as the new class prototype calculation process described in step S5, wherein the class prototype The calculation is as follows: Input the data of each category into the initial model ω respectively. t Calculate the mean of the local feature vectors: in, The set of category c in the local dataset; The prototype enhancement amplitude described in step S2 The calculation process for the new class prototype enhancement magnitude described in step S5 is the same, wherein the prototype enhancement magnitude... The average class variance of the local feature vectors is obtained by calculating: Where d is the dimension of the feature space, It is the covariance matrix of the feature vectors output by category c, and Tr(·) represents the trace operation of the matrix. This is the set of all data categories of task t that user k encounters.

5. The federated class incremental learning method based on feature distillation and class prototype as described in claim 1, characterized in that: The method for selecting local sample data in step S2 and the method for selecting new type sample data in step S5 are the same, wherein the process for selecting local sample data is as follows: Users use the L2 norm to measure the local feature vector and class prototype of data belonging to category c. Analyze the similarity between them and sort them: The top M data points with the highest similarity are selected as the local sample data, where M is much smaller than the total number of data points in category c.

6. The federated class incremental learning method based on feature distillation and class prototype as described in claim 1, characterized in that: Step S3 Global Class Prototype e c The calculation process for the new class global prototype described in step S6 is the same, wherein the global class prototype e c :

7. The federated class incremental learning method based on feature distillation and class prototype as described in claim 1, characterized in that: The local loss function used in training the new classification model in step S4.1 is: L local L CE +L ft +L protoAug Among them, L CE It is the cross-entropy loss function, L ft It is the characteristic distillation loss, L protoAug It is the prototype enhancement loss; The cosine similarity is used to measure the difference between the features output by the old and new models, and is used as the feature distillation loss L. ft : The user sets the data volume N for the old category, uses Gaussian noise n ~ N(0,1), and the prototype of the old category to amplify the amplitude. Generate around the global class prototype e c Fluctuating pseudo-eigenvectors that conform to independent and identical distribution pseudo feature vectors Input the classifier and calculate the prototype enhancement loss:

8. The federated class incremental learning method based on feature distillation and class prototype as described in claim 1, characterized in that: The process of fine-tuning the new global model classifier in step S4.4 is as follows: The server uses the shared dataset to fine-tune the new global model classifier in each round: in This indicates that the feature extractor is frozen, |S| represents the size of the shared dataset, and [:|C old The |] symbol indicates that only the classifier weights related to the old category are updated.

9. The federated class incremental learning method based on feature distillation and class prototype as described in claim 1, characterized in that: Step S6, which involves updating the global class prototype of the old category, is as follows: First, the server selects data belonging to category c from the shared dataset. o ∈C old The sample data is input into the feature extractor f of the new model. t Feature extractor f of the old model t-1 The eigenvectors z are obtained respectively. i,t and z i,t-1 The relative drift of the eigenvectors is calculated as follows: Calculate the eigenvector z i,t-1 Compared to the old global class prototype The distance to the weight w is used to measure the relative drift of the feature vector. i : Where σ is the standard deviation of the Gaussian kernel; Using the drift mean of eigenvectors to approximate the global class prototype The semantic drift is: Update the global class prototype of the old category:

Images