Feature Prototype-Based Federated Incremental Learning Approach for Distributed Machine Learning

Abstract

This invention discloses a federated incremental learning method based on feature prototypes: A federated server formulates a task plan and selects participants for federated incremental learning according to task requirements; the federated server sends the task plan and global feature prototypes to the selected participants; each participant receives the training plan and global feature prototypes from the federated server, performs incremental learning using local data, and generates local feature prototypes for each category; each participant uploads their local feature prototypes to the federated server; the federated server generates a server-side global feature prototype through prototype aggregation; this is a single iteration of the federated incremental learning process. The federated server iteratively updates the global feature prototype by repeatedly executing the above process. This invention solves the model heterogeneity problem in federated incremental learning by sharing feature prototypes, reducing communication overhead, improving learning efficiency, and strengthening privacy protection for participants.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a feature-prototype-based federated incremental learning method for distributed machine learning. Background Technology

[0002] With the development of artificial intelligence technology, deep learning is increasingly being applied across various industries. However, in real-world applications, the need for data privacy protection prevents the sharing of raw data, creating "data silos" between multiple data sources and hindering centralized learning. Federated learning, primarily aimed at privacy protection, has thus been proposed. It is a distributed machine learning method where participants jointly learn by sharing model parameters rather than raw data, with the goal of training a model with stronger generalization capabilities. In actual federated learning, participants' data is constantly updated, leading to an increase in data categories. Therefore, the model needs continuous updates to adapt to new tasks. Federated incremental learning was proposed to address this problem of federated learning with increasing data categories.

[0003] Current federated incremental learning methods, like classic federated learning, perform joint learning by sharing neural network model parameters. This approach requires all participants to have identical model structures, making it unsuitable for federated incremental learning scenarios with model heterogeneity. Furthermore, sharing the entire network's parameters incurs significant communication overhead and compromises privacy. Therefore, how to perform joint learning in federated incremental learning scenarios with model heterogeneity has become a pressing issue. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a feature-prototype-based federated incremental learning method for distributed machine learning. This method implements federated learning based on data feature prototypes, which is more suitable for federated incremental learning with heterogeneous models compared to federated learning that utilizes neural network model parameters. Furthermore, this method uses feature prototypes as the communication subject for joint learning, reducing communication overhead and improving operational efficiency. Since feature-prototype-based federated incremental learning does not share participants' original data or network parameters, it offers stronger privacy protection.

[0005] The objective of this invention is achieved through the following technical solutions.

[0006] This invention provides a feature-prototype-based federated incremental learning method for distributed machine learning, comprising the following steps:

[0007] Step S100: Participant Selection

[0008] The federated server develops a task plan and selects participants for federated incremental learning based on task requirements.

[0009] Step S200: Task Configuration

[0010] The federation server sends the task plan and global feature prototype to the federation participants selected in step S100;

[0011] Step S300: Each federation participant receives its own training plan and global feature prototype from the task plan sent by the federation server in step S200, performs incremental learning using local data, and generates local feature prototypes for each category.

[0012] Step S400: Each federation participant uploads its local feature prototype to the federation server;

[0013] Step S500: After receiving the local feature prototypes uploaded by each federation participant in step S400, the federation server generates a global feature prototype on the server side through prototype aggregation.

[0014] The above steps constitute a single iteration of federated incremental learning. The federated server executes steps S100 to S500 in a loop, distributing the latest aggregated global feature prototype to each federated participant selected in the next loop, and iteratively updating the global feature prototype.

[0015] The task requirements in step S100, participant selection, include participant data, computing resources, network communication resources, and participant behavior records; the participant selection steps include:

[0016] Step S101: Select participants who meet the data requirements in the task plan based on the data distribution and characteristics of each participant;

[0017] Step S102: Based on each participant's computing resources, network communication resources, and behavioral records, score and rank the participants who have met the data requirements in the task plan;

[0018] Step S103: Determine the participants for this round of federated incremental learning based on the set number of participants and their scores.

[0019] The participant behavior record used as the basis for participant scoring in step S102 is the participation record of the federal participants in historical tasks, including the number of tasks participated in, the amount of computation of the tasks participated in, the number of parameters uploaded, and the test performance of the uploaded parameters.

[0020] The specific process of task configuration in step S200:

[0021] The federated server sends a task plan to the federated participants selected in step S100, including the categories of data involved in local learning, the number of training updates for local parameters, and the deadline for sending local feature prototypes to the server.

[0022] After a federated participant confirms their participation in the task, the federated server sends a global feature prototype to that participant. The feature prototype represents the feature distribution of a certain category of data and is used for classification tasks.

[0023] The specific steps of step S300 include:

[0024] Step S301: Each federated participant constructs the training dataset for the current task based on the received task plan;

[0025] Step S302: Each federation participant sets the hyperparameters for local incremental learning according to the received task plan and makes incremental adjustments to the local model;

[0026] Step S303: Each federation participant uses its local training dataset and global feature prototype to calculate the loss term, uses the backpropagation algorithm to calculate the gradient of the local model parameters, and uses the gradient descent method to update the local model parameters.

[0027] Step S304: Each federated participant generates a local feature prototype using the local model with updated parameters and the local training dataset.

[0028] The training dataset mentioned in step S301 includes two parts: old task data and current task data.

[0029] The old task data refers to the dataset that the federated participants have already learned locally. Due to storage space limitations, the federated participants retain a portion of the old task dataset by sampling to avoid catastrophic forgetting during the subsequent incremental learning training process.

[0030] The current task data refers to a dataset that the federated participants have not yet learned locally, and that contains new data categories.

[0031] In step S302, the task performed locally by each federation participant is incremental learning, and the learning task is a classification task. During the incremental learning process, the local model structure needs to be incrementally adjusted, especially the structure of the classifier, to increase the number of categories it can classify.

[0032] In step S303, when each federated participant updates the local model parameters using the gradient descent method, the loss term includes three terms:

[0033] The classification loss term is used to measure the accuracy of the local model in the current task.

[0034] The purpose of the distillation loss term is to measure the degree to which the local model has forgotten old tasks;

[0035] The prototype loss term is used to measure the difference between the local feature prototypes of federated participants and the global feature prototypes of federated servers.

[0036] The local models used by each federal participant during the training process are all neural network models, which include two parts: a feature extractor and a classifier. The function of the feature extractor is to map the raw data to the feature space and obtain more discriminative feature vectors. The classifier takes the feature vectors as input and the classification label of the sample as output.

[0037] In step S400, the feature dimensions of the local feature prototypes uploaded by each federated participant are the same, and the categories corresponding to the uploaded local feature prototypes are the same as the categories of the local training data of each federated participant.

[0038] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0039] (1) The present invention utilizes feature prototypes for joint learning, which is more adaptable to learning scenarios with model heterogeneity than joint learning methods that utilize neural network model parameters.

[0040] (2) In the joint learning, the present invention uses feature prototypes as the medium for transmitting information. Compared with the use of neural network models, it has lower communication overhead and higher operating efficiency.

[0041] (3) In this invention, the original data and model parameters are kept locally by the participants and are not shared, which can effectively improve the privacy protection of federated participants. Attached Figure Description

[0042] Figure 1 This is a flowchart of the feature prototype-based federated incremental learning method for distributed machine learning according to the present invention.

[0043] Figure 2 This is a schematic diagram of the local dataset of the federated participants in an embodiment of the present invention;

[0044] Figure 3 This is a flowchart illustrating the local training process for federated participants in an embodiment of the present invention.

[0045] Figure 4 This is a schematic diagram illustrating the calculation of the classifier loss term in an embodiment of the present invention. Detailed Implementation

[0046] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Federated learning is a distributed machine learning method that primarily involves two types of entities: a federated server and federated participants. The federated server defines the tasks for the federated learning and coordinates multiple federated participants to jointly train a machine learning model. The federated participants possess the data required for the machine learning task, but due to data privacy and security concerns, the original data cannot be shared, making it impossible to integrate the data from all federated participants and perform centralized machine learning.

[0048] Federated incremental learning is a special type of federated learning problem. Unlike classic federated learning, the data possessed by the federated participants is not static but dynamically changing. The types of data change over time, and due to limitations in the local storage resources of the federated participants, not all data can be retained; each learning iteration can only use the newly acquired data. Furthermore, since the number and types of data categories possessed by each federated participant may differ, the local models of each participant will differ in their classification tasks, resulting in heterogeneity in their model structures. Similar to classic federated learning, it also prioritizes data privacy for the federated participants; the original data of the federated participants is not allowed to be shared.

[0049] This invention presents a feature-prototype-based federated incremental learning method for distributed machine learning. It addresses the heterogeneity of local data and models among federated participants, adapts to the dynamic changes in local data, and protects the privacy of each participant during joint learning. Through efficient joint learning, it improves the performance of each participant's local model. Federated incremental learning is a multi-round iterative training process, where a single round of training is as follows: Figure 1 As shown, the specific steps include:

[0050] Step S100: Participant Selection

[0051] The federated server formulates a task plan and selects participants for federated incremental learning, referred to as "federated participants," based on the task requirements. These requirements include the participants' data, computing resources, network communication resources, and activity logs. Specific steps include:

[0052] Step S101: Select participants who meet the data requirements in the task plan based on the data distribution and characteristics of each participant.

[0053] Step S102: Based on each participant's computing resources, network communication resources, and behavioral records, score and rank the participants who have met the data requirements in the task plan. The participant behavioral records used for scoring are the historical task participation records of the federated participants, including the number of tasks participated in, the computational load of the tasks, the number of parameters uploaded, and the test performance of the uploaded parameters.

[0054] Step S103: Determine the participants for this round of federated incremental learning based on the set number of participants and their scores.

[0055] Step S200: Task Configuration

[0056] The federation server sends the task plan and global feature prototype to the federation participants selected in step S100. The specific process is as follows:

[0057] The federated server sends a task plan to the federated participants selected in step S100, including the categories of data involved in the participants' local learning, the number of training updates for local parameters, and the deadline for sending local feature prototypes to the server.

[0058] After a federated participant confirms their participation in the task, the federated server sends a global feature prototype to that participant. The feature prototype represents the feature distribution of a certain category of data and is used for classification tasks.

[0059] Step S300: Each federation participant receives its own training plan and global feature prototype from the task plan sent by the federation server in step S200, performs incremental learning using local data, and generates local feature prototypes for each category.

[0060] The specific steps include:

[0061] Step S301: Each federation participant constructs the training dataset for the current task based on the received task plan.

[0062] The training dataset consists of two parts: old task data and current task data.

[0063] The old task data refers to the dataset that the federated participants have already learned locally. Due to storage space limitations, the federated participants retain a portion of the old task dataset by sampling to avoid catastrophic forgetting during the subsequent incremental learning training process.

[0064] The current task data refers to a dataset that the federated participants have not yet learned locally, and that contains new data categories.

[0065] Step S302: Each federation participant sets the hyperparameters for local incremental learning according to the received task plan and makes incremental adjustments to the local model.

[0066] The task performed locally by each federation participant is incremental learning, and the learning task is classification. During the incremental learning process, the local model structure needs to be incrementally adjusted, especially the structure of the classifier, to increase the number of categories it can classify.

[0067] Each federation participant used a local neural network model during training, which consisted of a feature extractor and a classifier. The feature extractor mapped the raw data to a feature space to obtain more discriminative feature vectors. The classifier took the feature vectors as input and the classification labels of the samples as output. However, the neural network models of the various federation participants exhibited heterogeneity, with differences in model structure, particularly in the output dimensions of the classifiers.

[0068] Step S303: Each federated participant uses its local training dataset and global feature prototype to calculate the loss term, uses the backpropagation algorithm to calculate the gradient of the local model parameters, and uses the gradient descent method to update the local model parameters.

[0069] When each federated participant updates the local model parameters using gradient descent, the loss term includes three terms:

[0070] The classification loss term is used to measure the accuracy of the local model in the current task.

[0071] The purpose of the distillation loss term is to measure the extent to which the local model has forgotten old tasks.

[0072] The prototype loss term is used to measure the difference between the local feature prototypes of federated participants and the global feature prototypes of federated servers.

[0073] Step S304: Each federated participant generates a local feature prototype using the local model with updated parameters and the local training dataset.

[0074] Step S400: Each federation participant uploads its local feature prototype to the federation server.

[0075] Among them, the feature dimensions of the local feature prototypes uploaded by each federation participant are the same, and the categories corresponding to the uploaded local feature prototypes are the same as the categories of the local training data of each federation participant.

[0076] Step S500: After receiving the local feature prototypes uploaded by each federation participant in step S400, the federation server generates a global feature prototype on the server side through prototype aggregation. The prototype aggregation methods include simple averaging and weighted averaging.

[0077] The above steps constitute a single iteration of federated incremental learning. The federated server executes steps S100 to S500 in a loop, distributing the latest aggregated global feature prototype to each federated participant selected in the next loop, and iteratively updating the global feature prototype.

[0078] Example

[0079] In federated incremental learning, participants are individual users learning image classification. These participants may use various computing devices such as smartphones, tablets, and laptops. The image data stored locally by each participant is continuously and dynamically updated; new images containing new data categories are added, while older images are removed due to storage limitations. Each participant possesses different data categories, their local image classification models address different specific tasks, and their model structures, particularly the classifier structures, differ. Each participant has limited data and, to obtain a more generalized image classification model, they wish to leverage the data of other participants to enhance their own model's performance. However, due to data privacy concerns, the original data may contain private information about the participants; therefore, sharing the original data is not an option for utilizing other participants' data. In this scenario, this invention presents a feature-prototype-based federated incremental learning method for distributed machine learning, the specific process of which is as follows:

[0080] Step S101: Participant Selection. The federated server formulates a task plan and selects participants for federated incremental learning based on the task requirements. These task requirements include participant data, computing resources, network communication resources, and participant behavior records.

[0081] The data requirements in this embodiment include image size, resolution, number of channels, color depth, number of images, whether the images are labeled, and the category of the labels. Participants who meet the data requirements are screened by the server as candidates for the current task. The server scores the candidates' data requirement compliance based on the quantity and quality of the images, giving priority to participants with high-quality, large-scale data.

[0082] Efficiency is a crucial issue in federated learning, primarily encompassing the time overhead of local training for participants and the communication overhead between participants and the server. The cost of local training is mainly determined by the computing power of the participant's own device, which varies depending on the device type (CPU, GPU, TPU, etc.) and model. Training time is also related to local memory performance, such as internal memory frequency and communication bandwidth, and external memory read / write speed. The bottleneck in communication between participants and the server is determined by the participant's network bandwidth. Typically, in synchronous communication, system communication efficiency is determined by the slowest participant; therefore, selecting participants with greater network bandwidth is essential for the operational efficiency of the federated incremental learning system.

[0083] Participant behavior records are crucial for screening trustworthy federated participants. Suppose an attacker posing as a participant exists within the federated incremental learning system, potentially harming the reliability of global parameters through data poisoning and model poisoning. Data poisoning involves the attacker training the model with fake or erroneous data and uploading the resulting incorrect parameters to the server. Model poisoning involves the attacker directly generating and uploading incorrect parameters, such as feature prototypes with all zero values. When these erroneous parameters are aggregated on the server, they affect the global parameters, rendering them unusable by other participants. Therefore, identifying and preventing attackers is critical to the security of the federated incremental learning system. Participant behavior records document the historical tasks participated in by each federated participant, including the number of tasks participated in, the computational load of each task, the number of parameters uploaded, and the test performance of the uploaded parameters. By analyzing these behavior records, the federated server can assess the trustworthiness and contribution of participants, identifying attackers and removing them from the learning tasks to ensure their smooth progress.

[0084] Step S200: Task Configuration. The Federation server sends the task plan and global feature prototype to the federation participants selected in step S100.

[0085] The task plan includes the categories of data involved in the participants' local learning, the number of training and updating times for local parameters, and the deadline for sending local feature prototypes to the server. For example, in this instance, the training data type is image categories; the training method includes the optimizer used to update parameters (SGD, Adam, RMSprop, etc.), whether regularization terms (L1, L2) are used, whether Dropout is used and its parameters, the learning rate and its variation method, the number of local iterations, and the local batch size. The latest deadline for communication with the server requires participants to upload the updated local feature prototypes to the server; otherwise, they are considered to have abandoned this round of learning.

[0086] The global feature prototype refers to the feature prototype obtained by the server by aggregating the feature prototypes uploaded by each participant in previous learning tasks. If the current task is the first task, the global feature prototype does not exist. If the number of categories in the learned classification task is C and the dimension of the feature prototype is N, then the global parameters are C feature vectors of dimension N. Each feature vector is the feature prototype of the corresponding category, representing the feature of that category, and this feature can be used for classification tasks.

[0087] Step S300: Each federation participant receives its own training plan and global feature prototype from the task plan sent by the federation server in step S200, performs incremental learning using local data, and generates local feature prototypes for each category.

[0088] The federated participants construct the training dataset for the current task based on the received task plan. For example... Figure 2 As shown, the dataset used by participants in this round of learning consists of two parts: old task data and current task data. The participants' local data is as follows: Figure 2 As shown in the task sequence, the dataset of the old tasks that the participants have learned cannot be fully retained due to storage space limitations. Therefore, only a portion of representative samples are retained. As new data is constantly being added to the task sequence, the participant's local model does not use this data for training. The participant's local model needs to learn the data of the new tasks while maintaining its ability to classify the old tasks it has learned, thereby increasing the number of data categories it can classify.

[0089] There are two storage strategies for retaining data from older tasks. The first is a fixed total storage space, where the number of samples retained in each category decreases as the number of categories increases. The second is a fixed storage space for each category, where the storage overhead for retaining samples from older tasks increases as the number of categories increases. In both strategies, to avoid class imbalance, the number of samples retained in each category is the same.

[0090] The selection of representative samples is based on the distribution of samples in the feature space. First, the feature mean of all samples in a certain class is calculated. This mean represents the center of all samples in the feature space. Then, the distance between the feature vector of each sample in the class and the feature mean of the class is calculated. The samples are sorted in non-decreasing order of distance, and the top K samples are retained as representative samples of the class.

[0091] After constructing the training dataset for the current task, participants update their local models, such as... Figure 3As shown, the local model of the federated participants consists of two parts: a feature extractor and a classifier. The feature extractor maps the raw data to the feature space to obtain more discriminative feature vectors. The classifier takes the feature vectors as input and the classification labels of the samples as output. The participant model contains multiple classifiers, each corresponding to a learned classification task, and each classifier can perform multi-class classification tasks.

[0092] When participants update the local model parameters using the gradient descent method, their loss term includes three terms: classification loss term, distillation loss term, and prototype loss term.

[0093] The purpose of the classification loss term is to measure the accuracy of the model in the current task. In image classification tasks, the cross-entropy loss term is generally used. M is the number of samples, and N is the number of classes. The sign function is set to 1 if the true class of sample i is c, and 0 otherwise. This is the probability that sample i belongs to class c as predicted by the model. The formula for calculating cross-entropy loss is as follows:

[0094]

[0095] The purpose of the distillation loss term is to measure the extent to which the model has forgotten old tasks. This loss term uses a variant of cross-entropy loss in classification tasks. This represents the output of the classifier, which is a fully connected layer in an image classification model. There are N dimensions, each representing a category. and Let T and T represent the classifier's outputs for class k and class i, respectively. In the distillation loss, the parameter T represents the distillation temperature. This parameter amplifies the information of the misclassified classes, helping to retain more information. The purpose of using the distillation loss term is to amplify the loss on old task samples, thereby preserving information from the old task model and preventing catastrophic forgetting. The predicted probability for class k. The calculation formula is:

[0096]

[0097] The distillation loss term is labeled with the probability output by the old model, while the prediction label is the predicted probability output by the current model. From the perspective of preventing forgetting old tasks, the loss term constrains the output of the new model, reducing its difference from the old model. From the perspective of knowledge transfer, the distillation loss term distills knowledge from the old model into the new model. M is the number of samples, and N is the number of classes. This represents the predicted probability of sample i belonging to class c using the old model. This represents the predicted probability of sample i belonging to class c by the new model. The formula for calculating the distillation loss term is as follows:

[0098]

[0099] The calculation of distillation loss and classification loss is as follows: Figure 4 As shown, if there are H classifiers, including H-1 classifiers for old tasks and 1 classifier for the current task, then all classifiers need to calculate classification loss, totaling H terms, while the distillation loss term only applies to the classifiers for the old tasks that have been learned, totaling H-1 terms.

[0100] The purpose of the prototype loss term is to measure the difference between the participant's local feature prototype and the server's global feature prototype. This loss term uses Euclidean distance to calculate the distance between the local feature prototype and the server's global feature prototype. For the feature prototype of local category i of the federal participant Let be the global feature prototype for server category i. The formula for calculating the prototype loss term is as follows:

[0101]

[0102] Step S400: Each federation participant uploads the locally computed feature prototype to the server.

[0103] Federation participants update parameters locally and calculate feature prototypes for each category using local data before uploading these prototypes to the server. It is required that the feature vectors output by the feature extractors of each participant have the same dimension, so that the feature prototypes can be aggregated on the server later. Due to the heterogeneity of the local data of each participant, and the different categories, there are differences between the output dimensions of the classifiers and the corresponding categories, making aggregation impossible. Sharing feature prototypes with the same dimension can solve the heterogeneity problem of the model.

[0104] Step S500: After receiving the local feature prototypes uploaded by each federation participant in step S400, the federation server generates a global feature prototype on the server side through prototype aggregation.

[0105] The federated server receives local feature prototypes from each federated participant. Each participant uploads multiple local feature prototypes. The sets of prototype categories across different participants may contain prototypes of the same category or may not have any common categories. Therefore, during prototype aggregation, aggregation is performed on all prototypes of the same category. The aggregation method can be simple averaging or weighted averaging. The final aggregation result is that the federated server has N global feature prototypes, where N is the number of image categories, and each global feature prototype is C-dimensional.

[0106] After the aggregation is completed, the federated server can start the next cycle, re-execute steps S100 to S500, distribute the latest aggregated global feature prototype to each participant, and iteratively update the global feature prototype.

[0107] Although the functions and working processes of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific functions and working processes described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these are within the protection scope of the present invention.

Claims

1. A feature-prototype-based federated incremental learning method for distributed machine learning, characterized in that, Includes the following steps: Step S100: The federated server formulates a task plan and selects participants for federated incremental learning based on the participants' data, computing resources, network communication resources, and behavioral records. Step S200: The federated server sends a task plan and a global feature prototype to the selected federated incremental learning participants; the task plan includes the categories of data involved in local learning, the number of training and update times of local parameters, and the deadline for sending the local feature prototype to the server; the global feature prototype is a feature vector representing the feature distribution of each category of data generated by a neural network feature extractor. Step S300: After receiving the task plan, each federation participant constructs a training dataset containing old task data and current task data, sets local incremental learning hyperparameters and adjusts the classifier structure, uses local data and global feature prototypes, updates local model parameters through a triple loss function containing classification loss, distillation loss and prototype loss, and generates local feature prototypes for each category. Step S400: Each federation participant uploads its local feature prototype to the federation server, wherein the local feature prototype has a uniform feature dimension. Step S500: After receiving the local feature prototypes uploaded by each federation participant, the federation server generates a new global feature prototype through prototype aggregation. The above steps constitute a single iteration of federated incremental learning. The federated server executes steps S100 to S500 in a loop, distributing the latest aggregated global feature prototype to each federated participant selected in the next loop, and iteratively updating the global feature prototype.

2. The feature-prototype-based federated incremental learning method for distributed machine learning according to claim 1, characterized in that, The participant selection step in step S100 includes: Step S101: Select participants who meet the data requirements in the task plan based on the data distribution and characteristics of each participant; Step S102: Based on each participant's computing resources, network communication resources, and behavioral records, score and rank the participants who have met the data requirements in the task plan; Step S103: Determine the participants for this round of federated incremental learning based on the set number of participants and their scores.

3. The feature-prototype-based federated incremental learning method for distributed machine learning according to claim 2, characterized in that, The participant behavior record used as the basis for participant scoring in step S102 is the participation record of the federal participants in historical tasks, including the number of tasks participated in, the amount of computation of the tasks participated in, the number of parameters uploaded, and the test performance of the uploaded parameters.

4. The feature-prototype-based federated incremental learning method for distributed machine learning according to claim 1, characterized in that, The specific process of task configuration in step S200: The federated server sends a task plan to the federated participants selected in step S100, including the types of data involved in local learning, the number of training updates for local parameters, and the deadline for sending local feature prototypes to the server. After a federated participant confirms their participation in the task, the federated server sends a global feature prototype to that participant. The feature prototype represents the feature distribution of a certain category of data and is used for classification tasks.

5. The feature-prototype-based federated incremental learning method for distributed machine learning according to claim 1, characterized in that, The specific steps of step S300 include: Step S301: Each federated participant constructs the training dataset for the current task based on the received task plan; Step S302: Each federation participant sets the hyperparameters for local incremental learning according to the received task plan and makes incremental adjustments to the local model; Step S303: Each federation participant uses its local training dataset and global feature prototype to calculate the loss term, uses the backpropagation algorithm to calculate the gradient of the local model parameters, and uses the gradient descent method to update the local model parameters. Step S304: Each federated participant generates a local feature prototype using the local model with updated parameters and the local training dataset.

6. The feature-prototype-based federated incremental learning method for distributed machine learning according to claim 5, characterized in that, The training dataset mentioned in step S301 includes two parts: old task data and current task data. The old task data refers to the dataset that the federated participants have already learned locally. Due to storage space limitations, the federated participants retain a portion of the old task dataset by sampling to avoid catastrophic forgetting during the subsequent incremental learning training process. The current task data refers to a dataset that the federated participants have not yet learned locally, and that contains new data categories.

7. The feature-prototype-based federated incremental learning method for distributed machine learning according to claim 5, characterized in that, In step S302, the task performed locally by each federated participant is incremental learning, and the learning task is a classification task. During the incremental learning process, the local model structure needs to be incrementally adjusted.

8. The feature-prototype-based federated incremental learning method for distributed machine learning according to claim 5, characterized in that, In step S303, when each federated participant updates the local model parameters using the gradient descent method, the loss term includes three terms: The classification loss term is used to measure the accuracy of the local model in the current task. The purpose of the distillation loss term is to measure the degree to which the local model has forgotten old tasks; The prototype loss term is used to measure the difference between the local feature prototypes of federated participants and the global feature prototypes of federated servers.

9. The feature-prototype-based federated incremental learning method for distributed machine learning according to claim 5, characterized in that, The local models used by each federal participant during the training process are all neural network models, which include two parts: a feature extractor and a classifier. The function of the feature extractor is to map the raw data to the feature space and obtain more discriminative feature vectors. The classifier takes the feature vectors as input and the classification label of the sample as output.

10. The feature-prototype-based federated incremental learning method for distributed machine learning according to claim 1, characterized in that, In step S400, the feature dimensions of the local feature prototypes uploaded by each federated participant are the same, and the categories corresponding to the uploaded local feature prototypes are the same as the categories of the local training data of each federated participant.

Images