Federated learning method and device based on feature comparison optimization and classifier dynamic integration

Abstract

The present invention belongs to the technical field of federated learning, and specifically relates to a federated learning method and device based on feature contrast optimization and dynamic integration of classifiers. The method includes: initializing a global model through a server and sending it to individual clients participating in federated learning, the client updates the local model based on the received global model and trains the updated local model, and then uses the trained local feature extractor to extract features from its local private data set to construct a local feature prototype set, and then uploads the local model, the local feature prototype set, and the total sample size of the client to the server. When the server aggregates globally, it uses contrast learning technology to improve the quality of the global feature prototype, and finally sends the aggregated global prototype and the global feature prototype set to each client, and performs the next round of learning until the local model converges or reaches the set communication round.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning, and more specifically, relates to a federated learning method and device based on feature comparison optimization and classifier dynamic integration. Background Art

[0002] With the rapid development of artificial intelligence, a large number of models that can cope with complex tasks have emerged. These models have penetrated into all aspects of people's lives and promoted social progress. The successful implementation of these models is inseparable from the support of massive data, which is the key for the model to learn effective knowledge. However, there are currently the following problems: Due to the continuous upgrading of demand, people are increasingly hoping to customize models that can excellently complete personalized tasks and have strong generalization capabilities when facing emergencies. Personalized tasks often require more private data. If a data-centralized approach is used for model training, privacy may be leaked. At the same time, even if data desensitization is performed, it may be attacked by joint inference and sensitive data may be recovered.

[0003] In order to solve this contradiction, federated learning came into being. Its core concept is to train data locally, avoid uploading data to a central server, and reduce the risk of privacy leakage. In this way, each data owner (client) trains the model locally and uploads the model to the server. The server aggregates these models and finally obtains a global model, which is then sent to all data owners.

[0004] The above method achieves model integration and knowledge sharing while protecting privacy. However, when there are large differences between the data of federated learning clients, that is, when data heterogeneity occurs, the traditional federated learning method that ultimately outputs a unified model performs poorly due to the offset of the training targets of each client. The personalized federated learning method that hopes to improve the generalization performance of the model while excellently completing local personalized tasks and outputs customized models for each client has shown certain advantages.

[0005] The personalized federated learning method based on model decoupling obtains the global prototype by uploading the local prototype (i.e., the centroid of the feature vector) to the server for simple aggregation, and uses the global prototype to guide the training of the local feature extractor. At the same time, the personalized federated learning method personalizes the classifier, which alleviates the data heterogeneity problem to a certain extent.

[0006] For example, Chinese patent document CN119005302A discloses a federated learning method based on a flexible combination of feature extractors and classifiers, which includes: the server initializes the global model; if it is the first round, the local classifier is updated to the global classifier, if not, the local classifier is allowed to use the local classifier trained in the previous round; the local classifier is trained, and then the local feature extractor is trained. The global feature extractor generates a global feature anchor to guide the training of the local feature extractor, and the local feature extractor is first combined with the global classifier for training, and then combined with the local classifier for training; based on the number of client samples, the aggregation weight is generated, and the local model is aggregated to obtain a new round of global model; the steps are repeated until the model converges or reaches the set communication round. The present invention solves the problems of model performance degradation and privacy leakage.

[0007] Also, Chinese patent document CN116579443A discloses a personalized federated learning method and storage medium for data heterogeneity. The present invention only requires each client to disclose the model classifier gradient information of its local personalized model, which can alleviate the problem of privacy leakage. At the same time, clustering based on the model classifier gradient information can more accurately identify clients with similar data distribution, reduce the computational cost, and construct the client's local optimization target by an aggregate model that is more valuable than the global model, so that the client can obtain a personalized model with better generalization performance and convergence.

[0008] However, existing methods still have significant limitations: First, simple local prototype aggregation requires knowing the amount of data for each client sample category, which may expose the client's data distribution and bring the risk of privacy leakage. In addition, this aggregation method cannot fully cope with changes in various data heterogeneity situations. Although it can ensure that the global prototype correctly reflects the local prototype in most cases, it does not optimize the separation between global prototypes of different categories, which may cause confusion of feature vectors of similar but different categories. Secondly, although the fully personalized classifier can achieve good performance on local tasks, its generalization ability is poor. Because it fails to fully utilize the consensus of the global model, when faced with new tasks that are different from the current task, the fully personalized classifier often has difficulty coping and shows low applicability.

[0009] Based on the above analysis, although the existing model decoupling personalized federated learning method has made some progress in alleviating the problem of data heterogeneity, it still has strong limitations. How to improve model performance while ensuring privacy is still a topic worthy of in-depth study. Summary of the invention

[0010] The present invention aims to overcome at least one defect of the above-mentioned prior art and provide a federated learning method and device based on feature comparison optimization and dynamic integration of classifiers to achieve data privacy protection and model performance improvement.

[0011] The detailed technical scheme of the present invention is as follows:

[0012] A federated learning method based on feature contrast optimization and classifier dynamic integration, the method comprising:

[0013] S1. Initialize the global model through the server and send it to the participants in federated learning Clients;

[0014] S2. The client updates the local model based on the received global model, where:

[0015] When performing the first local model update, the client initializes the local model to the global model it received;

[0016] In progress When the local model is updated, , the client updates the local feature extractor in the local model to the global feature extractor in the global model it has recently received; and, the client calculates the local adaptability weight of the global classifier in the global model it has recently received under the current update round, and integrates the local classifier and the global classifier based on this weight to update the local classifier;

[0017] S3. The client trains the updated local model in the order of training the local classifier first and then the local feature extractor, where:

[0018] When performing the first local model training, the local classifier and the local feature extractor are trained using cross entropy loss;

[0019] In progress When training the local model, , training the local classifier using a cross entropy loss, and, simultaneously training the local feature extractor using a cross entropy loss and a global prototype loss;

[0020] S4. The client uses the trained local feature extractor to perform feature extraction on its local private data set to calculate the local feature prototypes corresponding to various feature vectors, and composes the local feature prototypes into a local feature prototype set, and then uploads the local model and the local feature prototype set to the server; and, in the first round of federated learning, uploads the total sample size of the client to the server at the same time;

[0021] S5. The server sets the local model aggregation weight based on the total number of client samples received, and uses this weight to aggregate the local model to obtain the global prototype; and, based on the comparison score, aggregates the local feature prototype set it receives to generate an average feature prototype set, and then compares and optimizes the average feature prototype set to obtain the global feature prototype set; sends the aggregated global prototype and global feature prototype set to each client, and executes the next round of learning until the local model converges or reaches the set communication round.

[0022] Preferably, according to the present invention, step S2 specifically includes:

[0023] When performing the first local model update, the client initializes the local model to the received global model, that is: , Indicates an update operation. Indicates The local model after the first local update of the client, represents the initialized global model;

[0024] And use the global model The global feature extractor updates the local feature extractor of the local model, namely: , Indicates The local feature extractor after the first local update of the client, Represents the initialized global model A global feature extractor for

[0025] And, using the global model The global classifier updates the local classifier of the local model, that is: , Indicates The local classifier after the first local update of the client, Represents the initialized global model The global classifier.

[0026] Preferably, according to the present invention, step S2 specifically further includes:

[0027] In progress When the local model is updated, , for the local feature extractor of the local model, update it to the latest received global model The global feature extractor in is: , Indicates Client No. The local feature extractor after the local update, Represents the global model A global feature extractor for

[0028] For the local classifier of the local model, calculate the global model The local adaptability weight of the global classifier in the current round :

[0029]

[0030] In formula (1), Indicates Client No. The local adaptability weight in the local update, Indicates category index The Sample classes, represents the cross entropy loss, Indicates Client No. The local classifier after local training, represents the local classification function, Indicates that the global feature prototype Passed to the local classifier The classification prediction results obtained later are Indicates The global classifier after sub-global aggregation, represents the global classification function, Indicates that the local feature prototype Passed to the global classifier The classification prediction results obtained Indicates The label vector corresponding to the sample class;

[0031] Based on the local adaptability weight Performing a local classifier and the global classifier Integrations:

[0032]

[0033] In formula (2), Indicates an update operation. Indicates Client No. The local classifier after the local update, Indicates Client No. After the local training, The local classifier before the local update, Indicates Global classifier after sub-global aggregation.

[0034] Preferably, according to the present invention, in step S3, for the local classifier, only the cross entropy loss is used when performing local model training. To construct the loss function for training, namely:

[0035]

[0036] In formula (3), Indicates A local classifier trained by the client, Indicates that the gradient of the local classifier parameters is calculated. At this time, the parameters of the local feature extractor are frozen and no longer modified; Indicates that it belongs to the dataset Mini-batch datasets;

[0037] For the local feature extractor, the cross entropy loss is used when training the first local model. To construct the loss function for training, namely:

[0038]

[0039] In formula (4), Indicates The local feature extractor of each client after the first local training, Indicates The local feature extractor after the first local update of the client, Indicates that the gradient of the local feature extractor parameters is calculated. At this time, the parameters of the local classifier are frozen and no longer modified; Indicates The local classifier of each client after the first local training;

[0040] For the local feature extractor, When training the local model, , using the cross entropy loss While adding the global prototype loss Conduct training, namely:

[0041] (5);

[0042] (6);

[0043] In formula (5) and (6), represents the global prototype loss, represents the L2 norm, Indicates that the sample Enter to Client No. Local feature extractor after local update The feature vector generated in Indicates The first round of local training used The global trait prototype of the class, For sample The label of the corresponding class, Indicates Client No. Local feature extractor after local training.

[0044] Preferably, in step S4, the local feature prototypes corresponding to each type of feature vector are calculated, and the local feature prototypes are combined into a local feature prototype set, specifically:

[0045] (7);

[0046] (8);

[0047] In formula (7) and (8), Indicates Client No. After a round of local training, the The local feature prototype of the class, Indicates The client data set belongs to The samples of the class, Indicates The client data set belongs to The labels of samples of each class, Indicates that the sample Enter to Client No. Local feature extractor after sub-local training The feature vector generated in Indicates The client data set belongs to A sample dataset of classes, Indicates The client data set belongs to The size of the sample dataset for each class, Indicates The local feature prototype set consists of the local feature prototypes of each class of the client.

[0048] Preferably, according to the present invention, in step S5, the local model aggregation weight is:

[0049] (9);

[0050] In formula (9), represents the local model aggregation weight, Indicates the total sample size of the client. Indicates Client datasets size;

[0051] The global prototype obtained by aggregation is:

[0052] (10);

[0053] In formula (10), Indicates The global prototype obtained after the second global aggregation, Indicates Client No. The local model obtained after this training.

[0054] Preferably, in step S5, the received local feature prototype sets are aggregated based on the comparison scores to generate an average feature prototype set, specifically:

[0055] (11);

[0056] (12);

[0057] (13);

[0058] (14);

[0059] In formulas (11)-(14), Represents the local feature prototype The results after aggregation according to the sample size are The global feature prototype of each class, Indicates that it belongs to The global feature prototype of each class, Indicates Client No. The class in The comparison score when rounding, Indicates The class in The corresponding average feature prototype in the round aggregation, represents the average feature prototype set;

[0060] The average feature prototype set is compared and optimized to obtain the global feature prototype set:

[0061] (15);

[0062] (16);

[0063] (17);

[0064] In formulas (15)-(17), It represents the intermediate state from the average feature prototype to the global feature prototype. Indicates The global feature prototype corresponding to the class, represents the gradient associated with the feature prototype, Represents the average feature prototype set Belong to the The average feature prototype of each class, Represents a global feature prototype set.

[0065] In another aspect of the present invention, a device for implementing a federated learning method based on feature contrast optimization and classifier dynamic integration is provided, the device comprising a server and clients, of which:

[0066] The server is used to initialize the global model and send it to the participating federated learning clients; and, based on the total number of client samples received, setting a local model aggregation weight, and aggregating the local model with this weight to obtain a global prototype; aggregating the received local feature prototype set based on the comparison score to generate an average feature prototype set, and then performing comparison optimization on the average feature prototype set to obtain a global feature prototype set; sending the aggregated global prototype and global feature prototype set to each client, and executing the next round of learning until the local model converges or reaches the set communication round;

[0067] The client is used to update the local model based on the received global model, wherein: when performing the first local model update, the client initializes the local model to the global model it received; when performing the second local model update, the client initializes the local model to the global model it received; When the local model is updated, , the client updates the local feature extractor in the local model to the global feature extractor in the global model it has recently received; and, the client calculates the local adaptability weight of the global classifier in the global model it has recently received under the current update round, and integrates the local classifier and the global classifier based on this weight to update the local classifier;

[0068] And, for training the updated local model, and the training is performed in the order of first training the local classifier and then training the local feature extractor, wherein: when performing the first local model training, the local classifier and the local feature extractor are trained using the cross entropy loss; when performing the second local model training, the local classifier and the local feature extractor are trained using the cross entropy loss; When training the local model, , training the local classifier using a cross entropy loss, and, simultaneously training the local feature extractor using a cross entropy loss and a global prototype loss;

[0069] And, it is used to use the trained local feature extractor to extract features from its local private data set to calculate the local feature prototypes corresponding to each type of feature vector, and to form the local feature prototypes into a local feature prototype set, and then upload the local model and the local feature prototype set to the server; in the first round of federated learning, the total sample size of the client is uploaded to the server at the same time.

[0070] In another aspect of the present invention, there is also provided an electronic device, comprising:

[0071] at least one processor; and

[0072] A memory storing instructions, which, when executed by the at least one processor, enables the at least one processor to perform the above-mentioned federated learning method based on feature contrast optimization and dynamic integration of classifiers.

[0073] In another aspect of the present invention, a computer-readable storage medium is provided, which stores executable instructions, and when the instructions are executed, the machine executes the federated learning method based on feature comparison optimization and dynamic integration of classifiers as described above.

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] (1) The federated learning method based on feature comparison optimization and classifier dynamic integration provided by the present invention does not require each client to upload the data volume of each sample class when performing global feature prototype aggregation, thereby making it impossible for the server to obtain the client data distribution, thereby achieving better data privacy protection.

[0076] (2) The present invention adopts contrastive learning for the global feature prototype, so that the distance between the global feature prototype and the local feature prototype of the same class is as close as possible, and the distance between the global feature prototypes of different classes is as far as possible, thereby obtaining a higher quality feature prototype and achieving more accurate guidance for the local feature extractor.

[0077] (3) The present invention integrates the local classifier and the global classifier to help obtain a model that balances generalization and individuality. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a flow chart of the federated learning method based on feature comparison optimization and dynamic integration of classifiers described in the present invention.

[0079] Figure 2 It is a schematic diagram of the implementation process of the federated learning method described in Example 1 of the present invention.

[0080] Figure 3 The figure is a graph of experimental results of local task effectiveness testing of the feature contrast optimization component on the CIFAR-10 dataset.

[0081] Figure 4 The figure is a graph of experimental results of local task effectiveness testing of the dynamic integration component of the classifier on the CIFAR-10 dataset.

[0082] Figure 5 The figure is a graph of the experimental results of the generalization performance test of the dynamic integration component of the classifier on the CIFAR-10 dataset. DETAILED DESCRIPTION

[0083] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0084] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0085] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0086] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0087] In view of the shortcomings in the prior art, the present invention provides a federated learning method based on feature contrast optimization and dynamic integration of classifiers, which splits the entire model into two parts: a feature extractor and a classifier, and adopts different processing strategies; at the same time, in order to obtain a high-quality feature space, local feature prototypes are acquired, and during global aggregation, contrastive learning technology is used to improve the quality of global feature prototypes; in the local update and training stage, the global feature prototype is used to constrain the training of the local feature extractor, and the local classifier and the global classifier are integrated.

[0088] The present invention has broad application potential in many fields, especially in scenarios that require processing distributed, heterogeneous data. For example, in the field of medical health, the present invention can build a personalized disease prediction model that can adapt to the characteristics of different regions and different medical institutions by training patient data from multiple hospitals or clinics. Since the patient population and data distribution of each hospital are different, traditional federated learning methods often cannot effectively capture these differences, thus affecting the accuracy of the model. The present invention refines and optimizes the globally shared model with local data to ensure that the model of each hospital can not only utilize the knowledge of global data, but also fully consider the characteristics of local patient data, ultimately achieving higher accuracy.

[0089] The following is a further description of the federated learning method and device based on feature comparison optimization and dynamic integration of classifiers of the present invention in conjunction with specific embodiments.

[0090] Example 1

[0091] Ginseng Figure 1 This embodiment provides a federated learning method based on feature comparison optimization and classifier dynamic integration, the method comprising:

[0092] S1. Initialize the global model through the server , and send it to the participants in federated learning Clients.

[0093] In the present invention, a A federated learning scenario in which there are 3 clients and 1 server, and information can be uploaded and sent normally between the client and the server.

[0094] The following are the terms and definitions used in the present invention.

[0095] For the convenience of description, the following uses clients (i.e., the subscript in the parameter ), No. After the local update and training, local updates, before training (i.e., the superscript ) is introduced as an example, and the rest of the clients and rounds are the same as the example introduced.

[0096] The client has a local private dataset , where the specific categories are represented by , the category index is represented as , the total number of categories is expressed as , each sample is exists in the form of Indicates that it belongs to the dataset A small batch of data sets, Indicates the size of the dataset.

[0097] The client has a local model that has been trained. and the latest received global model , the initial global model is .

[0098] The following introduces the feature extractor and classifier. Client, After the local update and training, Take the local update and before training as an example, and the same applies to other clients and rounds.

[0099] In the present invention, the global model and the local model have the same structure, using ' ' represents the relevant parameters, that is, the local model currently trained is , the global model is ; and use the last layer of the model as a classifier, using Represents the relevant parameters, that is, the local classifier is , the global classifier is ; The combination of the remaining layers is used as a feature extractor, using ' ' represents the relevant parameters, that is, the local feature extractor is , the global feature extractor is .

[0100] At the same time, the feature extraction function and classification function composed of feature extractor and classifier are defined, where the local feature extraction function is , the global feature extraction function is , the local classification function is , the global classification function is .

[0101] The functions of the above two functions are as follows: For a data sample , firstly, feature extraction is performed to obtain feature vectors. Taking the local model as an example, that is, ; Then the feature vector is passed through the classification function to output the final classification decision, that is, .

[0102] S2. The client updates the local model based on the received global model, where:

[0103] When performing the first local model update, the client initializes the local model to the global model it received. ;

[0104] In progress When the local model is updated, , the client updates the local feature extractor in the local model to its latest received global model The global feature extractor in ; and the client computes its latest received global model The local adaptability weight of the global classifier in the current update round , and based on this weight, the local classifier and the global classifier are integrated to update the local classifier.

[0105] In this embodiment, a strategy of updating the local feature extractor and the local classifier separately is adopted.

[0106] Specifically, the Take a client as an example. During the first local model update, the client initializes the local model to the received global model, that is: , Indicates an update operation. Indicates The local model after the first local update of a client.

[0107] For local feature extractors of local models, use the global model The global feature extractor is updated, namely: , Indicates The local feature extractor after the first local update of the client, Represents the initialized global model A global feature extractor.

[0108] For local classifiers of local models, use the global model The global classifier is updated, namely: , Indicates The local classifier after the first local update of the client, Represents the initialized global model The global classifier.

[0109] If this is not the first local model update, continue with For example, in the first When the local model is updated, , for the local feature extractor of the local model, it will be updated to the latest received global model The global feature extractor in is: , Indicates Client No. The local feature extractor after the local update, Represents the global model A global feature extractor.

[0110] For the local classifier of the local model, the global model needs to be calculated The local adaptability weight of the global classifier in the current round , and based on this weight, a local classifier and the global classifier to update the local classifier.

[0111] The ensemble weights are based on the local adaptability of the global classifier, i.e., the cross entropy loss of the global classifier on the local feature prototype is calculated, where the definition of the local feature prototype is described in detail in S4; and the global generalization ability of the local classifier, i.e., the cross entropy loss of the local classifier on the global feature prototype is calculated, where the definition of the global feature prototype is described in detail in S5; then:

[0112]

[0113] In formula (1), Indicates Client No. The local adaptability weight in the local update, Indicates category index The Sample classes, represents the cross entropy loss, Indicates Client No. The local classifier after local training, represents the local classification function, Indicates that the global feature prototype Passed to the local classifier The classification prediction results obtained later are Indicates The global classifier after sub-global aggregation, represents the global classification function, Indicates that the local feature prototype Passed to the global classifier The classification prediction results obtained later are Indicates The label vector corresponding to the sample class.

[0114] Then, based on this weight Performing a local classifier and the global classifier The ensemble combination (i.e., local classifier update) can be expressed as:

[0115]

[0116] In formula (2), Indicates an update operation. Indicates Client No. The local classifier after the local update, Indicates Client No. After the local training, The local classifier before the local update, Indicates Global classifier after sub-global aggregation.

[0117] It should be understood that the above update process is similar for other clients and rounds, and will not be repeated here.

[0118] S3. The client trains the updated local model in the order of training the local classifier first and then the local feature extractor, where:

[0119] When performing the first local model training, the local classifier and the local feature extractor are trained using cross entropy loss;

[0120] In progress When training the local model, , the local classifier is trained using cross entropy loss, and the local feature extractor is trained using cross entropy loss and global prototype loss simultaneously.

[0121] After the local update phase is completed, the local training phase begins. The local feature extractor and the local classifier in the local model are trained separately. Meanwhile, this embodiment adopts the strategy of training the local classifier first and then training the local feature extractor.

[0122] Specifically, the Client, After the local update and training, Take local update and before training as an example.

[0123] In this embodiment, the training of the local classifier is only subject to the cross entropy loss That is, for the local classifier, after the first local model training and the Cross entropy loss is used in local model training. To construct the loss function for training:

[0124]

[0125] In formula (3), Indicates A local classifier trained by the client, Indicates that the gradient of the local classifier parameters is calculated. At this time, the parameters of the local feature extractor are frozen and no longer modified; Indicates that it belongs to the dataset Small batch dataset.

[0126] For the local feature extractor, the cross entropy loss is used when training the first local model. To construct the loss function and carry out training, that is:

[0127]

[0128] In formula (4), Indicates The local feature extractor of each client after the first local training, Indicates The local feature extractor after the first local update of the client, Indicates that the gradient of the local feature extractor parameters is calculated. At this time, the parameters of the local classifier are frozen and no longer modified; Indicates The local classifier of each client after the first local training.

[0129] For the local feature extractor, in non-first round training ( When training the local model, ), when using the cross entropy loss At the same time, join with the global feature prototype The relevant loss (global prototype loss) is used for training, namely:

[0130] (5);

[0131] (6);

[0132] In formula (5) and (6), represents the global prototype loss, represents the L2 norm, Indicates that the sample Enter to Client No. Local feature extractor after local update The feature vector generated in Indicates The first round of local training used The global trait prototype of the class, For sample The label of the corresponding class, Indicates Client No. Local feature extractor after local training.

[0133] It should be understood that the above training process is similar for other clients and rounds, and will not be repeated here.

[0134] S4. The client uses the trained local feature extractor to perform feature extraction on its local private data set to calculate the local feature prototypes corresponding to each type of feature vector, and organizes the local feature prototypes into a local feature prototype set, and then uploads the local model and the local feature prototype set to the server; and, in the first round of federated learning, uploads the total sample size of the client to the server at the same time.

[0135] Specifically, the Client, After the local update and training, Take local update and before training as an example.

[0136] Use the trained local feature extractor , for local private datasets Perform feature extraction, generate feature vectors for each class, and obtain the corresponding local feature prototype (local feature centroid), and then combine the local feature prototypes into a local feature prototype set, namely:

[0137] (7);

[0138] (8);

[0139] In formula (7) and (8), Indicates Client No. After a round of local training, the The local feature prototype of the class, Indicates The client data set belongs to The samples of the class, Indicates The client data set belongs to The labels of samples of each class, Indicates that the sample Enter to Client No. Local feature extractor after sub-local training The feature vector generated in Indicates The client data set belongs to A sample dataset of classes, Indicates The client data set belongs to The size of the sample dataset for each class, Indicates The local feature prototype set consists of the local feature prototypes of each class of the client.

[0140] Finally, the local model (composed of local feature extractor and local classifier), local feature prototype set and total client sample size are uploaded to the server. Among them, the total client sample size is uploaded only when the client uploads information to the server for the first time.

[0141] S5. The server sets the local model aggregation weight based on the total sample size received from the client, and aggregates the local model with this weight to obtain the global prototype; and, aggregates the local feature prototype set it receives based on the comparison score to generate an average feature prototype set, and then compares and optimizes the average feature prototype set to obtain the global feature prototype set.

[0142] On the server, assuming that Secondary server aggregation, First, obtain the local model, local feature prototype set and total sample size uploaded by the client, set the local model aggregation weight according to the total sample size of the client, and perform local model aggregation to obtain the global prototype.

[0143] Among them, the local model aggregation weight can be expressed as:

[0144] (9);

[0145] In formula (9), represents the local model aggregation weight, Indicates the total sample size of the client. Indicates Client datasets size.

[0146] The aggregated global prototype can be expressed as:

[0147] (10);

[0148] In formula (10), Indicates The global prototype obtained after the second global aggregation, Indicates Client No. The local model obtained after this training.

[0149] For the local feature prototype set, first calculate the average feature prototype set based on the comparison score, that is:

[0150] (11);

[0151] (12);

[0152] (13);

[0153] (14);

[0154] In formulas (11)-(14), Represents the local feature prototype The results after aggregation according to the sample size are The global feature prototype of each class, Indicates that it belongs to The global feature prototype of each class, Indicates Client No. The class in The comparison score when rounding, Indicates The class in The corresponding average feature prototype in the round aggregation, Represents the average feature prototype set.

[0155] Then, based on the average feature prototype set, comparative optimization is carried out to obtain the global feature prototype set, namely:

[0156] (15);

[0157] (16);

[0158] (17);

[0159] In formulas (15)-(17), It represents the intermediate state from the average feature prototype to the global feature prototype. Indicates The global feature prototype corresponding to the class, represents the gradient associated with the feature prototype, Represents the average feature prototype set Belong to the The average feature prototype of each class, Represents a global feature prototype set.

[0160] Finally, the aggregated global prototype and global feature prototype set are sent to each client, and the next round of learning is performed until the local model converges or reaches the set communication round.

[0161] That is, after the aggregation is completed, the aggregated global prototype and global feature prototype set are sent to each client, and the above operation is repeated to perform the next round of learning until the local model converges or reaches the set communication round.

[0162] The effectiveness of the present invention is verified by combining specific experiments below.

[0163] The experimental configuration and experimental results of the present invention include the following:

[0164] A. Dataset and model settings.

[0165] The present invention conducts experiments on the CIFAR-10 dataset, which consists of 60,000 32x32 pixel color images belonging to 10 different categories, of which 50,000 are used for training and 10,000 are used for testing. In addition, the present invention uses a model composed of three layers of convolutional networks, wherein the number of filters in each layer is 16, 32, and 64, respectively. The sizes of the convolution kernels are 5x5, 5x5, and 3x3, respectively, and the activation function uses ReLU, and the maximum pooling operation is combined to perform feature dimensionality reduction. At the same time, the model obtains the final prediction result through a fully connected layer containing 128 neurons and an output layer with a dimension of 10.

[0166] B. Training setup and data splitting.

[0167] During local training, the model feature extractor and classifier are trained separately and trained for 5 rounds respectively; the global communication rounds are set to 200 rounds; each client has 600 samples, and 20% of the data is evenly distributed to 10 sample classes, and the remaining 80% of the data is randomly distributed to 3 sample classes. In this way, data heterogeneity is constructed.

[0168] C. Experimental results.

[0169] In order to prove the effectiveness of the feature comparison optimization component and the classifier dynamic integration component used in the present invention, experiments were carried out on these two components respectively, and the present method was compared with the conventional FedAvg (federated average algorithm) method. The experimental results are shown in Figure 2. Figure 3 , Figure 4 As shown in Figure 2, these two components are helpful for improving model performance. The feature comparison optimization component helps the model improve 2~3% performance, and the classifier dynamic integration component helps the model improve 5~7% performance. In addition, we conducted generalization performance tests on the classifier integration component. The experimental results are shown in Figure 2. Figure 5 As shown in the figure, this component can improve the generalization performance of the model and achieve a 10~11% improvement in model performance.

[0170] Example 2

[0171] This embodiment provides a device for implementing a federated learning method based on feature comparison optimization and classifier dynamic integration, the device comprising a server and clients, of which:

[0172] The server is used to initialize the global model and send it to the participating federated learning clients; and, based on the total number of client samples received, setting a local model aggregation weight, and aggregating the local model with this weight to obtain a global prototype; aggregating the received local feature prototype set based on the comparison score to generate an average feature prototype set, and then performing comparison optimization on the average feature prototype set to obtain a global feature prototype set; sending the aggregated global prototype and global feature prototype set to each client, and executing the next round of learning until the local model converges or reaches the set communication round;

[0173] The client is used to update the local model based on the received global model, wherein: when performing the first local model update, the client initializes the local model to the global model it received; when performing the second local model update, the client initializes the local model to the global model it received; When the local model is updated, , the client updates the local feature extractor in the local model to the global feature extractor in the global model it has recently received; and, the client calculates the local adaptability weight of the global classifier in the global model it has recently received under the current update round, and integrates the local classifier and the global classifier based on this weight to update the local classifier;

[0174] And, for training the updated local model, and the training is performed in the order of first training the local classifier and then training the local feature extractor, wherein: when performing the first local model training, the local classifier and the local feature extractor are trained using the cross entropy loss; when performing the second local model training, the local classifier and the local feature extractor are trained using the cross entropy loss; When training the local model, , training the local classifier using a cross entropy loss, and, simultaneously training the local feature extractor using a cross entropy loss and a global prototype loss;

[0175] And, it is used to use the trained local feature extractor to extract features from its local private data set to calculate the local feature prototypes corresponding to each type of feature vector, and to form the local feature prototypes into a local feature prototype set, and then upload the local model and the local feature prototype set to the server; in the first round of federated learning, the total sample size of the client is uploaded to the server at the same time.

[0176] Example 3

[0177] This embodiment provides an electronic device, including:

[0178] at least one processor; and

[0179] A memory storing instructions, which, when executed by the at least one processor, enables the at least one processor to perform the above-mentioned federated learning method based on feature contrast optimization and dynamic integration of classifiers.

[0180] In this embodiment, electronic devices include but are not limited to: personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile computing devices, smart phones, tablet computers, cellular phones, personal digital assistants (PDAs), handheld devices, messaging devices, wearable computing devices, consumer electronic devices, etc.

[0181] Example 4

[0182] This embodiment also provides a computer-readable storage medium storing executable instructions, which, when executed, enable the machine to perform the above-mentioned federated learning method based on feature comparison optimization and dynamic integration of classifiers.

[0183] Specifically, a system or device equipped with a readable storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer or processor of the system or device can read and execute instructions stored in the readable storage medium.

[0184] In this case, the program code itself read from the computer-readable medium can realize the function of any one of the above embodiments, and thus the computer-readable code and the computer-readable storage medium storing the computer-readable code constitute part of this specification.

[0185] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code may be downloaded from a server computer or a cloud via a communication network.

[0186] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0187] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0188] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0189] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0190] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A federated learning method based on feature contrast optimization and classifier dynamic integration, characterized in that: The method comprises: S1. Initialize the global model through the server and send it to the participants in federated learning Clients; S2. The client updates the local model based on the received global model, where: When performing the first local model update, the client initializes the local model to the global model it received; In progress When the local model is updated, , the client updates the local feature extractor in the local model to the global feature extractor in the global model it has recently received; and, the client calculates the local adaptability weight of the global classifier in the global model it has recently received under the current update round, and integrates the local classifier and the global classifier based on this weight to update the local classifier; S3. The client trains the updated local model in the order of training the local classifier first and then the local feature extractor, where: When performing the first local model training, the local classifier and the local feature extractor are trained using cross entropy loss; In progress When training the local model, , training the local classifier using a cross entropy loss, and, simultaneously training the local feature extractor using a cross entropy loss and a global prototype loss; S4. The client uses the trained local feature extractor to perform feature extraction on its local private data set to calculate the local feature prototypes corresponding to various feature vectors, and the local feature prototypes are combined into a local feature prototype set, and then the local model and the local feature prototype set are uploaded to the server; and, in the first round of federated learning, the total sample size of the client is uploaded to the server at the same time; wherein the local private data set is an image data set, and the client sample is an image sample; S5. The server sets the local model aggregation weight based on the total number of client samples received, and aggregates the local model with this weight to obtain the global prototype; and, aggregates the local feature prototype set it receives based on the comparison score to generate an average feature prototype set, and then compares and optimizes the average feature prototype set to obtain the global feature prototype set; sends the aggregated global prototype and global feature prototype set to each client, and executes the next round of learning until the local model converges or reaches the set communication round, and uses the final local model to realize the classification of the local image data set.

2. The federated learning method based on feature comparison optimization and classifier dynamic integration according to claim 1, characterized in that: Step S2 specifically includes: When performing the first local model update, the client initializes the local model to the received global model, that is: , Indicates an update operation. Indicates The local model after the first local update of the client, represents the initialized global model; And use the global model The global feature extractor updates the local feature extractor of the local model, namely: , Indicates The local feature extractor after the first local update of the client, Represents the initialized global model A global feature extractor for And, using the global model The global classifier updates the local classifier of the local model, that is: , Indicates The local classifier after the first local update of the client, Represents the initialized global model The global classifier.

3. The federated learning method based on feature comparison optimization and classifier dynamic integration according to claim 2 is characterized in that: Step S2 specifically also includes: In progress When the local model is updated, , for the local feature extractor of the local model, update it to the latest received global model The global feature extractor in is: , Indicates Client No. The local feature extractor after the local update, Represents the global model A global feature extractor for For the local classifier of the local model, calculate the global model The local adaptability weight of the global classifier in the current round : In formula (1), Indicates Client No. The local adaptability weight in the local update, Indicates category index The Sample classes, represents the cross entropy loss, Indicates Client No. The local classifier after local training, represents the local classification function, Indicates that the global feature prototype Passed to the local classifier The classification prediction results obtained later are Indicates The global classifier after sub-global aggregation, represents the global classification function, Indicates that the local feature prototype Passed to the global classifier The classification prediction results obtained Indicates The label vector corresponding to the sample class; Based on the local adaptability weight Performing local classifier and the global classifier Integrations: In formula (2), Indicates an update operation. Indicates Client No. The local classifier after the local update, Indicates Client No. After the local training, The local classifier before the local update, Indicates Global classifier after sub-global aggregation.

4. The federated learning method based on feature comparison optimization and classifier dynamic integration according to claim 3 is characterized in that: In step S3, for the local classifier, only the cross entropy loss is used when training the local model. To construct the loss function for training, namely: In formula (3), Indicates A local classifier trained by the client, Indicates that the gradient of the local classifier parameters is calculated. At this time, the parameters of the local feature extractor are frozen and no longer modified; Indicates that it belongs to the dataset Mini-batch datasets; For the local feature extractor, the cross entropy loss is used when training the first local model. To construct the loss function for training, namely: In formula (4), Indicates The local feature extractor of each client after the first local training, Indicates The local feature extractor after the first local update of the client, Indicates that the gradient of the local feature extractor parameters is calculated. At this time, the parameters of the local classifier are frozen and no longer modified; Indicates The local classifier of each client after the first local training; For the local feature extractor, When training the local model, , using the cross entropy loss While adding the global prototype loss Conduct training, namely: （5）； （6）； In formula (5) and (6), represents the global prototype loss, represents the L2 norm, Indicates that the sample Enter to Client No. Local feature extractor after local update The feature vector generated in Indicates The first round of local training used The global trait prototype of the class, For sample The label of the corresponding class, Indicates Client No. Local feature extractor after local training.

5. The federated learning method based on feature comparison optimization and classifier dynamic integration according to claim 4 is characterized in that: In step S4, the local feature prototypes corresponding to each type of feature vector are calculated, and the local feature prototypes are combined into a local feature prototype set, specifically: （7）； （8）； In formula (7) and (8), Indicates Client No. After a round of local training, the The local feature prototype of the class, Indicates The client data set belongs to The samples of the class, Indicates The client data set belongs to The labels of samples of each class, Indicates that the sample Enter to Client No. Local feature extractor after sub-local training The feature vector generated in Indicates The client data set belongs to A sample dataset of classes, Indicates The client data set belongs to The size of the sample dataset for each class, Indicates The local feature prototype set consists of the local feature prototypes of each class of the client.

6. The method for federated learning based on feature comparison optimization and classifier dynamic integration according to claim 5, characterized in that: In step S5, the local model aggregation weight is: （9）； In formula (9), represents the local model aggregation weight, Indicates the total sample size of the client. Indicates Client datasets size; The global prototype obtained by aggregation is: （10）； In formula (10), Indicates The global prototype obtained after the second global aggregation, Indicates Client No. The local model obtained after this training.

7. The method for federated learning based on feature comparison optimization and classifier dynamic integration according to claim 6, characterized in that: In step S5, the received local feature prototype sets are aggregated based on the comparison scores to generate an average feature prototype set, specifically: （11）； （12）； （13）； （14）； In formulas (11)-(14), Represents the local feature prototype The results after aggregation according to the sample size are The global feature prototype of each class, Indicates that it belongs to The global feature prototype of each class, Indicates Client No. The class in The comparison score when rounding, Indicates The class in The corresponding average feature prototype in the round aggregation, represents the average feature prototype set; The average feature prototype set is compared and optimized to obtain the global feature prototype set: （15）； （16）； （17）； In formulas (15)-(17), It represents the intermediate state from the average feature prototype to the global feature prototype. Indicates The global feature prototype corresponding to the class, represents the gradient associated with the feature prototype, Represents the average feature prototype set Belong to the The average feature prototype of each class, Represents a global feature prototype set.

8. A device for implementing a federated learning method based on feature contrast optimization and dynamic integration of classifiers, characterized in that: The device includes a server and clients, of which: The server is used to initialize the global model and send it to the participating federated learning clients; and, based on the total number of client samples received, setting a local model aggregation weight, and aggregating the local model with this weight to obtain a global prototype; aggregating the received local feature prototype set based on the comparison score to generate an average feature prototype set, and then performing comparison optimization on the average feature prototype set to obtain a global feature prototype set; sending the aggregated global prototype and global feature prototype set to each client, and executing the next round of learning until the local model converges or reaches the set communication round; The client is used to update the local model based on the received global model, wherein: when performing the first local model update, the client initializes the local model to the global model it received; when performing the second local model update, the client initializes the local model to the global model it received; When the local model is updated, , the client updates the local feature extractor in the local model to the global feature extractor in the global model it has recently received; and, the client calculates the local adaptability weight of the global classifier in the global model it has recently received under the current update round, and integrates the local classifier and the global classifier based on this weight to update the local classifier; And, for training the updated local model, and the training is performed in the order of first training the local classifier and then training the local feature extractor, wherein: when performing the first local model training, the local classifier and the local feature extractor are trained using the cross entropy loss; when performing the second local model training, the local classifier and the local feature extractor are trained using the cross entropy loss; When training the local model, , training the local classifier using a cross entropy loss, and, simultaneously training the local feature extractor using a cross entropy loss and a global prototype loss; And, for extracting features from its local private data set using the trained local feature extractor to calculate the local feature prototypes corresponding to various feature vectors, and composing the local feature prototypes into a local feature prototype set, and then uploading the local model and the local feature prototype set to the server; in the first round of federated learning, the total sample size of the client is uploaded to the server at the same time; The local private data set is an image data set, the client samples are image samples, and the client uses the final local model to classify the local image data set.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and A memory storing instructions, which, when executed by the at least one processor, causes the at least one processor to execute the federated learning method based on feature contrast optimization and dynamic integration of classifiers as described in any one of claims 1 to 7.

10. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores executable instructions, which, when executed, enable the machine to perform the federated learning method based on feature comparison optimization and dynamic integration of classifiers as described in any one of claims 1 to 7.

Images