Personalized federal learning method for image classification

By generating and updating global prototypes in image classification federated learning, meta-training and meta-testing enhance the discriminant ability of local prototypes, the problem of model performance degradation caused by data heterogeneity is solved, the classification accuracy is improved and communication costs are reduced.

CN119006895APending Publication Date: 2024-11-22HARBIN ENG UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411011148.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Existing federated learning methods cannot effectively solve the problem of data heterogeneity when facing high heterogeneity data, resulting in a decline in model performance, large amount of calculation or slow convergence.

Method used

A personalized federated learning method for image classification is proposed. By generating the initial global prototype and performing meta-training and meta-testing on the client, the prototype learning module and prototype refinement module are used to enhance the discriminant ability of local prototypes, and the model parameters are updated round by round. and local prototypes until training converges.

Benefits of technology

It significantly improves the classification accuracy of the model, reduces the communication cost between the client and the server, and effectively alleviates the impact of data heterogeneity on model accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006895A_ABST
    Figure CN119006895A_ABST
Patent Text Reader

Abstract

The invention provides a personalized federal learning method for image classification. The method comprises the following steps: step 1, initializing a global prototype # imgabs0 # based on a server; step 2, sending the initialized global prototype to a client; 3, initializing a local model parameter # imgabs1 # by the client, randomly dividing a local data set, updating a global prototype, and obtaining an updated model parameter and a local prototype; 4, the client side reserves the updated model parameters and uploads the updated local prototype to the server; 5, the server aggregates the local prototypes uploaded by all the clients to generate a new global prototype; and step 6, repeating the steps 2-6, sending the global prototype obtained each time to the client for updating until the maximum cycle index is reached, and outputting the final global prototype. According to the method, the influence of data heterogeneity on the model precision can be better relieved, higher classification accuracy is obtained, and the communication cost between the client and the server is greatly reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a personalized federated learning method for image classification, and belongs to the technical field of artificial intelligence. Background Art

[0002] With the rapid development of artificial intelligence and machine learning technologies, image classification has become a key task in many application fields such as medical diagnosis, autonomous driving, and security monitoring. Traditional centralized image classification methods face many challenges, especially in terms of privacy security and ethics. In order to meet these challenges, federated learning came into being. Federated learning is a distributed machine learning method that can collaborate with multiple clients to jointly train models while ensuring data privacy. Although federated learning has significant advantages in protecting data privacy, there are still some problems in practical applications, especially due to the heterogeneity of client data, which leads to the performance degradation of the global model on some clients.

[0003] In order to solve the problem of data heterogeneity, the current federated learning methods are mainly divided into two categories according to different optimization objectives. One type of method is committed to learning a more generalized global model. Classic algorithms include FedProx and SCAFFOLD. The other type learns a personalized model for each client to adapt to the local data distribution. Classic algorithms include FedBN and MOON. However, these algorithms have their own defects. For example, FedProx will reduce the convergence speed, and MOON requires three times the amount of calculation. Although these federated learning methods have shown certain potential in theory and preliminary experiments, in practical applications, when faced with highly heterogeneous data, these methods still cannot solve the problem of data heterogeneity well. Engineering practicality is more precious.

[0004] In view of the above problems, the present invention proposes a personalized federated learning method for image classification, which aims to generate an initial global prototype so that it can generate a personalized model for each client. First, the server sends an initialized global prototype to each client, and each client performs training locally, wherein the proposed gradient-based meta-learning algorithm is used as a personalized strategy to generate a local prototype, including a meta-training phase and a meta-testing phase, and the prototype learning module and the prototype refinement module are used to enhance the discrimination ability of the local prototype. After the local training is completed, the local prototype is uploaded to the server. The server aggregates the local prototypes of each client and generates a new global prototype. The above steps will be repeated until the entire training process converges or the maximum number of training times is reached. Summary of the invention

[0005] The present invention aims to solve the technical problem that data heterogeneity cannot be solved in the prior art, and further proposes a personalized federated learning method for image classification.

[0006] The technical solution adopted by the present invention to solve the above problems is: the present invention proposes a personalized federated learning method for image classification, comprising:

[0007] Step 1: Initialize the global prototype based on the server

[0008] Step 2: Send the initialized global prototype to the client;

[0009] Step 3: Client initializes local model parameters The local data set is randomly divided, and the global prototype is updated based on the randomly divided local data to obtain the updated model parameters and local prototype;

[0010] Step 4: The client retains the updated model parameters and uploads the updated local prototype to the server;

[0011] Step 5: The server aggregates the local prototypes uploaded by all clients to generate a new global prototype;

[0012] Step 6: Repeat steps 2-6, and send the global prototype obtained each time to the client for updating until the maximum number of cycles is reached, and output the final global prototype and the personalized model of each client W = {w1, L, w N}.

[0013] Optionally, the step of obtaining updated model parameters and local prototype in step 3 includes:

[0014] Step 3.1: Randomly divide the local data into meta-training set D tr and meta-test set D te ;

[0015] Step 3.2: After the client receives the global prototype sent by the server, it uses the configured first loss function to train the meta-training set D tr Perform meta-training to obtain intermediate model parameters and intermediate local prototypes;

[0016] Step 3.3: Based on the meta-training set D te , meta-test the intermediate model parameters and intermediate local prototypes in combination with the configured total loss function to obtain updated model parameters and local prototypes.

[0017] Optionally, the step of obtaining intermediate model parameters and intermediate local prototypes in step 3.2 includes:

[0018] Step 3.2.1: Use the feature extractor to train the meta-training set D tr Extract features from the images in the image;

[0019] Step 3.2.2: Process the feature extracted image through the average pooling layer to obtain the feature vector;

[0020] Step 3.2.3: Calculate the output of the current round of model training based on the feature vector Using the output of the current round of model training Calculate the cross entropy loss value in, is the model parameter at the tth communication, i is the client number, P i t For local prototypes;

[0021] Step 3.2.4: Based on the cross entropy loss value The intermediate model parameters are calculated by combining the local model parameters and the intermediate local prototype

[0022] Intermediate model parameters and the intermediate local prototype The calculation formula is:

[0023]

[0024] In formula (1), a is the preset learning rate, and ~ is the derivative operation.

[0025] Optionally, the step of obtaining updated model parameters and local prototype in step 3.3 includes:

[0026] Step 3.3.1: Based on the meta-test set D te Perform feature extraction on the intermediate local prototype;

[0027] Step 3.3.2: Process the intermediate local prototype after feature extraction through the average pooling layer to obtain the feature vector

[0028] Step 3.3.3: Adjust the intermediate model parameters based on the configured total loss function and the intermediate local prototype Conduct tests and optimize model parameters by reverse optimization and the local prototype P i t ;

[0029] Step 3.3.4: The server updates the local model parameters and the local prototype according to the total loss to obtain updated model parameters and the local prototype;

[0030] The expression of the total loss function of the configuration is:

[0031]

[0032] In formula (2), λ1, λ2 and λ3 are all preset weights. is the prototype alignment loss, Improve the loss for prototype discrimination;

[0033] Prototype alignment loss L PA The calculation formula is:

[0034]

[0035] In formula (3), P i is the local prototype, P G is the global prototype;

[0036] Prototype discrimination improvement loss L PDE The calculation formula is:

[0037]

[0038] The updated model parameters and local prototypes are calculated as:

[0039]

[0040] Optionally, the expression for generating a new global prototype in step 5 is:

[0041]

[0042] In formula (6), is the set of all clients that contain the prototype of type c, is the k-th prototype of the c-th type prototype in the local prototype.

[0043] The beneficial effects of the present invention are:

[0044] 1. The present invention can better alleviate the impact of data heterogeneity on model accuracy, obtain higher classification accuracy, and greatly reduce the communication cost between the client and the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of a personalized federated learning method for image classification provided by the present invention;

[0046] Figure 2 A schematic diagram of the meta-training and meta-testing process of the local client provided by the present invention;

[0047] Figure 3A schematic diagram of loss calculation of a local client provided by the present invention;

[0048] Figure 4 A comparison chart of the test BACC curves of a personalized federated learning method for image classification provided by the present invention and six benchmark methods in the prior art. Example

[0049] Combination Figure 1-4 This embodiment is described as follows. Figure 1 As shown, the steps of a personalized federated learning method for image classification described in this embodiment include:

[0050] S1: Server initializes global prototype The client initializes local model parameters The local models of each client can be heterogeneous

[0051] S2: The server sends the global prototype to each client

[0052] S3: Each client randomly divides the local image dataset into meta-training set D tr and meta-test set D te ;

[0053] S4: Figure 2 As shown, after receiving the global prototype, the meta-training set is used for meta-training. For the t-th communication, ResNet18 is first used as a feature extractor to train the meta-training set D tr The image in the image is extracted for feature extraction, and then processed by the average pooling layer to obtain the feature vector. Based on the image classification network model, the output of the current round of model training is used Calculate the cross entropy loss value in is the model parameter, i represents the client number, P i t Represents the local prototype. Gets the intermediate model parameters and the intermediate local prototype The calculation method is as follows:

[0054]

[0055] In formula (1), a is the preset learning rate, and ~ is the derivative operation.

[0056] S5: Figure 2 As shown, for the meta-test set D te , using the local model updated by S4 Perform feature extraction, and then process it through the average pooling layer to obtain the feature vector The intermediate model parameters obtained by using the configured total loss function for S4 and the intermediate local prototype Test and optimize by back propagation and P i t The total loss function of the configuration is

[0057]

[0058] In formula (2), λ1, λ2 and λ3 are all preset weights. is the prototype alignment loss, Improve the loss for prototype discrimination;

[0059] like Figure 3 As shown, Figure 3 Schematic diagram of the loss calculation for the local client, prototype alignment loss L PA The calculation formula is:

[0060]

[0061] In formula (3), P i is the local prototype, P G is the global prototype;

[0062] By reducing their distance in the embedding space, prototypes of the same category are ensured to be in the same feature space.

[0063] Represents the prototype discrimination improvement loss:

[0064]

[0065] Improve the discriminative power of prototypes by enhancing intra-class compactness;

[0066] The server side adjusts the local model parameters based on the total loss and the local prototype P i t+1 Update, the calculation formula is:

[0067]

[0068] S5: Each client uploads the updated local prototype to the server;

[0069] S6: The server aggregates the local prototypes to generate a new global prototype:

[0070]

[0071] In formula (6), is the set of all clients that contain the prototype of type c, is the k-th prototype of the c-th type prototype in the local prototype.

[0072] S7: Repeat S2-S6 until the preset training stop condition is met, and obtain the global prototype and the personalized model of each client W = {w1, L, w N}.

[0073] In summary, in order to verify the classification accuracy of the present invention, the following experiments are conducted: First, the real-world medical image dataset used in the experiment is introduced: the dermoscopy dataset Fed-ISIC2019, which contains a total of 23,247 images, divided into six clients, including eight types of skin lesions. The image size is 200×200×3. This embodiment uses FedAvg, FedProx, perFedAvg, FedBN, FedProto and PRR-Imbalance as benchmark methods. This embodiment uses BACC (balanced accuracy) as the classification indicator. Figure 4 As shown in Table 1, the image classification method proposed in the present invention can obtain a higher classification accuracy, which is improved by 16.51%, 15.54%, 13.47%, 5.83%, 7.32% and 11.86% compared with FedAvg, FedProx, perFedAvg, FedBN, FedProto and PRR-Imbalance, respectively.

[0074] Table 1

[0075] FedAvg FedProx perFedAvg FedBN FedProto PRR-Imbalance Proposed method 50.59% 50.21% 52.28% 59.92% 58.43% 53.89% 65.75%

[0076] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A personalized federated learning method for image classification, characterized in that: The steps of the personalized federated learning method for image classification include: Step 1: Initialize the global prototype based on the server Step 2: Send the initialized global prototype to the client; Step 3: The client initializes local model parameters The local data set is randomly divided, and the global prototype is updated based on the randomly divided local data to obtain the updated model parameters and local prototype; Step 4: The client retains the updated model parameters and uploads the updated local prototype to the server; Step 5: The server aggregates the local prototypes uploaded by all clients to generate a new global prototype; Step 6: Repeat steps 2-6, and send the global prototype obtained each time to the client for updating until the maximum number of cycles is reached, and output the final global prototype and the personalized model of each client W = {w1, L, w N }.

2. The personalized federated learning method for image classification according to claim 1, characterized in that: The steps of obtaining updated model parameters and local prototypes in step 3 include: Step 3.1: Randomly divide the local data into meta-training set D tr and meta-test set D te ; Step 3.2: After receiving the global prototype sent by the server, the client performs a loss function on the meta-training set D based on the configured first loss function. tr Perform meta-training to obtain intermediate model parameters and intermediate local prototypes; Step 3.3: Based on the meta-training set D te , meta-test the intermediate model parameters and intermediate local prototypes in combination with the configured total loss function to obtain updated model parameters and local prototypes.

3. The personalized federated learning method for image classification according to claim 2, characterized in that: The steps of obtaining the intermediate model parameters and the intermediate local prototype in step 3.2 include: Step 3.2.1: Use the feature extractor to extract the meta-training set D tr Extract features from the images in the image; Step 3.2.2: Process the feature extracted image through the average pooling layer to obtain the feature vector; Step 3.2.3: Calculate the output of the current round of model training based on the feature vector Using the output of the current round of model training Calculate the cross entropy loss value in, is the model parameter at the tth communication, i is the client number, P i t For local prototypes; Step 3.2.4: Based on the cross entropy loss value The intermediate model parameters are calculated by combining the local model parameters and the intermediate local prototype Intermediate model parameters and the intermediate local prototype The calculation formula is: In formula (1), a is the preset learning rate, and ~ is the derivative operation.

4. The personalized federated learning method for image classification according to claim 2, characterized in that: The steps to obtain the updated model parameters and local prototype in step 3.3 include: Step 3.3.1: Based on the meta-test set D te Perform feature extraction on the intermediate local prototype; Step 3.3.2: Process the intermediate local prototype after feature extraction through the average pooling layer to obtain the feature vector Step 3.3.3: Adjust the intermediate model parameters based on the configured total loss function and the intermediate local prototype Conduct tests and optimize model parameters by reverse optimization and the local prototype P i t ; Step 3.3.4: The server updates the local model parameters and the local prototype according to the total loss to obtain updated model parameters and the local prototype; The expression of the total loss function of the configuration is: In formula (2), λ1, λ2 and λ3 are all preset weights. is the prototype alignment loss, Improve the loss for prototype discrimination; Prototype alignment loss L PA The calculation formula is: In formula (3), P i is the local prototype, P G is the global prototype; Prototype discrimination improvement loss L PDE The calculation formula is: The updated model parameters and local prototypes are calculated as:

5. The personalized federated learning method for image classification according to claim 1, characterized in that: The expression for generating a new global prototype in step 5 is: In formula (6), is the set of all clients that contain the prototype of type c, is the k-th prototype of the c-th type prototype in the local prototype.

Citation Information

Cited By

  • Federal learning method and system for model heterogeneous scene

    CN119990262A