A Federated Learning Method and System Based on Unified Representation and Classifier Correction

Abstract

This invention discloses a federated learning method and system based on unified representation and classifier correction, designed to address the federated learning problem under long-tailed data distributions. First, this invention obtains a globally unified prototype by aggregating local prototypes (i.e., the average features of categories extracted by the global model). These prototypes are then used to adjust the feature space, bringing features within the same category closer to the corresponding globally unified prototype while pushing away features from other categories. Furthermore, this invention reduces classifier bias through prototype mixing using the global prototype. It generates a balanced virtual feature set by fusing the globally unified prototype and local features. The classifier is then retrained on this feature set to correct the decision boundary and mitigate bias. This invention can effectively improve the classification performance of models under long-tailed data while reducing the bias in the feature space and the classifier. The superior performance of the method has been verified on multiple datasets.

Description

Technical Field

[0001] This invention belongs to the field of machine learning technology and relates to a federated learning method and system, specifically a federated learning method and system that improves model performance on long-tailed datasets through unified representation and classifier correction. Background Technology

[0002] Federated learning is a distributed learning method that allows different clients to collaboratively learn a shared model while protecting data privacy. Existing federated learning research typically assumes a balanced global class distribution. However, in real-world scenarios, data class distributions often exhibit a long-tail distribution, meaning that the head classes have a large number of samples, while the tail classes have only a small number of samples. Furthermore, when learning from long-tail data, the model tends to favor the head classes, resulting in poor classification performance for the tail classes. Therefore, when federated learning faces the dilemma of long-tail distribution, we argue that the global optimization direction is biased and reveal that the global bias can be decomposed into two parts: a distorted feature space and biased classifier weights.

[0003] To address the problem of non-independent and identically distributed data in federated learning, many methods have been proposed to process them, which can be categorized into the following main directions.

[0004] (1) Improve the local training strategy of the client: FedProx [Reference 1], SCAFFOLD [Reference 2] and MOON [Reference 4] and other methods focus on modifying the training process of the client to reduce the data bias of the client.

[0005] (2) Optimize the global model on the server: CCVR [Reference 3] and other methods improve the performance of the global model by generating virtual data on the server side.

[0006] However, these methods still have limitations. They lack consideration for the global long-tail distribution, limiting the model's ability to represent and classify tail categories, leading to a decline in overall model performance. Furthermore, various methods such as resampling, reweighting, logical adjustment, and decoupled training have been proposed for the long-tail learning problem. While these methods are effective in centralized training scenarios, they tend to degenerate and fail in distributed training and data heterogeneity scenarios. In conclusion, developing long-tail data processing methods suitable for federated learning scenarios remains a crucial issue in federated learning.

[0007] [Document 1] T.Li, AKSahu, M.Zaheer, M.Sanjabi, A.Talwalkar, and V.Smith, "Federated optimization in heterogeneous networks," arXiv preprintarXiv:1812.06127, 2018.

[0008] [Document 2] SP Karimireddy, S. Kale, M. Mohri, S. J. Reddi, SU Stich, and A. T. Suresh, "Scaffold: Stochastic controlled averaging for on-devicefederated learning," in ICML, 2020.

[0009] [Document 3] M.Luo, F.Chen, D.Hu, Y.Zhang, J.Liang, and J.Feng, "No fear of heterogeneity: Classifier calibration for federated learning with non-iiddata," in NeurIPS, 2021.

[0010] [Document 4] Q.Li, B.He, and D.Song, "Model-contrastive federated learning," inCVPR, 2021, pp.10713-10 722. Summary of the Invention

[0011] To address the issues of feature space distortion and classifier bias caused by long-tailed data distribution in federated learning, this invention proposes a novel federated learning method and system that utilizes a globally unified prototype to calibrate the feature space and classifier weights, thus tackling the challenges posed by long-tailed data distribution.

[0012] The technical solution adopted by the method of the present invention is: a federated learning method based on unified representation and classifier correction, comprising the following steps:

[0013] Step 1: Build a globally unified prototype;

[0014] First, each client computes a local prototype based on its local dataset; then, the client uploads the normalized local prototype set to the server; next, the server aggregates the local prototypes from different clients through a weighted average operation to generate a globally unified prototype; the client uploads the local label ratio, and the server estimates the global label ratio; each client downloads the globally unified prototype set and the global label ratio set.

[0015] Step 2: Use a globally unified prototype to shape the feature space, enhancing intra-class compactness and inter-class distance;

[0016] Step 3: Federal Prototype Hybridization;

[0017] First, the client uses a global model to extract a local feature set from the local dataset, and randomly selects a subset from the local feature set to generate virtual features for the category; then, by mixing the local features with a global unified prototype, a balanced virtual feature set is generated; finally, the client retrains the classifier based on the generated balanced virtual feature set; wherein, the global model is the result of aggregating multi-participant network models.

[0018] Preferably, step 1 includes the following sub-steps:

[0019] Step 1.1: For each client m and each category k, based on the local dataset Calculate the local prototype

[0020]

[0021]

[0022] Where (x,y) represents the sample and sample label, Let m be a subset of the dataset containing samples of class k. f is the sample size. m (x) represents the global model downloaded by client m, where I is the number of categories and d is the feature dimension;

[0023] Step 1.2: The client sets the L2-normalized local prototype set O m Uploaded to the server;

[0024] Step 1.3: The server aggregates the received prototype set through a weighted average operation to generate a globally unified prototype. and the global unified prototype set

[0025]

[0026]

[0027] Where A is the client set, n and m are the client indices, m represents the current client, and n represents all participating clients;

[0028] Step 1.4: The client uploads the local tag ratio N, and the server estimates the global tag ratio Q;

[0029]

[0030]

[0031] Where, N m Let m be the total number of samples from client m.

[0032] Step 1.5: Each client downloads the globally unified prototype set G and the globally proportional tag set Q.

[0033] As a preferred option, in step 2, for each instance (x,y) and its feature z, comparative learning is performed. A global unified prototype is used to bring features of the same category closer together, and the degree of repulsion between different categories is adjusted according to the global label ratio Q to straighten the distorted feature space.

[0034]

[0035]

[0036]

[0037] Where k is the category number and I is the number of categories. and Let φ(k) represent the global unified prototypes for class y and class k, respectively, where τ is the temperature parameter, φ(k) is the indicator function representing the weight of class k, and q k denoted as the proportion of global tags in the k-th class.

[0038] As a preferred embodiment, step 3 includes the following sub-steps:

[0039] Step 3.1: The client uses the global model to extract the local feature set Z. m ;

[0040] Step 3.2: For each category k, from the local feature set Z m A subset is randomly selected from the data to generate a virtual feature set for that category. It is represented as follows;

[0041]

[0042] in, Represents the local feature set Z in client m m The i-th feature randomly selected from the data. Let S be the label corresponding to this feature, and S be a subset. The number of features in;

[0043] Step 3.3: Each in the subset Features and Global Unified Prototype Mixing to generate a balanced virtual feature set R m ;

[0044]

[0045]

[0046]

[0047] in, Represents the i-th feature in the k-th feature subset on client m. Its corresponding global unified prototype The virtual features obtained by fusion, α represents and The fusion ratio between them can be obtained by sampling from a balanced distribution. For client m, the virtual feature set of class k is... represent The i-th virtual feature in The label corresponding to this virtual feature;

[0048] Step 3.4: The client uses the virtual feature set R m Retrain the classifier g according to the following loss function. m And upload the updated local model. To the server:

[0049]

[0050] Among them, R m For a balanced virtual feature set, (r,y) represents the set of features derived from R. m The virtual features and corresponding labels obtained from sampling, g m (r) represents inputting virtual features into the classifier g. m In this context, σ represents the softmax function, and L... FPMU To utilize the balanced virtual feature set R m Retrain the classifier g m loss function, Represents retraining g m The subsequent classifier.

[0051] The technical solution adopted by the system of this invention is: a federated learning system based on unified representation and classifier correction, comprising the following modules:

[0052] Module 1 is used to build a globally unified prototype;

[0053] First, each client computes a local prototype based on its local dataset; then, the client uploads the normalized local prototype set to the server; next, the server aggregates the local prototypes from different clients through a weighted average operation to generate a globally unified prototype; the client uploads the local label ratio, and the server estimates the global label ratio; each client downloads the globally unified prototype set and the global label ratio set.

[0054] Module 2 is used to shape the feature space using a globally unified prototype, enhancing intra-class compactness and inter-class distance;

[0055] Module 3, for federal prototype mixing;

[0056] First, the client uses a global model to extract a local feature set from the local dataset, and randomly selects a subset from the local feature set to generate virtual features for the category; then, by mixing the local features with a global unified prototype, a balanced virtual feature set is generated; finally, the client retrains the classifier based on the generated balanced virtual feature set; wherein, the global model is the result of aggregating multi-participant network models.

[0057] Preferably, module 1 includes the following sub-modules:

[0058] Module 1.1 is used for each client m and each category k, based on the local dataset. Calculate the local prototype

[0059]

[0060]

[0061] Where (x,y) represents the sample and sample label, Let m be a subset of the dataset containing samples of class k. f is the sample size. m (x) represents the global model downloaded by client m, where I is the number of categories and d is the feature dimension;

[0062] Module 1.2 is used by the client to normalize the local prototype set O of L2. m Uploaded to the server;

[0063] Module 1.3 is used by the server to aggregate the received prototype set through a weighted average operation to generate a globally unified prototype. and the global unified prototype set

[0064]

[0065]

[0066] Where A is the client set, n and m are the client indices, m represents the current client, and n represents all participating clients;

[0067] Module 1.4 is used for the client to upload the local tag ratio N, and the server to estimate the global tag ratio Q;

[0068]

[0069]

[0070] Where, N m Let m be the total number of samples from client m.

[0071] Module 1.5 is used for each client to download the globally unified prototype set G and the globally proportional tag set Q.

[0072] As a preferred embodiment, in Module 2, for each instance (x,y) and its feature z, comparative learning is performed. A global unified prototype is used to bring features of the same category closer together, and the degree of repulsion between different categories is adjusted according to the global label ratio Q to straighten the distorted feature space.

[0073]

[0074]

[0075]

[0076] Where k is the category number and I is the number of categories. and Let φ(k) represent the global unified prototypes for class y and class k, respectively, where τ is the temperature parameter, φ(k) is the indicator function representing the weight of class k, and q k denoted as the proportion of global tags in the k-th class.

[0077] Preferably, module 3 includes the following sub-modules:

[0078] Module 3.1 is used by the client to extract the local feature set Z using the global model. m ;

[0079] Module 3.2, for each category k, from the local feature set Z m A subset is randomly selected from the data to generate a virtual feature set for that category. It is represented as follows;

[0080]

[0081] in, Represents the local feature set Z in client m m The i-th feature randomly selected from the data. Let S be the label corresponding to this feature, and S be a subset. The number of features in;

[0082] Module 3.3, for each in subset Features and Global Unified Prototype Mixing to generate a balanced virtual feature set R m ;

[0083]

[0084]

[0085]

[0086] in, Represents the i-th feature in the k-th feature subset on client m. Its corresponding global unified prototype The virtual features obtained by fusion, α represents and The fusion ratio between them can be obtained by sampling from a balanced distribution. For client m, the virtual feature set of class k is... represent The i-th virtual feature in The label corresponding to this virtual feature;

[0087] Module 3.4 is used by the client based on the virtual feature set R m Retrain the classifier g according to the following loss function. m And upload the updated local model. To the server:

[0088]

[0089] Among them, R m For a balanced virtual feature set, (r,y) represents the set of features derived from R. m The virtual features and corresponding labels obtained from sampling, g m (r) represents inputting virtual features into the classifier g. m In this context, σ represents the softmax function, and L... FPMU To utilize the balanced virtual feature set R m Retrain the classifier g m loss function, Represents retraining g m The subsequent classifier.

[0090] Compared with the prior art, the beneficial effects of the present invention include:

[0091] (1) The local prototype calculated by each client based on its local dataset reflects the central features of each category in the local data.

[0092] (2) The client of this invention uploads the standardized local prototype set to the server, ensuring consistency and effectiveness in the prototype aggregation process;

[0093] (3) The server of this invention aggregates the local prototypes of different clients through a weighted average operation to generate a globally unified category prototype, which effectively integrates the category feature information from different clients.

[0094] (4) The global label ratio estimated by the server in this invention is based on the local label ratio uploaded by the client, which reflects the distribution of each category in the overall data and helps with subsequent feature space balancing processing.

[0095] (5) The global prototype set and global label ratio set downloaded by each client in this invention provide the local model with category features and distribution information from a global perspective;

[0096] (6) This invention uses a globally unified prototype to shape the feature space, thereby improving the feature space distortion caused by long-tail distribution by enhancing intra-class compactness and inter-class distance.

[0097] (7) The client of this invention uses a global model to extract feature sets from the local dataset, laying the foundation for generating a balanced virtual feature set;

[0098] (8) The present invention randomly selects a subset from the local feature set to generate virtual features of the category, thereby improving the diversity and representativeness of the feature set;

[0099] (9) The present invention generates balanced virtual features by mixing local features with global prototypes, which retains local data characteristics and incorporates global data characteristics.

[0100] (10) The client of this invention retrains the classifier based on the generated balanced virtual feature set, which effectively reduces the bias of the classifier and improves the overall performance of the model. Attached Figure Description

[0101] The technical solutions of the present invention will be further illustrated below using embodiments and specific implementation methods. In addition, some accompanying drawings are used in the description of the technical solutions. Those skilled in the art can obtain other drawings and the intent of the present invention from these drawings without any creative effort.

[0102] Figure 1 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0103] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0104] This embodiment provides a client set A, client index m, category number k, global model f, and local model f. m Feature vector z, number of classes I, feature dimension d, sample set D m and Sample size Local Prototype Set O m , global prototype set G, global tag ratio Q.

[0105] Please see Figure 1 This embodiment provides a federated learning method based on unified representation and classifier correction, which includes the following steps:

[0106] Step 1: Build a globally unified prototype;

[0107] First, each client computes a local prototype based on its local dataset; then, the client uploads the normalized local prototype set to the server; next, the server aggregates the local prototypes from different clients through a weighted average operation to generate a globally unified prototype; the client uploads the local label ratio, and the server estimates the global label ratio; each client downloads the globally unified prototype set and the global label ratio set.

[0108] In one implementation, step 1 specifically includes the following sub-steps:

[0109] Step 1.1: For each client m and each category k, based on the local dataset Calculate the local prototype

[0110]

[0111]

[0112] Where (x,y) represents the sample and sample label, Let m be a subset of the dataset containing samples of class k. f is the sample size. m (x) represents the global model downloaded by client m, where I is the number of categories and d is the feature dimension;

[0113] Step 1.2: The client sets the L2-normalized local prototype set O m Uploaded to the server;

[0114] Step 1.3: The server aggregates the received prototype set through a weighted average operation to generate a globally unified prototype. and the global unified prototype set

[0115]

[0116]

[0117] Where A is the client set, n and m are the client indices, m represents the current client, and n represents all participating clients;

[0118] Step 1.4: The client uploads the local tag ratio N, and the server estimates the global tag ratio Q;

[0119]

[0120]

[0121] Where, N m Let m be the total number of samples from client m.

[0122] Step 1.5: Each client downloads the globally unified prototype set G and the globally proportional tag set Q.

[0123] Step 2: Use a globally unified prototype to shape the feature space, enhancing intra-class compactness and inter-class distance;

[0124] In one implementation, for each instance (x,y) and its feature z, comparative learning is performed, using a global unified prototype to bring features of the same category closer together, and adjusting the degree of repulsion between different categories according to the global label ratio Q to straighten the distorted feature space.

[0125]

[0126]

[0127]

[0128] Where k is the category number and I is the number of categories. and Let φ(k) represent the global unified prototypes for class y and class k, respectively, where τ is the temperature parameter, φ(k) is the indicator function representing the weight of class k, and q k denoted as the proportion of global tags in the k-th class.

[0129] Step 3: Federal Prototype Hybridization;

[0130] First, the client uses a global model to extract a local feature set from the local dataset, and randomly selects a subset from the local feature set to generate virtual features for the category; then, by mixing the local features with a global unified prototype, a balanced virtual feature set is generated; finally, the client retrains the classifier based on the generated balanced virtual feature set; wherein, the global model is the result of aggregating multi-participant network models.

[0131] In one implementation, step 3 specifically includes the following sub-steps:

[0132] Step 3.1: The client uses the global model to extract the local feature set Z. m ;

[0133] Step 3.2: For each category k, from the local feature set Z m A subset is randomly selected from the data to generate a virtual feature set for that category. It is represented as follows;

[0134]

[0135] in, Represents the local feature set Z in client m m The i-th feature randomly selected from the data. Let S be the label corresponding to this feature, and S be a subset. The number of features in;

[0136] Step 3.3: Each in the subset Features and Global Unified Prototype Mixing to generate a balanced virtual feature set R m ;

[0137]

[0138]

[0139]

[0140] in, Represents the i-th feature in the k-th feature subset on client m. Its corresponding global unified prototype The virtual features obtained by fusion, α represents and The fusion ratio between them can be obtained by sampling from a balanced distribution. For client m, the virtual feature set of class k is... represent The i-th virtual feature in The label corresponding to this virtual feature;

[0141] Step 3.4: The client uses the virtual feature set R m Retrain the classifier g according to the following loss function. m And upload the updated local model. To the server:

[0142]

[0143] Among them, R m For a balanced virtual feature set, (r,y) represents the set of features derived from R. m The virtual features and corresponding labels obtained from sampling, g m (r) represents inputting virtual features into the classifier g. m In this context, σ represents the softmax function, and L... FPMU To utilize the balanced virtual feature set R m Retrain the classifier g m loss function, Represents retraining g m The subsequent classifier.

[0144] This embodiment also provides a federated learning system based on unified representation and classifier correction, including the following modules:

[0145] Module 1 is used to build a globally unified prototype;

[0146] First, each client computes a local prototype based on its local dataset; then, the client uploads the normalized local prototype set to the server; next, the server aggregates the local prototypes from different clients through a weighted average operation to generate a globally unified prototype; the client uploads the local label ratio, and the server estimates the global label ratio; each client downloads the globally unified prototype set and the global label ratio set.

[0147] Module 2 is used to shape the feature space using a globally unified prototype, enhancing intra-class compactness and inter-class distance;

[0148] Module 3, for federal prototype mixing;

[0149] First, the client uses a global model to extract a local feature set from the local dataset, and randomly selects a subset from the local feature set to generate virtual features for the category; then, it generates balanced virtual features by mixing the local features with a global unified prototype; finally, the client retrains the classifier based on the generated balanced virtual feature set.

[0150] The invention will be further illustrated below through specific experiments.

[0151] The deep learning framework used in this experiment is PyTorch, version 1.13.0, with CUDA version 11.4. The hardware environment for the experiment is an NVIDIA GeForce RTX 3090 graphics card.

[0152] The specific implementation procedure for this experiment is as follows:

[0153] Step 1: Building the model and dataset. Our experiments were conducted on the CIFAR-10-LT, CIFAR-100-LT, and Tiny-Imagenet-LT datasets.

[0154] 1. Dataset Construction: We chose to generate a long-tailed dataset based on the CIFAR10 / CIFAR100 and Tiny-Imagenet datasets by applying the imbalance factor (IF) and Dirichlet distribution parameter (β).

[0155] 2. Model construction: We used ResNet-18 as the base model.

[0156] Step 2: Conduct federated learning training: Train the model for federated learning in the order of steps one, two, and three in the invention description.

[0157] 1. Dataset;

[0158] This experiment uses the CIFAR-10-LT, CIFAR-100-LT, and Tiny-Imagenet-LT datasets. These datasets were generated based on the CIFAR10 / CIFAR100 and Tiny-Imagenet datasets, respectively, and long-tailed distribution and data heterogeneity were simulated by adjusting the imbalance factor (IF) and Dirichlet distribution parameter (β).

[0159] In the experiment, the imbalance factors were set to 100, 50 and 10 for CIFAR-10-LT and CIFAR-100-LT, respectively, while the imbalance factors for Tiny-Imagenet-LT were set to 100, 50 and 20.

[0160] 2. Model training settings;

[0161] The SGD optimizer is used with a learning rate of 0.1. The communication rounds for all federated learning methods are set to 150 rounds. Local training consists of 10 rounds, with a total of 20 clients and 40% of clients participating in each round.

[0162] 3. Comparison of model accuracy;

[0163] To demonstrate the effectiveness of this invention under long-tailed data distributions, experiments were conducted comparing it with state-of-the-art federated learning methods on the CIFAR-10-LT dataset, including:

[0164] (1) FedAvg: B.McMahan, E.Moore, D.Ramage, S.Hampson, and BAyArcas, "Communication-efficient learning of deep networks from decentralized data," inAISTATS, 2017, pp.1273-1282

[0165] (2)FedProx: T.Li, AKSahu, M.Zaheer, M.Sanjabi, A.Talwalkar, and V.Smith, "Federated optimization in heterogeneous networks," arXiv preprintarXiv:1812.06127, 2018.

[0166] (3)MOON: Q.Li, B.He, and D.Song, "Model-contrastive federated learning," inCVPR, 2021, pp.10 713-10 722

[0167] (4)CCVR: X.Shang, Y.Lu, G.Huang, and H.Wang, "Federated learning onheterogeneous and long-tailed data via classifier re-training with federatedfeatures," in IJCAI, 2022.

[0168] (5)LWS: B.Kang, S.Xie, M.Rohrbach, Z.Yan, A.Gordo, J.Feng, andY.Kalantidis, "Decoupling representation and classifier for long-tailedrecognition," arXivpreprint arXiv:1910.09217, 2019.

[0169] (6)FedProc:

[0170] (7)FedProto: Y.Tan, G.Long, L.Liu, T.Zhou, Q.Lu, J.Jiang, and C.Zhang, "Fedproto: Federated prototype learning across heterogeneous clients," in AAAI, 2022.

[0171] (8)CReFF: X.Shang, Y.Lu, G.Huang, and H.Wang, “Federated learning onheterogeneous and long-tailed data via classifier re-training with federatedfeatures,” in IJCAI, 2022.

[0172] Method Name IF=100 IF=50 IF=10 FedAvg 60.06 64.15 79.20 FedProx 59.23 64.81 79.48 MOON 58.94 62.41 76.62 CCVR 68.82 72.20 80.52 τ-norm 41.58 41.56 51.68 LWS 37.47 36.92 49.16 FedProc 39.14 43.51 57.19 FedProto 33.96 41.53 50.72 CReFF 70.57 72.43 79.33 RUCR 71.41 74.99 82.75

[0173] As can be seen from the table, this invention demonstrates superior performance under long-tailed data distribution conditions compared to existing methods. This is mainly because this invention not only improves the bias in the feature space but also calibrates the classifier weights. When dealing with non-independent and identically distributed long-tailed data, this invention provides a more balanced and discriminative feature representation compared to other methods, while effectively improving the overall performance of the model and achieving better data distribution adaptability and model generalization ability.

[0174] The core innovations of this invention include:

[0175] (1) Improved feature space and classifier bias: This invention effectively shapes the feature space by constructing a globally unified prototype and prototype-first contrastive learning, reducing the feature space distortion caused by long-tailed data distribution. This enables the model to maintain better discriminative ability among different categories and provides more balanced and discriminative feature representations.

[0176] (2) Privacy Protection and Data Efficiency: Without violating privacy protection principles, this invention effectively generates a balanced virtual feature set through a federated prototype hybrid strategy. This allows the classifier to be effectively trained on local data from different clients, enabling the model to better adapt to various data distributions while maintaining data privacy.

[0177] (3) Broad Applicability and Improved Efficiency: This invention is not only applicable to handling long-tailed data distribution problems in federated learning environments, but also has higher model performance and adaptability due to its comprehensive improvement on feature space and classifier bias. This further enhances the overall efficiency of federated learning models on diverse and imbalanced datasets.

[0178] It should be understood that the embodiments described above are only some, not all, of the embodiments of the present invention. Furthermore, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0179] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A federated learning method based on unified representation and classifier rectification, characterized in that, The method comprises the following steps: Step 1: constructing a global uniform prototype; First, each client calculates a local prototype based on its local data set; Then, the client uploads the normalized local prototype set to the server; then, the server aggregates the local prototypes of different clients through a weighted average operation to generate a global uniform prototype; The client uploads the local label proportion, and the server estimates the global label proportion; each client downloads the global uniform prototype set and the global label proportion set; Step 2: shaping the feature space using the global uniform prototype to enhance the intra-class compactness and inter-class distance; For each instance and its features , contrastive learning is performed using a global uniform prototype to pull features of the same class closer together and adjust the degree of repulsion between different classes according to the global label proportion to regularize the distorted feature space; wherein, is a class number, is a class number, and represents a global uniform prototype of the y-th class and the k-th class, respectively, is a temperature parameter, is an indicator function representing the weight of the k-th class, is a global label proportion of the k-th class; Step 3: federated prototype mixing; The client extracts a local feature set from the local data set using the global model, and randomly selects a subset from the local feature set to generate virtual features of each class; specifically, balanced virtual features are generated by mixing the local features in the subset with the global uniform prototype; finally, the client re-trains the classifier according to the generated balanced virtual feature set; wherein the global model is the result of the aggregation of the multi-party participant network model.

2. The federated learning method based on unified representation and classifier rectification according to claim 1, characterized in that, The specific implementation of step 1 comprises the following sub-steps: Step 1.1: For each client and each category , compute local prototypes based on the local dataset ; wherein, is a sample and a sample label, is a subset of the dataset containing samples of the k-th class in client m, is the number of samples, is the global model downloaded by client m, is the number of classes, is the feature dimension; Step 1.2: The client will Normalized local prototype set Upload to server; Step 1.3: The server aggregates the received prototype set by a weighted average operation to generate a global uniform prototype and the global uniform prototype set ; wherein, is a set of clients, is a client number, m represents a current client, and n represents all participating clients; Step 1.4: Client uploads local label proportions Server estimates global label proportions ; wherein, is the total number of samples for client m; Step 1.5: Each client downloads the global uniform set of prototypes and the global set of label proportions .

3. The federated learning method based on unified representation and corrector classifier according to any one of claims 1-2, characterized in that, The specific implementation of step 3 comprises the following sub-steps: Step 3.1: Client extracts local feature set using global model ; Step 3.2: For each class , a subset is randomly selected from the local feature set , used to generate the virtual feature set for that class, the subset is denoted as follows; wherein, represents the i-th feature randomly sampled from the local feature set in the client m, is the label corresponding to the feature, is the number of features in the subset . Step 3.3: Each feature in the subset is mixed with the global uniform prototype and generates a balanced virtual feature set ; wherein, represents the i-th feature in the k-th feature subset on the client m its corresponding global unified prototype the fused virtual feature; represents and the fusion ratio between them, sampled from a balanced distribution; is the virtual feature set of the k-th class on the client m, represents the i-th virtual feature in is the label corresponding to the virtual feature; Step 3.4: Client re-trains the classifier according to the virtual feature set , with the following loss function , and uploads the updated local model to the server: wherein, is a balanced virtual feature set, is a virtual feature set, is a virtual feature and corresponding label sampled from represents inputting the virtual feature into a classifier , represents a softmax function, is a loss function for retraining the classifier using the balanced virtual feature set , represents the classifier after retraining.

4. A federated learning system based on unified representation and classifier rectification, characterized in that, The method comprises the following modules: Module 1, for constructing a global uniform prototype; First, each client calculates a local prototype based on its local data set; Then, the client uploads the normalized local prototype set to the server; then, the server aggregates the local prototypes of different clients through a weighted average operation to generate a global uniform prototype; The client uploads the local label proportion, and the server estimates the global label proportion; each client downloads the global uniform prototype set and the global label proportion set; Module 2, for shaping the feature space using the global uniform prototype to enhance the intra-class compactness and inter-class distance; For each instance and its features , contrastive learning is performed using a global uniform prototype to pull features of the same class closer together and adjust the degree of repulsion between different classes according to the global label proportion to regularize the distorted feature space; wherein, is a class number, is a class number, and represent the global uniform prototypes of the y-th and k-th class, respectively, is a temperature parameter, is an indicator function representing the k-th class weight, is the k-th global label proportion; Module 3, for federated prototype mixing; First, the client extracts a local feature set from the local data set using the global model, and randomly selects a subset from the local feature set to generate virtual features of each class; then, balanced virtual features are generated by mixing the local features with the global uniform prototype; finally, the client re-trains the classifier according to the generated balanced virtual feature set; wherein the global model is the result of the aggregation of the multi-party participant network model.

5. The federated learning system based on unified representation and classifier rectification according to claim 4, wherein, Module 1 comprises the following sub-modules: Module 1.1, for each client and each category based on the local dataset computes a local prototype ; wherein, is a sample and a sample label, is a subset of the dataset containing samples of the k-th class in client m, is the number of samples, is the global model downloaded by client m, is the number of classes, is the feature dimension; Module 1.2, for the client to upload Normalized local prototype set to the server; Module 1.3 for the server to generate a global uniform prototype by aggregating the received set of prototypes with a weighted average operation and the set of global uniform prototypes ; wherein, is a set of clients, is a client sequence number, m represents a current client, and n represents all participating clients; Module 1.4, for client uploading local label proportions server estimating global label proportions ; wherein, is the total number of samples for client m; Module 1.5, for downloading a global uniform set of prototypes for each client and a global set of label proportions .

6. The federated learning system based on unified representation and corrector classifier according to any one of claims 4-5, characterized in that, Module 3 comprises the following sub-modules: Module 3.1, for the client to extract a local feature set using a global model ; Module 3.2, for each category randomly selecting a subset from the local feature set for generating a virtual feature set for that category, the subset is represented as follows; wherein, represents the i-th feature randomly sampled from the local feature set in the client m, is the label corresponding to the feature, is the number of features in the subset in the client m. Module 3.3, for each feature in the subset of global unified prototypes mixing, generating balanced virtual feature sets ; wherein, represents the i-th feature in the k-th feature subset on the client m its corresponding global unified prototype the fused virtual feature; represents and the fusion ratio between them, sampled from a balanced distribution; is the virtual feature set of the k-th class on the client m, represents the i-th virtual feature in the set is the label corresponding to the virtual feature; Module 3.4, for the client to retrain the classifier according to the virtual feature set , retrain the classifier according to the following loss function , and upload the updated local model to the server: wherein, is a balanced virtual feature set, is a virtual feature set, is a virtual feature and corresponding label sampled from represents inputting the virtual feature into a classifier , represents a softmax function, is a loss function for retraining the classifier using the balanced virtual feature set , represents the classifier after retraining.

Images