Data and model heterogeneous federal learning method for global decision boundary distillation learning

Abstract

The invention discloses a data and model heterogeneous federated learning method of global decision boundary distillation learning. Two key problems of data heterogeneity and heterogeneous model performance difference are respectively solved through two sub-methods. According to the first sub-method, aiming at the problem of data heterogeneity, multiple prototypes are locally clustered, private data field distribution information is fully condensed, irrelevant distillation is introduced in a local supervised learning stage, and local knowledge forgetting is relieved. According to the second sub-method, global decision boundary distillation learning is provided for the problem of performance difference of the heterogeneous model, a global decision boundary learner is maintained and updated in a server, the local model is optimized from the perspective of the global decision boundary, and the influence of conflict information on the local model is reduced. The performance of the method is superior to that of various most advanced federal learning methods, the method can better adapt to challenging scenes, and meanwhile, the method has higher communication efficiency and better privacy reliability.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a data and model heterogeneous federated learning method for global decision boundary distillation learning. Background Art

[0002] With the rapid development of artificial intelligence technology, the size and quality of datasets have become key factors in determining model performance. In real-world scenarios, data is often distributed across a large number of terminal devices or institutions. However, growing concerns about data privacy, coupled with increasingly stringent laws and regulations on data sharing, have made aggregating and building large-scale datasets a significant challenge. Federated learning (FL), as an emerging distributed machine learning paradigm focused on data privacy protection, enables collaborative model training without transmitting private data. This concept was first proposed by McMahan et al. from Google Research in 2017. In traditional federated learning algorithms, clients achieve communication and collaboration by exchanging model parameters, ensuring that private data remains locally, while the server aggregates updates to optimize the global model.

[0003] Federated learning has a wide range of applications, but its practical deployment faces significant challenges, particularly data and model heterogeneity. In real-world scenarios, private data often exhibits non-independent and identically distributed (non-IID) characteristics, leading to conflicts between local model optimization and global optimization objectives, thereby reducing overall model performance. To address this issue, researchers have proposed a number of personalized federated learning algorithms, including FedProx, FedAMP, and SCAFFOLD. These methods primarily leverage gradient-based information for personalized aggregation and mitigate the negative impact of data heterogeneity by constraining optimization directions.

[0004] While these algorithms have achieved some success, they still rely on the transmission of model parameters for collaborative training, which requires all participants to use the same model architecture. However, in real-world scenarios, participants often have varying local computing capabilities and bandwidth constraints, leading to the development of heterogeneous federated learning. This paradigm avoids sharing private model parameters and is compatible with diverse local model architectures, making it more practical and valuable.

[0005] Some heterogeneous federated learning algorithms use public datasets to promote communication and collaboration between heterogeneous models. These methods transmit the logit outputs of private models on public datasets and use knowledge distillation (KD) technology for collaborative training. However, their performance is highly dependent on the quality and adaptability of public datasets. Other heterogeneous federated learning algorithms (such as LG-FEDAVG and FedGH) address model heterogeneity by adopting a global classifier: the server updates the global classifier and distributes it to the client to replace the local classifier. This requires repeated uploading of the local classifier in each round, which in turn causes a certain amount of communication overhead pressure.

[0006] Prototype-based methods, such as FedProto and FedTGP, have been proposed to achieve heterogeneous federated learning by transferring category prototypes from the client's local feature space and generating global prototypes to guide local model updates. However, the effectiveness of these methods relies on the quality of the global prototypes. Given the heterogeneity of local models between clients, differences in their feature extraction capabilities often lead to conflicts between the global and local prototypes in FedProto. Similarly, FedTGP faces the challenge of effective convergence during global prototype generation, which limits its performance in real-world scenarios. Summary of the Invention

[0007] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0008] To this end, this paper proposes a data and model heterogeneous federated learning method for global decision boundary distillation learning. On the server side, a global decision boundary learner is established and maintained using local multiple prototypes. The multiple prototypes formed by local clustering efficiently condense the feature distribution of the private data domain, helping the global decision boundary learner learn the global decision boundary from a global perspective. During the local update phase, distillation learning is used to optimize and guide local model updates, reducing the negative impact of heterogeneous feature extractor capability differences and improving local model performance and generalization. Unrelated knowledge distillation is introduced during the local supervised learning phase to decouple traditional knowledge distillation and alleviate the forgetting of private data domain knowledge during multi-round collaborative learning.

[0009] To achieve the above objectives, the present invention proposes a data and model heterogeneous federated learning method for global decision boundary distillation learning, including:

[0010] Obtain local classifier parameters obtained by each client performing local supervised training on the local classifier using local private data;

[0011] Establishing a global decision boundary learner based on the local classifier parameters, initializing the learner parameters to obtain global decision boundary information, and sending the global decision boundary information to the client;

[0012] Notify the client to introduce irrelevant class knowledge distillation during the local training phase, select the best model in the collaborative iteration round as the teacher model, and guide the local model to capture the relationship between categories to obtain optimized feature representation;

[0013] Obtain the local prototype set generated by the client based on the optimized feature representation clustering;

[0014] Merge the local prototype sets into a global prototype set, define the learner training loss function according to the optimization objective formula to train the global decision boundary learner, and send the trained learner parameters to the clients participating in the collaborative training;

[0015] Obtain an optimized local model obtained by guiding the client to optimize the updated local classifier based on the trained learner parameters from the perspective of the global decision boundary.

[0016] The data and model heterogeneous federated learning method for global decision boundary distillation learning in an embodiment of the present invention may also have the following additional technical features:

[0017] In one embodiment of the present invention, obtaining a local prototype set generated by the client based on optimized feature representation clustering includes:

[0018] The client uses the optimized clustering algorithm Kmeans++ and the optimized feature representation based on the probability distribution to select the initial center point of each cluster, and clusters and updates the center points of other feature vectors to obtain the clustering results. Among them, other feature vectors include all feature vectors except the feature vector currently selected as the cluster center point;

[0019] Based on the clustering results, multiple local prototype sets are generated for each class using an unsupervised clustering algorithm.

[0020] In one embodiment of the present invention, the local supervised training loss function is:

[0021]

[0022] in, is a sample space, is the feature space, It is a label space. There are M clients participating in the federated learning collaborative training. Each client can only access the local private dataset. The local model of client i is composed of feature extractor f i and classifier h i Composition, feature extractor f i :X→Z, with parameters Classifier h i :Z→Y, the parameters are

[0023] The overall collaborative training objectives are:

[0024]

[0025] In one embodiment of the present invention, the feature space is input into the global decision boundary learner g and the output is the label space, with the parameter θ g , the structure is the same as h;

[0026] The server initializes the global decision boundary learner parameters, the formula is:

[0027]

[0028] After initialization, the learner g learns the decision boundary of each client for the private data domain through local clustering multi-prototype learning to form a global decision boundary; the optimization objective is defined as:

[0029]

[0030] Among them, P g It represents the set of all client local clustered multi-prototypes selected in the current round, defined as P g ={P 1 ∪P 2 ∪…∪P N}, N is the number of clients participating in collaborative training in each round, and c is the category label corresponding to the prototype p.

[0031] In one embodiment of the present invention, the local training loss of client i is defined as:

[0032]

[0033] Knowledge distillation chooses to introduce the temperature parameter softening probability in the output of the logits layer. The calculation formula is:

[0034]

[0035] where s s,c is the softened probability of category c, z c is the value of the corresponding category c in the output of the logits layer, and the softened label is

[0036] The distillation loss function is:

[0037]

[0038] The distillation loss is decomposed and decoupled into two parts, namely the label class distillation loss and the irrelevant class distillation loss. The decoupled loss is:

[0039]

[0040] The two parts of the loss have different degrees of influence during the gradient calculation process:

[0041]

[0042] in represents the logit layer output of the student model for label class c,

[0043] Define irrelevant class knowledge distillation as:

[0044]

[0045] In one embodiment of the present invention, obtaining a local prototype set generated by the client based on the optimized feature representation clustering further includes:

[0046] Client i randomly selects the first prototype center point of private data c feature After that, other eigenvalues Probability of being selected The calculation formula is as follows:

[0047]

[0048] in for The minimum distance to the selected center point is calculated as follows:

[0049] Local multi-prototype clustering is expressed as:

[0050]

[0051] The prototype set formed by the local clustering of client i is recorded as:

[0052] In one embodiment of the present invention, the learner training loss function is:

[0053]

[0054] Where c is the prototype p corresponding to the real one-hot label, g(p,θ g ) i represents the predicted probability of the learner g for the i-th category.

[0055] In one embodiment of the present invention, the loss function for optimizing the local decision boundary and the spatial distribution of data features is defined as:

[0056]

[0057] Where α is the global decision boundary learning ratio;

[0058] The loss of the global decision boundary learner g guiding the optimization training of the local model of client i is defined as:

[0059]

[0060] Where σ() is the softmax function and T is the distillation temperature parameter.

[0061] The data and model heterogeneous federated learning method for global decision boundary distillation learning in the embodiment of the present invention has communication overhead in the collaboration stage mainly divided into two parts: the client uploading local clustered multi-prototypes and the server sending the global decision boundary learner. The local clustered multi-prototypes uploaded by the client are essentially multiple class representatives of each type of local data in the feature space of the client. Compared with other algorithms that upload local model complete parameters, generator parameters or public data, the prototype has a smaller size and its communication overhead can be ignored. The global decision boundary learner sent by the server is a lightweight and small learner with a small number of parameters and is only sent in the model update stage. It has a smaller communication overhead compared to the FedAvg algorithm broadcasting the complete parameters of the global model. Therefore, the present invention has higher communication efficiency.

[0062] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0064] Figure 1 is a flowchart of a data and model heterogeneous federated learning method for global decision boundary distillation learning according to an embodiment of the present invention;

[0065] Figure 2 2 is an architectural diagram of a data and model heterogeneous federated learning method for global decision boundary distillation learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0067] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0068] The following describes the data and model heterogeneous federated learning method for global decision boundary distillation learning proposed according to an embodiment of the present invention with reference to the accompanying drawings.

[0069] Figure 1 is a flowchart of a data and model heterogeneous federated learning method for global decision boundary distillation learning according to an embodiment of the present invention. Figure 2 This is the architecture diagram of this method. Figure 1 and Figure 2 As shown, the method includes but is not limited to the following steps:

[0070] S1, obtaining local classifier parameters obtained by each client performing local supervised training on the local classifier using local private data.

[0071] Specifically, before collaborative training, each client needs to perform pre-training and use private data for supervised learning. The local supervised training loss function can be written as:

[0072]

[0073] in, is an instance space, is the feature space, is the label space. A total of M clients participate in the federated learning collaborative training, and each client can only access the local private dataset The local model of client i is composed of feature extractor f i and classifier h i Composition, feature extractor f i : Its parameters are Classifier h i : Its parameters are Feature extractors f for different clients i The structure is different, the classifier h of all clients i Have the same structure.

[0074] At the same time, in terms of form, the overall collaborative training objectives are:

[0075]

[0076] S2: Establish a global decision boundary learner based on the local classifier parameters, initialize the learner parameters to obtain global decision boundary information, and send it to the client.

[0077] Specifically, after the local pre-training is completed, all clients send their local classifiers to the server. The server establishes a global decision boundary learner, initializes the learner parameters, forms a global decision boundary, and sends it to the client. The global decision boundary learner g established by the server has the feature space as input and the label space as output. The parameter is θ g , the structure is the same as h.

[0078] The server initializes the global decision boundary learner parameters, which are:

[0079]

[0080] After initialization, the learner g learns the decision boundary of each client for the private data domain through local clustering multi-prototype learning to form a global decision boundary. Its optimization goal is defined as:

[0081]

[0082] Among them, P g It represents the set of all client local clustered multi-prototypes selected in the current round, defined as P g ={P 1 ∪P 2 ∪…∪P N}, N is the number of clients participating in collaborative training in each round, and c is the category label corresponding to the prototype p.

[0083] S3 notifies the client to introduce irrelevant class knowledge distillation in the local training phase, selects the best model in the collaborative iteration round as the teacher model, and guides the local model to capture the relationship between categories to obtain optimized feature representation.

[0084] Specifically, in the local training phase, we introduce unrelated class knowledge distillation into supervised learning, select the optimal model in the collaborative iteration round as the teacher model, and guide the local model to capture the relationship between categories. The local training loss of client i is defined as:

[0085]

[0086] Traditional knowledge distillation chooses to introduce the temperature parameter softening probability in the output of the logits layer. The specific calculation formula is:

[0087]

[0088] where s s,c is the softened probability of category c, zc is the value of the corresponding category c in the output of the logits layer, and the softened label is

[0089] The distillation loss function can be written as:

[0090]

[0091] The distillation loss is further decomposed into two parts, namely the label class distillation loss and the irrelevant class distillation loss. The decoupled loss can be written as:

[0092]

[0093] Backpropagation in the model update process is implemented based on the calculated gradient. This paper explores the different impacts of the two losses in the gradient calculation process:

[0094]

[0095] in It represents the logit layer output of the student model for the label class c. According to the above formula, it can be found that the gradient value range of the label class distillation part is the same as The value is related to the performance of the teacher model.

[0096] The present invention hopes that the student model can learn the posterior class relationship of the private data domain in the teacher model, and alleviate the forgetting of private data domain knowledge in multiple rounds of collaborative training. Therefore, the present invention defines irrelevant class knowledge distillation as:

[0097]

[0098] S4, obtains the local prototype set generated by the client based on the optimized feature representation clustering.

[0099] Specifically, after local training, the selected clients cluster to generate a local prototype set and send it to the server.

[0100] Step S4.1: Select the optimized clustering algorithm Kmeans++ and continuously select v initial center points based on the probability distribution. Then cluster and update the center points for other feature vectors, repeating the process until the center points converge or the maximum number of iterations is reached. Client i randomly selects the first prototype center point of the private data c class feature. After that, other eigenvalues Probability of being selected The calculation formula is as follows:

[0101]

[0102] in for The minimum distance to the selected center point is calculated as follows:

[0103] Step S4.2: Perform local multi-prototype clustering. Generate v prototypes for each class using an unsupervised clustering algorithm, effectively condensing the feature information of each class of data while retaining the marginal feature information of the class data. The local prototype set is then sent to the server. Local multi-prototype clustering can be expressed as:

[0104]

[0105] The prototype set formed by the local clustering of client i is recorded as:

[0106] For each client, execute step S3 and step S4.

[0107] S5, merge the local prototype sets into the global prototype set, and define the learner training loss function according to the optimization objective formula to train the global decision boundary learner, and send the trained learner parameters to the clients participating in the collaborative training.

[0108] Specifically, the server merges the local prototype sets received from the selected clients into a global prototype set, defines the learner training loss function according to the optimization objective formula, trains the global decision boundary learner, and sends it to the clients participating in the collaborative training. The learner training loss function is:

[0109]

[0110] Where c is the prototype p corresponding to the real one-hot label, g(p,θ g ) i represents the predicted probability of the learner g for the i-th category.

[0111] S6, obtaining an optimized local model obtained by guiding the client to optimize the updated local classifier based on the trained learner parameters from the perspective of the global decision boundary.

[0112] Specifically, the client uses the received learner to guide the optimization of the local model from the perspective of the global decision boundary. The local model is updated and trained. The loss function for optimizing the local decision boundary and the spatial distribution of data features is defined as:

[0113]

[0114] Where α is the global decision boundary learning ratio, which controls the learning ratio of the local model to the global decision boundary.

[0115] The loss of the global decision boundary learner g guiding the optimization training of the local model of client i is defined as:

[0116]

[0117] Where σ() is the softmax function and T is the distillation temperature parameter. This paper utilizes the idea of knowledge distillation to soften labels in order to allow the local model to learn the posterior class relationship of the global decision boundary learner g for the private data domain feature space, further optimizing the local decision boundary.

[0118] For each client, execute step S6.

[0119] Repeat steps S3 to S6 until the predetermined number of iterations is reached.

[0120] To demonstrate the effectiveness of this method, we evaluated it on several commonly used image datasets for multi-class classification tasks, including CIFAR-10, CIFAR-100, TinyImageNet, and Flowers102. We used four different feature extractors: a 4-layer CNN, GoogleNet, MobileNet v2, and ResNet18. During initialization, each client was randomly assigned one of these four models.

[0121] Data distribution method. In the experiment, the present invention adopts the widely used statistical heterogeneity setting to evaluate the data non-independent and identically distributed (non-IID) scenario, namely the pathological setting and the practical setting. In the pathological setting, the present invention allocates 2 categories of CIFAR-10, 10 categories of CIFAR-100, and 20 categories of Tiny-ImageNet and Flowers102 to each client as private data. In the practical setting, the present invention uses the Dirichlet distribution Dir(β) to sample each data category, and the default β value is set to 0.1. All data sets are divided into a ratio of 75% training set and 25% test set.

[0122] Baseline methods. We select four popular and relevant heterogeneous federated learning algorithms as baseline methods, including the global classifier-based methods LG-FedAvg (NeurIPS 2019) and FedGH (MM 2023), and the prototype learning-based methods FedProto (AAAI 2022) and FedTGP (AAAI 2024).

[0123] Implementation details. In the experiment, we simulated 20 clients participating in federated learning, with a client participation rate of 1. In each global iteration, the client had 1 local training round, 5 local model update cycles, and 30 training rounds for the server-side learner. The batch size was set to 10, the learning rate was 0.01, and global training consisted of 100 iterations. The hyperparameters v and α were set to 5 and 0.7, respectively. The accuracy of each global experiment was the average of the test set results of each client, and the best global accuracy across all experiments was reported.

[0124] Experimental results under different data distributions. Table 1 shows the accuracy (%) of the algorithms under different statistical heterogeneity settings. The present invention achieves the best performance in the four datasets under both data distribution settings, with the highest improvement of 9.27%, demonstrating the superiority of the present invention. Due to the differences between heterogeneous models, FedProto generates global prototypes by simple averaging, resulting in significant deviations between these prototypes and the category centers in the private data domain. Therefore, its performance is weaker on datasets with more categories. On the other hand, the global prototypes generated by FedTGP during client training are difficult to converge. LG-FEDAVG and FedGH replace local classifiers with global classifiers, but fail to effectively resolve information conflicts between clients, resulting in a certain degree of local knowledge forgetting. In contrast, the present invention reduces the negative impact of performance differences between heterogeneous feature extractors by optimizing the decision boundary. The algorithm optimizes the local decision boundary from a global perspective and introduces irrelevant category distillation to alleviate local knowledge forgetting. Therefore, it significantly outperforms other baseline algorithms on more challenging datasets.

[0125] Table 1

[0126]

[0127] Hyperparameter study. The present invention has two hyperparameters: the number of local prototype clusters v and the global decision boundary learning rate α. Under actual settings, the present invention conducted experiments on the Cifar100 and Cifar10 datasets to explore the impact of different hyperparameter configurations on performance. The experimental results are shown in Tables 2 and 3. Table 2 shows the accuracy (%) under different numbers of local prototype clusters, and Table 3 shows the accuracy (%) under different global decision boundary learning ratio values. When the number of local prototype clusters v is set to 1, the algorithm performance will decrease compared to using multiple prototypes, because a single prototype cannot effectively capture the distribution of the local data feature space. When the global decision boundary learning rate α is set to a higher value, it will inhibit the model's learning of the local decision boundary, but the performance is still better than all baseline algorithms. A suitable α value can enable the global decision boundary learner to better guide the optimization of the local model, thereby bringing better performance.

[0128] Table 2

[0129] Dataset v＝1 v＝5 v＝10 CIFAR-100 44.78 45.09 44.96 CIIAR-10 87.25 87.90 87.84

[0130] Table 3

[0131]

[0132]

[0133] Ablation experiments. We conducted ablation experiments on four datasets to evaluate the effectiveness of the key components of the present invention. The results are shown in Table 4, which shows the accuracy (%) of the ablation experiments conducted by the key components of the present invention. The experiments show that GDBD-MP (Global Decision Boundary - Multiple Prototypes) effectively optimizes the local decision boundary. In addition, the introduction of irrelevant category distillation enables the model to learn the posterior category relations unique to the private data domain, retaining local knowledge and further improving the performance of the local model.

[0134] Table 4

[0135]

[0136] More challenging task scenarios. In order to evaluate the performance of the present invention under more challenging settings, the present invention increases the number of heterogeneous models from four to eight by adding ResNet34, ResNet50, ResNet101 and ResNet152, thereby increasing the diversity between models. The present invention conducted experiments on the Tiny-ImageNet dataset for two data distribution settings. The results are shown in Table 5, which is the accuracy (%) of the present invention in more challenging scenarios. In the case of multi-category few-shot learning, thanks to the global decision boundary distillation, the present invention maintains superior performance under both data distribution settings, with the accuracy improvement reaching up to 6.75%. When faced with a situation where there are significant differences between heterogeneous models, optimizing local decision boundaries from a global perspective, rather than adopting prototype learning and sharing global heads, can better adapt to more complex data distributions and alleviate the negative impact of differences between heterogeneous models.

[0137] Table 5

[0138] set up Actual Settings Pathology setting FadProto 24.01 32.78 FedTGP 27.53 32.29 LG-FedAvg 29.17 34.87 FedGH 26.88 34.49 The present invention 30.76 35.26

[0139] The data and model heterogeneous federated learning method for global decision boundary distillation learning of the embodiment of the present invention is adopted. It takes the form of transmitting local clustered multiple prototypes. The prototype is the class center of the feature space. This low-dimensional data is irreversible, and the adversary cannot perform data reverse reasoning attacks, which has good privacy reliability. The input of the global decision boundary learner is the feature space, and it mainly learns the decision boundary, not the original data. In view of the security of low-dimensional data, the adversary cannot infer the original data through the global decision boundary learner. The present invention improves the accuracy of the local model under data heterogeneity. The global prototype generated by simple averaging is far from the class center of the private data domain feature space, and its performance is poor on data sets with more data categories; the global prototype generated by client training is difficult to converge. Directly replacing the local classifier with a global classifier cannot effectively resolve the conflicting information between different clients, resulting in a certain degree of local knowledge forgetting. From the perspective of optimizing the decision boundary, the negative impact of performance differences among heterogeneous feature extractors is reduced, the local decision boundary is optimized from a global perspective, the local model performance is improved, and irrelevant class distillation is introduced to alleviate the forgetting of local knowledge. Therefore, it has greater performance advantages than other baseline algorithms on more challenging datasets.

[0140] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0141] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

Claims

1. A data and model heterogeneous federated learning method for global decision boundary distillation learning, characterized by: include: Obtain local classifier parameters obtained by each client performing local supervised training on the local classifier using local private data; Establishing a global decision boundary learner based on the local classifier parameters, initializing the learner parameters to obtain global decision boundary information, and sending the global decision boundary information to the client; Notify the client to introduce irrelevant class knowledge distillation during the local training phase, select the best model in the collaborative iteration round as the teacher model, and guide the local model to capture the relationship between categories to obtain optimized feature representation; Obtain the local prototype set generated by the client based on the optimized feature representation clustering; Merge the local prototype sets into a global prototype set, define the learner training loss function according to the optimization objective formula to train the global decision boundary learner, and send the trained learner parameters to the clients participating in the collaborative training; Obtain an optimized local model obtained by guiding the client to optimize the updated local classifier based on the trained learner parameters from the perspective of the global decision boundary.

2. The method according to claim 1, characterized in that Get the local prototype set generated by the client based on the optimized feature representation clustering, including: The client uses the optimized clustering algorithm Kmeans++ and the optimized feature representation based on the probability distribution to select the initial center point of each cluster, and clusters and updates the center points of other feature vectors to obtain the clustering results. Among them, other feature vectors include all feature vectors except the feature vector currently selected as the cluster center point; Based on the clustering results, multiple local prototype sets are generated for each class using an unsupervised clustering algorithm.

3. The method according to claim 1, characterized in that The local supervised training loss function is: in, is a sample space, is the feature space, It is a label space. There are M clients participating in the federated learning collaborative training. Each client can only access the local private dataset. The local model of client i is composed of feature extractor f i and classifier h i Composition, feature extractor f i :X→Z, with parameters Classifier h i :Z→Y, the parameters are The overall collaborative training objectives are:

4. The method according to claim 3, characterized in that The feature space is input into the global decision boundary learner g and output as label space with parameter θ g , the structure is the same as h; The server initializes the global decision boundary learner parameters, the formula is: After initialization, the learner g learns the decision boundary of each client for the private data domain through local clustering multi-prototype learning to form a global decision boundary; the optimization objective is defined as: Among them, P g It represents the set of all client local clustered multi-prototypes selected in the current round, defined as P g ={P 1 ∪P 2 ∪…∪P N }, N is the number of clients participating in collaborative training in each round, and c is the category label corresponding to the prototype p.

5. The method according to claim 4, characterized in that Define the local training loss of client i as: Knowledge distillation chooses to introduce the temperature parameter softening probability in the output of the logits layer. The calculation formula is: where s a,c is the softened probability of category c, z c is the value of the corresponding category c in the output of the logits layer, and the softened label is The distillation loss function is: The distillation loss is decomposed and decoupled into two parts, namely the label class distillation loss and the irrelevant class distillation loss. The decoupled loss is: The two parts of the loss have different degrees of influence during the gradient calculation process: in represents the logit layer output of the student model for label class c, Define irrelevant class knowledge distillation as:

6. The method according to claim 5, characterized in that Get the local prototype set generated by the client based on the optimized feature representation clustering, including: Client i randomly selects the first prototype center point of private data c feature After that, other eigenvalues Probability of being selected The calculation formula is as follows: in for The minimum distance to the selected center point is calculated as follows: Local multi-prototype clustering is expressed as: The prototype set formed by the local clustering of client i is recorded as:

7. The method according to claim 6, characterized in that The loss function of the learner training is: Where c is the prototype p corresponding to the real one-hot label, g(p,θ g ) i represents the predicted probability of the learner g for the i-th category.

8. The method according to claim 7, characterized in that The loss function that optimizes the local decision boundary and the spatial distribution of data features is defined as: Where α is the global decision boundary learning ratio; The loss of the global decision boundary learner g guiding the optimization training of the local model of client i is defined as: Where σ() is the softmax function and T is the distillation temperature parameter.

Images