A personalized global prototype federated learning method and system based on adaptive feature alignment

Abstract

The application discloses a kind of personalized global prototype federated learning method and system based on adaptive feature alignment, the method adopts adaptive feature alignment mechanism to realize the two-way adaptive alignment of local feature space and global feature space, generates personalized global prototype as the final guidance of local model extraction feature, and introduces frozen global classifier to explicitly introduce global information into client, input the frozen global classifier with personalized global prototype, and then fuse its output with the output of client training classifier to obtain final prediction result output.Compared with other personalized federated learning methods, the method and system of the application can more effectively utilize global information, and through the design of personalized global prototype, the generalization ability of local model is significantly enhanced, thereby improving the overall model performance.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of distributed machine learning, and particularly relates to a personalized global prototype federated learning method and system based on adaptive feature alignment. BACKGROUND

[0002] As a distributed machine learning method, federated learning can coordinate multiple data islands to jointly train a model while ensuring data privacy, and has been widely applied in the fields of medical health, financial risk control, and Internet of Things (McMahan et al., "Communication-Efficient Learning of Deep Networks from Decentralized Data", AISTATS 2017).

[0003] Traditional federated learning methods (such as FedAvg) aim to learn a unified global model with good generalization ability, but in practical applications, there are often Non-IID characteristics in the data distribution between clients, leading to a decline in the performance of the unified model (Tian Li et al., "Federated Optimization in Heterogeneous Networks", MLSys 2020).

[0004] To address the above problems, personalized federated learning (pFL) methods have emerged, aiming to optimize the performance of local models for each client. Existing personalized federated learning methods mainly include: methods based on model parameter alignment, such as pFedMe (Canh T. Dinh et al., "Personalized Federated Learning with Moreau Envelopes", NeurIPS 2020), which introduces regularization between local and global models to improve personalization; methods based on prototype alignment, such as FedProto (Yue Tan et al., "FedProto: Federated Prototype Learning Across Heterogeneous Clients", AAAI 2022), which uploads and aggregates class prototypes to provide global knowledge to guide local training.

[0005] However, existing personalized federated learning methods still have the following problems:

[0006] Global information bias: existing methods rely on global information (global model or global prototype) to guide local training, and usually generate global model or global prototype through weighted average, which is easily affected by large amount of data of clients, leading to global information deviation to dominant clients, and difficult to effectively represent data distribution diversity of all clients.

[0007] Local feature noise amplification: local embedding features are easily disturbed by noise in the uploading process, and in the global information generation process, unstable factors are easily introduced by directly aggregating local embedding features, reducing the guiding effect of global prototype.

[0008] Implicit alignment limitation: most existing feature alignment methods mainly reduce the distance between local embedding and global prototype, that is, only by simply narrowing the distance between local features and global prototype to align, lack of fine-grained adaptability regulation to distinguish similar and dissimilar features, resulting in insufficient distinguishability in feature space.

[0009] Single global information integration method: existing personalized federated learning methods mostly use implicit methods in integrating global information, which fails to fully utilize global model and prototype knowledge, and does not explicitly and fully inject global knowledge into local model, limiting the improvement of model generalization ability.

[0010] Therefore, there is an urgent need for a new personalized federated learning method that can ensure the stability and diversity of global information while enhancing the mutual adaptability of local features and global information, and explicitly integrating global knowledge, to improve the performance of local model in heterogeneous data environment. SUMMARY

[0011] The present application aims to overcome the shortcomings of the prior art and provides a personalized global prototype federated learning method and system based on adaptive feature alignment, which generates personalized global prototypes through an adaptive feature alignment mechanism to improve the individualization effect and generalization ability of local models in heterogeneous data environments.

[0012] To achieve the above technical purposes, the present application adopts the following solutions:

[0013] A personalized global prototype federated learning method based on adaptive feature alignment, comprising:

[0014] For a system comprising a server and N clients, the client comprises a feature extractor and a classifier;

[0015] Adaptive feature alignment mechanism is used to realize bidirectional adaptive alignment between local feature space and global feature space, including: the client converts the original global prototype P aggregated by the server into personalized prototype P' by mapping G-P model, which integrates local domain information , and then process the personalized prototype through the frozen GP model Obtain a stable personalized prototype PGP. The global parameters of the frozen GP model downloaded from the server remain fixed and not updated during the local training process. Then, the original global prototype P and the stable personalized prototype PGP are mean-fused to obtain a personalized global prototype as the final guide for feature extraction by the local model.

[0016] In addition, a frozen global classifier is introduced to explicitly introduce global information into the client. The frozen global classifier is generated by the server by aggregating the local classifiers of all clients. The personalized global prototype is input into the frozen global classifier, and then it is fused with the output of the client's local model to obtain the final prediction result output.

[0017] Furthermore, the comprehensive optimization goal of the client local update phase is the update objective function of the personalized global prototype , loss function for local model feature extraction training And the supervised objective function predicted by the local model All three are minimized.

[0018] Furthermore, the update objective function of the personalized global prototype is is a loss function based on negative log-likelihood, which is used to measure the local embedding features that do not participate in parameter updates. The degree of match between the personalized prototype of the corresponding category. It is expressed as the negative logarithm of a ratio value plus 1, where the numerator is Represents local embedded features that do not participate in parameter updates A personalized prototype of its similar labels The similarity score between the two is the value after exponential decay; Represents personalized prototypes corresponding to all different class labels Local embedding features that do not participate in parameter updates The sum of the similarity scores is also processed with exponential decay. Finally, the ratio of the numerator and denominator is increased by 1, the natural logarithm is taken, and the negative value is taken to form the final loss value. This makes the obtained personalized prototype closer to the local embedded feature. Specifically, it can be expressed as:

[0019]

[0020] in, Indicates that the local embedded features do not participate in parameter update , For local embedding features Personalized prototypes for data of the same label category, For local embedding features Personalized prototypes for data of different label categories, is a function, indicating .

[0021] Furthermore, the loss function of local model feature extraction training is The objective function is constructed in the form of negative log-likelihood, which measures the similarity between the local features and the personalized prototype after fusion with the corresponding category. It is expressed as the negative logarithm of a ratio value plus 1, where the numerator is Representing local features The similarity score between the personalized prototype and its similar tags is enhanced by exponential decay. Indicates all The fused personalized prototypes and local features corresponding to different class labels The sum of the similarity scores is also processed with exponential decay. Finally, the ratio of the numerator and denominator is increased by 1, the natural logarithm is taken, and the negative value is taken to form the final loss value. It can be expressed as:

[0022]

[0023] in, It is the global parameter downloaded from the server. Indicates local embedded features that do not participate in the update Personalized prototypes for data of the same label category, For local embedded features that do not participate in the update Personalized prototypes for data of different label categories, For frozen GP model processing The output, For frozen GP model processing Output.

[0024] Furthermore, the supervised objective function of the local model prediction is , which calculates the one-hot encoding vector of the true label The cross entropy between the prediction output and the model is used to measure the degree of match between the two. The model prediction output is obtained by the sum of two parts: the first part is the classifier result during the training process The second part is the result of a frozen classifier processing the fusion of personalized features The outputs of these classifiers are passed through the softmax function Normalization is performed to generate the final probability distribution. It can be expressed as:

[0025]

[0026] in, a one-hot encoding vector for a true label, a softmax function, an output of the local model classifier, an output of the frozen global classifier, parameters of the local model classifier, local embedding features collected by the client, parameters of the frozen global classifier.

[0027] Further, in the training process, the server randomly initializes the global G-P model, the global model and the global prototype, each client downloads them all from the server to the local, the local data is input into the feature extractor to obtain the local feature, the global prototype is input into the G-P model to obtain the training prototype, the training prototype and the local feature are supervised by the loss function to adjust the parameters of the G-P model, and the training prototype is also input into the frozen G-P model, and the result is averaged with the global prototype to obtain a personalized global prototype, and the personalized global prototype and the local feature are supervised by the loss function to adjust the parameters of the local model. Further, in the training process, the server randomly initializes the global G-P model, the global model and the global prototype, each client downloads them all from the server to the local, the local data is input into the feature extractor to obtain the local feature, the global prototype is input into the G-P model to obtain the training prototype, the training prototype and the local feature are supervised by the loss function to adjust the parameters of the G-P model, and the training prototype is also input into the frozen G-P model, and the result is averaged with the global prototype to obtain a personalized global prototype, and the personalized global prototype and the local feature are supervised by the loss function to adjust the parameters of the local model. Further, in the training process, the server randomly initializes the global G-P model, the global model and the global prototype, each client downloads them all from the server to the local, the local data is input into the feature extractor to obtain the local feature, the global prototype is input into the G-P model to obtain the training prototype, the training prototype and the local feature are supervised by the loss function to adjust the parameters of the G-P model, and the training prototype is also input into the frozen G-P model, and the result is averaged with the global prototype to obtain a personalized global prototype, and the personalized global prototype and the local feature are supervised by the loss function to adjust the parameters of the local model. Further, in the training process, the server randomly initializes the global G-P model, the global model and the global prototype, each client downloads them all from the server to the local, the local data is input into the feature extractor to obtain the local feature, the global prototype is input into the G-P model to obtain the training prototype, the training prototype and the local feature are supervised by the loss function to adjust the parameters of the G-P model, and the training prototype is also input into the frozen G-P model, and the result is averaged with the global prototype to obtain a personalized global prototype, and the personalized global prototype and the local feature are supervised by the loss function to adjust the parameters of the local model.

[0028] Further, in each communication iteration, the server and the client exchange the following information: local model parameters, G-P model parameters, and local embedding feature set.

[0029] In the local update phase, each client optimizes its G-P model and local model according to the comprehensive optimization target based on the local data.

[0030] The application also provides an electronic device, comprising:

[0031] one or more processors;

[0032] a memory for storing one or more programs;

[0033] When the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of the above.

[0034] A computer readable storage medium storing computer executable instructions, the instructions when executed to implement the method of any one of the above.

[0035] The application has the following beneficial effects:

[0036] Compared with other personalized federated learning methods, the personalized global prototype federated learning method based on adaptive feature alignment of the application can more effectively utilize global information, and through the design of personalized global prototype, the generalization ability of the local model is significantly enhanced, thereby improving the model performance as a whole. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The training process schematic diagram (left) and the communication process schematic diagram (right) of the method of the application;

[0038] Figure 2 The inference process schematic diagram of the method of the application;

[0039] Figure 3 The precision test results of different federated learning methods on Amazon Review and FMNIST data sets;

[0040] Figure 4 The maximum Euclidean distance between the same category data prototype and the local embedding feature, (a) global prototype in FedProto, (c) PGP in the method; and the minimum Euclidean distance between different category data prototypes and local embedding features, (b) global prototype in FedProto, (d) PGP in the method. DETAILED DESCRIPTION

[0041] The specific embodiments of the embodiments of the application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the embodiments of the application, and are not used to limit the embodiments of the application.

[0042] The application proposes a personalized global prototype federated learning method based on adaptive feature alignment, which is referred to as FedPGP in the application, and will be described in detail:

[0043] I. Prototype-based personalized federated learning

[0044] In the prototype-based personalized federated learning algorithm, it is assumed that there are N clients, and the i-th client has a private data set , is the local data, is the category label. The local model of the i-th client can be represented as where , and represent the parameters of the feature extractor and the classifier respectively.

[0045] After each local training iteration, the client will collect the local embedding feature and the local prototype of the j-th class is computed by averaging the embedding features belonging to the same class , as follows:

[0046]

[0047] where .

[0048] Subsequently, each client uploads its local prototype to the server. The server aggregates the global prototype by weightedly averaging the local prototypes of the j-th class from all clients , as follows:

[0049]

[0050] where denotes the set of clients containing the j-th class samples, , denotes the number of samples in the set.

[0051] Therefore, the global prototype is obtained by aggregating the local embedding features of all clients, which comprehensively reflects the collective information of all clients. However, due to the difference in the size of the local data set of each client, the global prototype tends to be biased towards the local prototype of the client with a larger data volume. In addition, the simple averaging process also weakens the expression ability of the global prototype to the diversity of different fields.

[0052] II. Adaptive feature alignment

[0053] To solve the limitations of the global prototype, the present application proposes an adaptive feature alignment mechanism (AFA) for converting the global prototype into a personalized global prototype that incorporates local domain information, as shown in Figure 1 .

[0054] The input of the AFA module is , where the global prototype is represented as , where c is the total number of classes and d is the vector dimension. For each data sample , its corresponding local embedding feature is extracted by the feature extractor .

[0055] When the AFA module receives the input , it first maps the global prototype P through a global-to-personal (G-P) mapping model to obtain a personalized prototype .

[0056] To avoid mismatch with the local feature space, thus destroying the feature mapping space of the local model and leading to performance degradation, it is necessary to ensure that the mapping model The generated feature space is aligned with the local domain feature space.

[0057] This alignment is achieved by:

[0058] ① Pulling with local embedding features with the same semantic label closer;

[0059] ② Pulling with local embedding features with different semantic labels away.

[0060] Similarly, for local embedding features , global information guidance is introduced: pull closer with the corresponding category, and pull away with different categories. This process promotes bidirectional adaptive alignment between global and local domains.

[0061] To achieve the above goal, the training objective function is defined as follows:

[0062]

[0063] where , and .

[0064] Although through the above loss function, and adaptive alignment can be achieved, due to the continuous backpropagation of each mini-batch during training, dynamic changes, especially in the early stages of training, change dramatically, causing instability and interfering with the optimization direction of the local model.

[0065] On the other hand, The main goal is to promote domain space alignment, rather than fine-grained adaptation to specific samples, so it needs to maintain the stability of its spatial mapping. In addition, as tends to the local domain, if not protected, there may be a problem of catastrophic forgetting (i.e., gradual loss of global information).

[0066] Therefore, the present application additionally equips each client with a frozen G-P model where is the global parameter downloaded from the server side, which remains fixed during local training. The frozen model ensures the consistency and stability of the mapping space.

[0067] By processing , a stable personalized global prototype is obtained, denoted as:

[0068]

[0069] Finally, in order to alleviate the problem of catastrophic forgetting, the present application performs mean fusion on the original global prototype P and the stable personalized global prototype PGP to obtain a personalized global prototype, which is used as the final guidance of the local model.

[0070] Personalized prototype The update target function is defined as:

[0071]

[0072] Wherein, represents the value not participating in parameter update.

[0073] For the update of the local model, the loss function is defined as:

[0074]

[0075] By minimizing the above formula and formula , bidirectional adaptive alignment of local feature space and global feature space can be realized, further enhancing the generalization ability of the local feature space.

[0076] In order to supervise the prediction of the local model, the following target function is used:

[0077]

[0078] Wherein, represents the softmax function.

[0079] In the process of using , and the above formula to optimize the model, we observe that the current supplement of global information mainly depends on the alignment with the personalized global prototype. Although this method can introduce global information to a certain extent, it is essentially implicit fusion, which has deficiencies in efficiency and intuitiveness.

[0080] In order to solve the above problems, the present application proposes a method of explicitly introducing global information into the client model, specifically using two global information carriers: global model and global prototype. This explicit integration method can significantly enhance the generalization ability of the local model.

[0081] Specifically, the present application introduces a frozen global classifier Since each client local classifier accumulates rich experience of processing prototypes after multiple rounds of training, the server generates a more general global classifier by aggregating all client local classifiers.

[0082] The global classifier takes the personalized global prototype as input and fuses with the output of the local model to form the final prediction result. Therefore, the supervised objective function predicted by the local model is redefined as:

[0083]

[0084] Finally, in the local update phase, the following comprehensive optimization objective is executed:

[0085]

[0086] Thus, the training, communication process and inference process of the method of the present application are shown in Figure 1 , Figure 2 respectively:

[0087] Training process. The server randomly initializes the global G-P model, the global image model and the global prototype, and each client downloads them all from the server to the local. During training, a batch of images is input into the image encoder to obtain image features, and the global prototype is input into the G-P model to obtain a training prototype. The training prototype and the image features are supervised to adjust the parameters of the G-P model. The training prototype is also input into the frozen G-P model, and the result is averaged with the global prototype to obtain a personalized global prototype. The personalized global prototype and the image features are supervised to adjust the parameters of the image model. Subsequently, the result obtained by averaging the global prototype and the training prototype through the frozen classifier is added to the result obtained by the training classifier to obtain the final result, which is supervised to adjust the parameters of the image model.

[0088] Inference process. An image is input into the image encoder to obtain image features. The global prototype is input into the trained G-P model to obtain a personalized global prototype, which is combined with the global prototype and input into the result obtained by the frozen head. The result is added to the result obtained by the training classifier to obtain the final result.

[0089] In each communication iteration, the following information is exchanged between the server and the client: image model parameters; G-P model parameters; and local embedding feature set.

[0090] In the local update phase, each client adjusts the local data according to the formula ​​​The defined objective function jointly optimizes the G-P model and the image model.

[0091] Through this explicit information integration and joint optimization strategy, the FedPGP framework can effectively balance between personalization and generalization, thus exhibiting robust and superior model performance in the field of diverse clients.

[0092] III. Experimental results

[0093] The effectiveness of FedPGP is evaluated in the present application. Specifically, the performance of FedPGP is compared with 13 state-of-the-art (SOTA) methods in the fields of computer vision (CV) and natural language processing (NLP). These methods include: traditional federated learning methods (such as FedAvg, FedProx), methods based on separation of global and local components (such as FedPer, FedRoD, FedRep), methods guided by global models (such as Ditto, pFedMe), methods guided by global prototypes (such as FedProto, FedPHP), and other methods (such as Per-FedAvg, FedFomo, FedAMP, FedALA).

[0094] Datasets: In computer vision tasks, five public datasets are used: MNIST, Fashion-MNIST (FMNIST), Cifar-10, Cifar-100, and Tiny-ImageNet. In natural language processing tasks, two public datasets are used: Amazon Review and AG News.

[0095] Model backbone network: On the Cifar-10, Cifar-100, MNIST, FMNIST, and Tiny-ImageNet datasets, a 4-layer convolutional neural network (4-layer CNN) is used as the backbone network. To verify the effectiveness of FedPGP on larger-scale models, ResNet-18 is also used on Tiny-ImageNet. For the AG News dataset, fastText is used; for the Amazon Review dataset, a 3-layer multilayer perceptron (3-layer MLP) is used. In local training, the learning rate is set to: for 3-layer MLP and 4-layer CNN, = 0.005, and for ResNet-18 and fastText, = 0.1. All G-P mapping models are implemented using a 2-layer MLP structure.

[0096] Data heterogeneity settings: To simulate data heterogeneity in federated learning environments, two settings are designed: Pathological setting: In MNIST / Cifar10 / Cifar100 datasets, 2 / 2 / 10 classes of data are sampled for each client from 10 / 10 / 100 classes respectively, and there is no data overlap between different clients, and the sample size is unbalanced. Realistic setting: Dirichlet distribution (denoted as Dir( ) is used for data sampling. Specifically, for each class c, its samples are allocated to client i according to the proportion . By default, .

[0097] Implementation details: According to the settings of pFedMe, all experiments assume 20 clients (i.e., the total number of clients is 20) unless otherwise specified, and the client participation rate . For each client, the local dataset is divided into 75% for training and 25% for testing. The batch size for local training is set to 10, and the number of local iterations per round is set to 1. FedPGP is implemented using the PyTorch-1.13 framework, and all experiments are conducted on a server with the following configurations: 2 Intel(R) Xeon(R) Silver 4210 CPUs, 256GB of memory, 4 NVIDIA 3090 GPUs, and Ubuntu 20.04 operating system.

[0098] Under the pathological setting and the realistic setting, the effectiveness of FedPGP relative to other federated learning methods is evaluated using 20 clients, .

[0099] As shown in the results in the following table, FedPGP outperforms all other methods on six datasets and two settings.

[0100]

[0101] Further experimental results on the Amazon Review and FMNIST datasets (see Figure 3 ) also further verify the trend that FedPGP consistently outperforms other methods in various tasks, with FedProto having the slowest convergence speed and the lowest accuracy.

[0102] The above experimental results show that FedPGP can effectively utilize global information, and by designing personalized global prototypes, it significantly enhances the generalization ability of local models, thereby improving the overall model performance.

[0103] The present application further studies the adaptability of global prototype and personalized global prototype (PGP) to local embedding feature space. Specifically, the maximum Euclidean distance (MAX) between the prototype and the features of the same class in all clients, and the minimum Euclidean distance (MIN) between the prototype and the features of different classes are measured respectively.

[0104] For the convenience of visualization, the Euclidean distances of each method are normalized, and the results are shown in FIG. 2, where the horizontal axis (x-axis) represents the class index, and the vertical axis (y-axis) represents the normalized Euclidean distance. Through analysis, it is found that the MAX value of the personalized global prototype (PGP) is significantly smaller than that of the global prototype, indicating that the PGP can better adapt to the feature space of the same class. At the same time, the MIN value of the PGP is much larger than that of the global prototype, indicating that the PGP has stronger discriminant ability in distinguishing features of different classes. Figure 4

[0105] These results verify that the PGP has good adaptability in the local feature space.

[0106] Those skilled in the art will understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0107] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in the flowcharts and / or block diagrams Figure 1 The functions specified in the flowcharts and / or block diagrams

[0108] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction devices that implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in the flowcharts and / or block diagrams Figure 1 ​the function specified in the one or more blocks.

[0109] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide processes for implementing the flows Figure 1 the flow or flows and / or blocks Figure 1 the steps of the function specified in the one or more blocks.

[0110] The above-described embodiments are merely some of the preferred schemes of the present application, and are not intended to limit the present application. Those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Therefore, any technical solutions obtained by equivalent replacement or equivalent transformation shall fall within the scope of the present application.

Claims

1. A personalized global prototype federated learning method based on adaptive feature alignment, characterized by: include: For a system with a server and N clients, the client includes a feature extractor and a classifier; In computer vision tasks, an adaptive feature alignment mechanism is used to achieve bidirectional adaptive alignment between local feature space and global feature space, including: the client converts the original global prototype P aggregated by the server into a personalized prototype that incorporates local image features through the global to personalized mapping GP model , and then process the personalized prototype through the frozen GP model The stable personalized prototype PGP is obtained. The global parameters of the frozen GP model downloaded from the server are kept fixed and not updated during the local training process. Then the original global prototype P and the stable personalized prototype PGP are averaged and fused to obtain the personalized global prototype as the final guide for the local model to extract image features. The bidirectional adaptive alignment is achieved by: Close to local image features with the same semantic label; personalized prototype Distancing from local image features of different semantic labels; Similarly, for local image features , introduce global information guidance: close to the personalized prototype of the corresponding category c , zoom out and different categories Personalized prototype , ; In addition, a frozen global classifier is introduced to explicitly introduce global information into the client. The frozen global classifier is generated by the server by aggregating the local classifiers of all clients. The personalized global prototype is input into the frozen global classifier, and then it is fused with the output of the client's local model to obtain the final prediction result output.

2. The personalized global prototype federated learning method based on adaptive feature alignment according to claim 1 is characterized in that: The comprehensive optimization goal of the client local update phase is the update objective function of the personalized global prototype , loss function for local model feature extraction training And the supervised objective function predicted by the local model All three are minimized.

3. The personalized global prototype federated learning method based on adaptive feature alignment according to claim 2 is characterized in that: The updated objective function of the personalized global prototype is is a loss function based on the negative log-likelihood form, which is used to measure local image features that do not participate in parameter update The degree of match between the personalized prototype of the corresponding category is expressed as the negative logarithm of a ratio plus 1, where the numerator of the ratio is the local image feature that does not participate in parameter update A personalized prototype of its similar labels The similarity score between them is the value after exponential decay, and the denominator is the value of all Personalized prototypes corresponding to different class labels Local image features that do not participate in parameter updates The sum of the similarity scores is also processed by exponential decay.

4. The personalized global prototype federated learning method based on adaptive feature alignment according to claim 2, characterized in that: Loss function for local model feature extraction training It is an objective function constructed in the form of negative log-likelihood, which is used to measure the similarity between the local image features and the personalized global prototype after fusion with their corresponding categories. It is expressed as the negative logarithm of a ratio value plus 1, where the numerator of the ratio value is the local image feature. The similarity score between the personalized global prototype after fusion with its similar labels is calculated by exponential decay, and the denominator is the sum of all local image features. The fused personalized global prototype and local image features corresponding to different class labels The sum of the similarity scores of , and the value is also processed by exponential decay.

5. The personalized global prototype federated learning method based on adaptive feature alignment according to claim 2, characterized in that: Supervised objective function for local model prediction To calculate the one-hot encoding vector of the true label The cross entropy between the predicted output and the model is used to measure the degree of match between the two. The predicted output of the model is obtained by the sum of two parts: the first part is the classifier result during training. The second part is the result of freezing the classifier to process the fusion personalized features. , the outputs of these classifiers are passed through the softmax function Normalization is performed to generate the final probability distribution.

6. The personalized global prototype federated learning method based on adaptive feature alignment according to claim 2, characterized in that: During the training process, the server randomly initializes the global GP model, global image model and global prototype. Each client downloads them all from the server to the local computer. The local image data is input into the feature extractor to obtain the local image features. The global prototype is input into the GP model to obtain the training prototype. The training prototype and the local image features are Supervision, adjust the parameters of the GP model, the training prototype is also input into the frozen GP model, the result is averaged with the global prototype to obtain the personalized global prototype, the personalized global prototype and the local image features are averaged. Supervise and adjust the local model parameters; then the results of the personalized global prototype obtained by the frozen classifier and the results of the local image features obtained by the trained classifier are added to obtain the final result, which is obtained by Supervised tuning of local model parameters.

7. The personalized global prototype federated learning method based on adaptive feature alignment according to claim 2, characterized in that: In each communication iteration, the server and client exchange the following information: local model parameters, GP model parameters, and local image feature set; In the local update phase, each client jointly optimizes its own GP model and local model based on local image data and comprehensive optimization objectives.

8. A personalized global prototype federated learning system based on adaptive feature alignment, characterized by: The system is implemented by the method according to any one of claims 1 to 7.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions, wherein the instructions are used to implement the method according to any one of claims 1 to 7 when executed.

Images