Federal prototype learning method based on double-branch feature projector and personalized global fusion

Abstract

The invention discloses a federal prototype learning method based on a double-branch feature projector and personalized global fusion, and the method comprises the steps: introducing the double-branch feature projector between a feature extractor and a classifier of a local model of a client, so as to give consideration to and balance the fidelity and discrimination in the same feature extraction process, the double-branch feature projectors comprise a shared branch feature projector and a decision branch feature projector; the client freezes local model parameters, calculates the features of the shared branch feature projector, and calculates the features of the decision branch feature projector; the client side calculates importance scores of the channels; the client uploads the local prototype and the importance score to a server; the server generates a personalized global prototype corresponding to each client; and distributing each personalized global prototype to a corresponding client. According to the method, fidelity and discrimination are both considered, and meanwhile, the global prototype retains key information of the client, so that the federal prototype learning performance is improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, specifically relating to a federated prototype learning method based on a dual-branch feature projector and personalized global fusion. Background Technology

[0002] The development of modern artificial intelligence technology exhibits two significant trends: first, models are becoming increasingly reliant on data; and second, the sources of data generation are migrating to the network edge at an unprecedented rate. Edge computing units such as smartphones, IoT devices, and smart cars have become the primary carriers of data creation. This native data generated locally by users, due to its high fidelity, immediacy, and high degree of personalization, holds enormous potential to drive breakthroughs in next-generation AI applications. How to construct a completely new technological framework to efficiently and securely utilize these massive, heterogeneous, and physically dispersed data resources has become a core challenge for technological innovation in this era.

[0003] Against this backdrop, a series of emerging technological prerequisites and the evolution of the regulatory environment paved the way for the birth of federated learning. First, the awakening of global awareness of data privacy protection has become a major driving force for building new data collaboration paradigms. The regulatory system, represented by the EU's General Data Protection Regulation (GDPR), has established core principles such as "data minimization" and "privacy by design" globally. This requires that technical architectures must consider data privacy protection as an inherent, non-additional function from the outset, prompting the industry to explore technological paths that can unlock data value without moving or exposing raw data. Secondly, the leap in computing power of edge devices has provided a solid foundation for the localized execution of computing tasks. Modern edge devices are no longer simply data acquisition terminals; their built-in processors are now capable of executing complex machine learning model training tasks. This rise of "edge intelligence" makes it possible to migrate model training tasks from cloud data centers to the source of data generation, thus giving rise to a new computing paradigm that distributes models to data rather than aggregating data into models.

[0004] Furthermore, the market's urgent demand for highly personalized and instant-response services constitutes a direct driving force for applications. Whether it's word prediction and personalized content recommendation in intelligent input methods, or scene awareness in intelligent voice assistants, modern AI applications increasingly pursue "living" models that can adapt to individual user behavior and environmental changes in real time. This demand requires models to have the ability to continuously learn and iterate on the device to capture the latest user preferences, while traditional model update cycles can no longer meet such real-time requirements.

[0005] Federated learning is a new paradigm of distributed machine learning that emerged from the convergence of the aforementioned multiple driving forces. It fundamentally reshapes the way data is collaborated, and its core concept can be summarized as "data doesn't move, model moves." This framework organizes and manages the training process of a global model through a single server. The server first distributes the initial model to each participating client. Each client trains the model using its unique local data, and the original data remains locally throughout the process without any migration. After training, the client uploads only the updated model parameters, not the original data, to the server after encryption and differential privacy techniques. The server securely aggregates the model parameters from multiple clients to optimize and iterate the global model. Subsequently, the updated global model is redistributed to the clients to begin the next round of local training. This process repeats until the global model reaches the preset performance target. In this way, federated learning cleverly constructs a closed-loop collaborative ecosystem that leverages the advantages of massive distributed data while meeting stringent privacy protection requirements, opening up a new technological path for the application and development of artificial intelligence in sensitive data domains.

[0006] However, traditional federated learning frameworks are typically built on idealized assumptions, namely that all participating clients possess the same model architecture and independently distributed data. In real-world applications, these idealized assumptions often fail, leading to the core challenge of heterogeneity. Heterogeneity is mainly divided into two dimensions: data heterogeneity and model heterogeneity. These are key factors that cause significant performance degradation or even complete failure of federated learning models. To address these real-world complexities, heterogeneous federated learning has emerged and has been widely researched and applied.

[0007] Data heterogeneity is the most prevalent challenge in federated learning, referring to the non-independent and identically distributed nature of local datasets across clients. Specifically, for two different clients, the probability distributions of their local data differ. This difference stems from the inherent properties of data generation in the real world. For example, in mobile applications, different users' input method habits, photo content, and app usage behaviors are highly personalized; in the medical field, patient data from different hospitals exhibit significantly different feature distributions due to differences in region, specialty, and equipment. When traditional federated learning algorithms (such as FedAvg) attempt to simply average these model updates derived from heterogeneous data, it leads to conflicts and fluctuations in the optimization direction of global model parameters, making it difficult for the model to converge to an optimal solution effective for all clients, ultimately severely impairing the model's generalization ability and personalized performance.

[0008] Model heterogeneity refers to the different machine learning models used by various clients participating in federated learning. These models differ not only in the number of parameters but also, and more importantly, in their overall structure, such as the number of network layers, neuron types, or underlying architectures (e.g., CNNs and Transformers). This model heterogeneity also has practical causes. On one hand, the hardware capabilities of different clients vary greatly; a flagship smartphone can run a complex, large neural network, while a resource-constrained IoT sensor may only be able to support a lightweight, simple model. On the other hand, participating organizations may use their own customized proprietary models due to technology selection, historical reasons, or considerations for protecting intellectual property. This lack of uniformity in model structure fundamentally renders traditional federated learning algorithms, which rely on directly aggregating model parameters, ineffective because the parameter spaces of different model structures cannot be directly aligned or averaged.

[0009] Federated prototype learning is a method for implementing heterogeneous federated learning, enabling knowledge sharing between different clients in settings with heterogeneous data and models. At the client level, L2 alignment-based strategies (such as FedProto and FedTGP) align local features with the global prototype. While this ensures fidelity by bringing features closer to their semantic centers, it neglects the relative distances between features of different categories, leading to insufficient discriminative power. Contrastive learning-based strategies (such as FedProc and FedLFP) enhance discriminative power by bringing positive sample pairs closer and pushing negative sample pairs further apart, focusing on widening the gap between features of different categories. However, this approach overemphasizes the relative positions of features, potentially causing features to deviate from their true semantic centers, ultimately resulting in low-quality, low-fidelity global prototypes (local models). Even attempts to combine these two conflicting goals, simultaneously imposing these conflicting constraints on the same feature extractor, can cause confusion and negatively impact training performance.

[0010] At the server level, the core deficiency of existing federated prototype learning methods lies in their limitation to a single global prototype. This problem stems from data heterogeneity, where data of the same category can exhibit different feature distributions on different clients due to data heterogeneity. Existing servers use simple aggregation methods (such as averaging or clustering) or learnable networks to mix these inherently different local prototypes with semantic information from different clients. The direct consequence is the loss of client-specific local information. Ultimately, this aggregation method produces a semantically ambiguous global prototype. When this prototype is distributed back to clients to guide local training, it can mislead client training and thus degrade model performance. Summary of the Invention

[0011] To address the limitations of existing federated prototype learning methods, which cannot simultaneously achieve both feature fidelity and discriminativeness and are constrained by a single global prototype, this invention provides a federated prototype learning method based on a dual-branch feature projector and personalized global fusion. By generating two different feature attributes through the dual-branch feature projector, it balances fidelity and discriminativeness, while the global prototype retains key client information, thereby improving the performance of federated prototype learning.

[0012] To achieve the objectives of this invention, this invention provides a federated prototype learning method based on a dual-branch feature projector and personalized global fusion, characterized by the following steps: The client uses local data and a global prototype to train and update the parameters of the local model. A dual-branch feature projector is introduced between the feature extractor and classifier of the client's local model to balance fidelity and discriminability in the same feature extraction process. The dual-branch feature projector includes a shared branch feature projector and a decision branch feature projector. The client freezes local model parameters, calculates the features of the shared branch feature projector, and collects local prototypes. If the client needs to perform inference and prediction of unknown samples, it calculates the features of the decision branch feature projector and makes a judgment based on the nearest neighbor principle of the global prototype. The client calculates the channel importance score based on the unnormalized prediction score of the local model and the shared branch features; The client uploads the local prototype and importance score to the server; The server performs average aggregation of client prototypes and fusion of personalized global prototypes to generate a personalized global prototype for each client. The server distributes each personalized global prototype to the corresponding client.

[0013] The present invention also provides a computer device.

[0014] The present invention also provides a computer-readable storage medium.

[0015] This invention employs a dual-branch feature projector and personalized global fusion, enabling the seemingly contradictory goals of "fidelity" and "discriminability" to be considered and balanced during the same feature extraction process. At the same time, it allows the server to generate a more targeted global prototype for each client, preserving global common knowledge while taking into account the client's personalized needs, thus forming a novel federated prototype learning scheme.

[0016] Compared with the prior art, the present invention can achieve at least the following beneficial effects: 1. This invention designs a dual-branch feature projector on the client side, which cleverly combines the two strategies of L2 alignment and contrastive learning, but applies them to different branches, ensuring that the local features generated by the client side are both faithful to the global semantic center and have clear category discrimination ability.

[0017] 2. This invention designs a contrastive learning method with hard negative sample mining, adaptively calculates the negative sample boundary, introduces a boundary penalty term, increases the weight of hard negative samples, and makes the training process more focused on those samples that are easily confused. 3. This invention introduces a personalized global prototype fusion method based on Fisher information on the server side. It uses Fisher information to quantify and identify the most important feature channels for each client. Through a Top-K selection strategy, it allows the client to retain its own local prototype information on these important channels. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the framework of a federated prototype learning method based on a dual-branch feature projector and personalized global fusion provided by an embodiment of the present invention.

[0019] Figure 2 This is a flowchart illustrating a federated prototype learning method based on a dual-branch feature projector and personalized global fusion, provided by an embodiment of the present invention.

[0020] Figure 3 This is a flowchart of the steps provided in the embodiments of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Federal Learning 联邦学习 Federated learning is a distributed machine learning paradigm that allows multiple participants to collaboratively train a shared global model without exchanging their private data. Its core mechanism is "data doesn't move, model moves": the initial model is distributed to each party from a central server, trained on their local data, and only the updated model parameters are sent back to the server for secure aggregation, thus iteratively optimizing the global model. This approach aims to fundamentally solve the data silo problem while meeting increasingly stringent data privacy and security compliance requirements.

[0023] Heterogeneous Federated Learning(异构联邦学习) Heterogeneous federated learning is an important branch of federated learning, aiming to address the significant differences between clients in the real world. It primarily studies two categories of heterogeneity: statistical heterogeneity arising from the fact that client data does not satisfy the independent and identically distributed property; and system heterogeneity arising from differences in device computing power, network conditions, and online status. These differences can severely hinder the convergence and generalization performance of the global model; therefore, this field is dedicated to developing more robust algorithms to ensure training efficiency and model performance in non-ideal environments.

[0024] Independent and identically distributed (独立同分布) The independent and identically distributed (IOD) assumption is a fundamental hypothesis in probability statistics and machine learning. It has two meanings: "identical distribution" means that all random variables in a sequence originate from the same probability distribution and have the same statistical properties; "independence" means that the observation of any variable in the sequence will not affect the observation of other variables. This assumption is the cornerstone of many statistical inference theories, such as the central limit theorem, and is the theoretical premise for standard machine learning models to effectively generalize from the training set to the unknown test set, because it guarantees the unbiasedness of data sampling.

[0025] Euclidean distance ( 欧几里得距离 Euclidean distance, also known as the Euclidean metric, is a standard measure of the true "straight-line" distance between two points in n-dimensional Euclidean space. It is rigorously defined as the square root of the sum of the squares of the differences between the corresponding coordinates of two points in each dimension. This metric is a direct generalization of the Pythagorean theorem to higher-dimensional spaces and is equivalent to the L2 norm of the vector difference between two points. As the most intuitive and widely used distance indicator, it is used in cluster analysis, the k-nearest neighbor algorithm, and many machine learning fields to quantify the similarity between samples.

[0026] Prototype learning (原型学习) Prototype learning is an instance-based machine learning paradigm that, at its core, learns one or more representative data points or vectors called "prototypes" for each category. This method adjusts the positions of prototypes in the feature space through optimization algorithms to generalize the distribution characteristics of the corresponding category. During the prediction phase, the category assignment of a new data point is determined by calculating its distance from all prototypes, and it is ultimately assigned to the category of the nearest prototype.

[0027] Comparative learning (对比学习)Contrastive learning is a machine learning paradigm aimed at learning effective representations of data, particularly renowned for its application in self-supervised learning. Its core mechanism involves maximizing the similarity between semantically similar samples (positive pairs) and minimizing the similarity between semantically dissimilar samples (negative pairs) within a learnable embedding space using a contrastive loss function (such as InfoNCE). In a self-supervised setting, positive pairs typically consist of different augmented views of the same source data, forcing the model to learn transferable, general features that are invariant to transformations.

[0028] Fisher Information Matrix (费舍尔信息矩阵) The Fisher information matrix is ​​a core tool in parameter estimation theory, used to precisely quantify the information contained in data about unknown parameters. Mathematically, it is obtained by calculating the expectation of the second derivative of the log-likelihood function with respect to the parameters, essentially measuring the curvature of the likelihood function near the true parameter points. Its fundamental importance lies in the fact that its inverse matrix defines the Cramer-Rhodes lower bound, that is, the theoretical minimum achievable variance of any unbiased estimator, thus linking data information to the highest accuracy of parameter estimation.

[0029] Federated prototype learning, as a method of heterogeneous federated learning, enables knowledge sharing among different clients in settings with heterogeneous data and models. However, existing federated prototype learning methods face two core technical challenges: the first is how to simultaneously ensure the fidelity and discriminativeness of local features at the client level; the second is how to overcome the limitations of a single global prototype at the server level to adapt to the heterogeneity of data from different clients. For the first challenge, fidelity refers to the requirement that features extracted by local clients faithfully adhere to the semantic center of the global prototype, ensuring consistent alignment of knowledge across clients. Discriminativeness requires features to clearly distinguish different categories, enhancing the model's classification ability. For the second challenge, in scenarios where data is not independent and identically distributed, even data of the same category from different clients may exhibit significant differences in feature distribution. Existing federated prototype learning methods aggregate the local prototypes uploaded by all clients to form a single global prototype. This "one-size-fits-all" approach mixes semantic information from different clients, resulting in a blurred global prototype that loses client-specific local information.

[0030] To address at least one of the problems existing in the prior art, this invention provides a federated prototype learning method based on a dual-branch feature projector and personalized global fusion. A novel federated prototype learning framework is proposed, which improves local feature quality and achieves personalization of the global prototype through effective client-side training and adaptive server aggregation. Specifically, this invention designs an innovative dual-branch feature projector on the client side, integrating L2 alignment and contrastive learning on different branches, thus cleverly solving the problem of balancing feature fidelity and discriminability that is difficult to achieve in existing methods. Furthermore, this invention introduces a channel importance analysis method based on Fisher information. This method can adaptively assign weights to feature channels, thereby achieving personalized global prototype fusion and effectively overcoming the limitations of traditional single global prototypes. The novel federated prototype learning framework is formed by fusing a dual-branch feature projector on the client side and a personalized global prototype based on Fisher information, as shown in Figure 1.

[0031] Please see Figures 1 to 3 The present invention provides a federated prototype learning method based on a dual-branch feature projector and personalized global fusion, comprising the following steps: Step S1: The client uses local data and the global prototype to train and update the parameters of the local model, where the shared branch feature projector and the decision branch feature projector are calculated using the corresponding loss functions.

[0032] Step S2: The client freezes the local model parameters, calculates the features of the shared branch feature projector, and collects local prototypes. If the client needs to perform inference and prediction for unknown samples, it calculates the features of the decision branch feature projector and makes a judgment based on the nearest neighbor principle of the global prototype. (The client has a set of global prototypes. For each sample that needs to be predicted, it calculates the features of the decision branch feature projector and the Euclidean distance between them and the global prototype of each category. The category of the sample is the category of the global prototype with the smallest Euclidean distance.) Step S3: The client calculates the importance score of the channel based on the unnormalized prediction score of the local model and the shared branch features.

[0033] Step S4: The client uploads the local prototype and importance score to the server.

[0034] Step S5: The server performs average aggregation of client prototypes and fusion of personalized global prototypes.

[0035] Step S6: The server distributes the personalized global prototype to the corresponding clients.

[0036] The term "client prototype" refers to all local prototypes of all clients, while "local prototype" refers to the prototypes of all categories of a particular client.

[0037] For each heterogeneous local model, the last linear layer of the local model is called the "classification head," which performs the transformation from the feature space to the classification space. The rest of the local model is collectively referred to as the "feature extractor." Specifically, the client's local data first passes through the feature extractor of the local model to obtain extracted low-dimensional features. Then, the low-dimensional features are projected through a two-branch feature projector and finally input into the classification head.

[0038] To balance feature fidelity and distinguishability, this embodiment of the invention designs a dual-branch feature projector on the client side. A dual-branch feature projector is introduced between the feature extractor and the classification head of the local model. The dual-branch feature projector includes a shared branch feature projector and a decision branch feature projector. The two branch feature projectors each use their own cross-entropy loss. and Optimization is then performed. Both the shared branch feature projector and the decision branch feature projector are two-layer perceptrons that do not change the dimensionality of the features, but simply project the features onto different regions. Different loss functions are used for optimization, ensuring that the shared branch features have accuracy and the decision branch features have discriminative power.

[0039] This invention aims to construct a local prototype of a client using shared branch features from a shared branch feature projector. Specifically, the local prototype is a representation of a certain category of the client, calculated by averaging all shared branch features belonging to that category. Since these shared branch features effectively represent the global prototype, this invention designs the following loss function, which aligns the shared branch features and the global prototype in the semantic space by minimizing the Euclidean distance between them, as expressed below:

[0040] in, Indicates the training batch size. Indicate category The corresponding global prototype, Indicates the first Shared branch features of each sample The loss function for the shared branch represents the average Euclidean distance between the shared branch features and the global prototype within each training batch, which is minimized. The items are aligned to achieve the desired alignment.

[0041] Furthermore, embodiments of the present invention utilize decision branch features generated by a decision branch feature projector to perform inference prediction based on a global prototype. Decision branches should generate features with clear inter-class discriminability. This invention applies contrastive learning to address the problem of insufficient inter-class feature discriminability caused by L2 alignment. Therefore, this invention designs the following loss function for the decision branch feature projector, which enhances the discriminative power of features, expressed as:

[0042] in,

[0043] Represents the normalized decision branch characteristics With normalized global prototype The Euclidean distance between them It's a temperature parameter. Indicates the total number of categories. It is the loss function for the decision branch, representing the average InfoNCE loss within each training batch.

[0044] It is worth noting that, in Assigning equal weights to simple and difficult negative samples is inefficient. Simple negative samples contribute almost no gradient, while difficult negative samples near the class boundary are more important for distinguishing features. To address this issue, this invention introduces an adaptive hard negative sample mining strategy that dynamically selects challenging negative samples to improve effectiveness. Specifically, a boundary penalty term is incorporated into the denominator of the loss function. and will Redefining as follows:

[0045] in,

[0046] Where the adaptive boundary Defined as:

[0047] in, and and represent the average Euclidean distance between positive and negative samples in the batch, respectively. The adaptive boundary is calculated based on the statistical data of the current batch. This ensures that the boundary can adapt to different data distributions and training processes.

[0048] Therefore, the overall objective function for local training of the framework proposed in this invention is defined as:

[0049] in , and It is a hyperparameter that balances the contributions of each loss term.

[0050] Because heterogeneity renders a single global prototype ineffective, this invention introduces a method based on Fisher information. This method generates a global prototype tailored to a specific client—a personalized global prototype—by estimating the channel importance in shared branch features. Fisher information quantifies the sensitivity of the output distribution to small changes in feature channels. A larger Fisher information value indicates that the channel has a significant impact on the client's predictions and should therefore be retained. In one embodiment, the importance score is approximated as follows:

[0051] in Indicates belonging to a category The number of samples, The parameters representing the local model, Indicates the local model in terms of parameters Below, for input Predict its label as The probability, i.e., the log-likelihood of the input sample. It is a shared branch feature The One channel, This represents the channel importance score in the shared branch features.

[0052] During the computation of the local prototype, the client performs forward propagation using its local dataset, sequentially passing through a feature extractor, a shared branch feature projector, and a classifier to obtain unnormalized prediction scores (logits). Then, the log probabilities of the true labels are calculated. Through backpropagation, gradients with respect to the shared branch features are obtained. Based on these gradients, the class can be estimated. Fisher information for each channel. Finally, Fisher information at the category level is obtained by averaging all samples in the same category, and this is used as the channel importance score.

[0053] In this embodiment of the invention, a client-level average is used to calculate the global prototype for each category, that is, to perform average aggregation of client prototypes.

[0054] After the fusion of personalized global prototypes, each client receives a set of personalized global prototypes. The fusion of personalized global prototypes utilizes each client's channel importance score and employs a Top-K selection strategy. Specifically, in one embodiment, the K most important channels containing more local information are retained, while the global prototype is used on other channels, thus achieving channel-level personalized fusion. The personalized global prototype replaces the average prototype and is distributed to participating clients.

[0055]

[0056]

[0057] Indicate category The most important set of channels, including the front Index of each channel, Indicate category Next The importance score of each shared branch feature channel measures the importance of that channel to the category. The sensitivity of the prediction result is calculated as shown in Equation (8). This represents the total number of channels that share branch characteristics. - Indicates all Select the top K channels with the highest importance scores from the available channels. Indicates client In category ,aisle The local prototype value on. Indicate category ,aisle The global prototype value. This represents the personalized fusion coefficient, controlling the balance between local and global information. When... The prototype was used entirely locally at that time. The global prototype is used entirely at that time. Indicates client In category ,aisle Personalized prototype values ​​on, when When the prototype is obtained, it is obtained through a weighted fusion of the local and global prototypes; otherwise, the global prototype is used directly.

[0058] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the methods described in the foregoing embodiments.

[0059] In one embodiment, a computer-readable storage medium stores a computer program, characterized in that the computer program, when executed by a processor, implements the method described in the foregoing embodiments.

[0060] In the context of heterogeneous federated learning, the heterogeneity of data and models poses a significant challenge to traditional federated learning. While federated prototype learning provides an effective approach to address this problem, especially with heterogeneous models, existing methods have serious shortcomings: the client struggles to balance feature fidelity and discriminativity, while the single global prototype generated by the server becomes ambiguous due to the mixing of semantics from different clients, and may even mislead local training. This invention proposes a novel federated prototype learning framework. Its client-side dual-branch design ensures both feature fidelity and discriminativity; the server-side performs personalized prototype fusion based on Fisher information, preserving key local information from the client. This effectively addresses the deficiencies of existing federated prototype learning methods.

[0061] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in this invention may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A federated prototype learning method based on a dual-branch feature projector and personalized global fusion, characterized in that, Includes the following steps: The client uses local data and a global prototype to train and update the parameters of the local model. A dual-branch feature projector is introduced between the feature extractor and classifier of the client's local model to balance fidelity and discriminability in the same feature extraction process. The dual-branch feature projector includes a shared branch feature projector and a decision branch feature projector. The client freezes local model parameters, calculates the features of the shared branch feature projector, and collects local prototypes. If the client needs to perform inference and prediction of unknown samples, it calculates the features of the decision branch feature projector and makes a judgment based on the nearest neighbor principle of the global prototype. The client calculates the channel importance score based on the unnormalized prediction score of the local model and the shared branch features; The client uploads the local prototype and importance score to the server; The server performs average aggregation of client prototypes and fusion of personalized global prototypes to generate a personalized global prototype for each client. The server distributes each personalized global prototype to the corresponding client.

2. The federated prototype learning method based on a dual-branch feature projector and personalized global fusion as described in claim 1, characterized in that, The shared branch features of the shared branch feature projector are used to build a local prototype for the client, and the decision branch features generated by the decision branch feature projector are used to perform inference prediction based on the global prototype.

3. The federated prototype learning method based on a dual-branch feature projector and personalized global fusion according to claim 1, characterized in that, The overall objective function of local training for: In the formula, For the cross-entropy loss of the shared branch feature projector, The cross-entropy loss is for the feature projector in the decision branch. The loss function for the shared branch feature projector. Let the loss function be the feature projector for the decision branch. , and It is a hyperparameter that balances the contributions of each loss term.

4. The federated prototype learning method based on a dual-branch feature projector and personalized global fusion according to claim 1, characterized in that, The loss functions of the shared branch feature projector and the decision branch feature projector are obtained by using L2 alignment and contrastive learning to balance fidelity and discriminability.

5. The federated prototype learning method based on a dual-branch feature projector and personalized global fusion according to claim 1, characterized in that, Loss function of decision branch feature projector Introducing boundary penalty terms To dynamically select challenging negative samples, The expression is: In the formula, For adaptive boundary, It's a temperature parameter. This represents the Euclidean distance between the normalized decision branch features and the normalized global prototype. For category index number, Indicates a label.

6. The federated prototype learning method based on a dual-branch feature projector and personalized global fusion according to claim 1, characterized in that, During the computation of the local prototype, the client uses its local data for forward propagation, sequentially passing through a feature extractor, a shared branch feature projector, and a classifier to obtain unnormalized predicted scores; it then calculates the log probability of the true label; through backpropagation, it obtains the gradient with respect to the shared branch features; and based on the gradient, it obtains the category. Fisher information for each channel; Fisher information at the category level is obtained by averaging all samples in the same category and used as the importance score for each channel.

7. The federated prototype learning method based on a dual-branch feature projector and personalized global fusion according to claim 1, characterized in that, For the fusion of personalized global prototypes, the channel importance score of each client is used, and a Top-K selection strategy is adopted to achieve personalized fusion at the channel level. The Top-K selection strategy is to retain the K most important channels that contain more local information, while using the global prototype on other channels.

8. A federated prototype learning method based on a dual-branch feature projector and personalized global fusion according to any one of claims 1-7, characterized in that, The average aggregation of the client prototypes is calculated by using client-level averaging to calculate the global prototype for each category.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-8.

Images