Heterogeneous federated learning method and system based on frozen backbone and double loss cooperation

Abstract

The present application relates to a kind of heterogeneous federal learning method and system based on frozen backbone and double loss cooperation.The method comprises: central server initializes global semantic reference point, global projection head parameter, global classifier parameter and is issued to each client participating in federal learning;Each client freezes local heterogeneous backbone network parameter, extracts the heterogeneous feature of local data, and maps heterogeneous feature to latent semantic space by architecture-aware projection head;Each client calculates the composite loss function including alignment-contrast loss, and updates local projection head parameter and classifier parameter;Each client uploads the updated parameter to central server, and does not upload the parameter of completely frozen heterogeneous backbone network;Central server updates momentum to global semantic reference point, obtains updated global semantic reference point.Can realize the efficient cooperation and semantic alignment between heterogeneous devices, improve communication efficiency and global model precision.

Description

Technical Field

[0001] This invention relates to the field of heterogeneous federated learning technology, and in particular to a heterogeneous federated learning method and system based on frozen backbone and dual-loss synergy. Background Technology

[0002] With the rapid popularization of the Internet of Things (IoT), smart terminals, and edge computing, distributed artificial intelligence (AI) has increasingly broad application prospects in smart cities, autonomous driving, industrial internet, telemedicine, and other fields. In these application scenarios, massive amounts of business data are generated at the edge devices, and much of it contains user privacy and sensitive information. Traditional centralized machine learning faces the dual contradiction of data silos and privacy leaks. Federated learning (FL), as a privacy-preserving distributed machine learning paradigm, allows participating parties to collaboratively train a global model by exchanging model parameters or gradient information without sharing the original data, becoming a core technology for solving the problems of data silos and privacy protection. In actual large-scale deployments, federated learning faces severe challenges: edge devices are usually connected via wireless networks, with extremely limited and unstable communication bandwidth, while modern deep neural network models have a huge number of parameters, and frequent model transmissions lead to unbearable communication overhead; moreover, due to differences in device hardware configurations, different participating parties often need to deploy neural network models with different architectures. This "system heterogeneity" makes traditional aggregation methods based on averaging model parameters unsuitable for direct application.

[0003] To address this, existing technical solutions mainly focus on three aspects: communication optimization, heterogeneity handling, and personalized federated learning. The most classic federated averaging algorithm (FedAvg) updates the global model by iteratively aggregating the mean of model parameters uploaded by clients, but it strictly requires that the model architecture of all clients be completely identical. To address the heterogeneity problem of model architecture, some research has turned to federated learning methods based on knowledge distillation. These methods (such as FedDistill and FedGen) do not directly aggregate model parameters, but rather transfer knowledge by exchanging model outputs (Logits) on public datasets or by using synthetic data generated by generative adversarial networks (GANs). However, existing technical solutions still have the following significant technical shortcomings when facing resource-constrained and highly heterogeneous edge computing environments:

[0004] First, communication overhead remains enormous, making it difficult to adapt to weak network environments. Whether it's FedAvg, FedProx, or SCAFFOLD, their core mechanisms rely on transmitting complete model parameters or gradients / control variables of equal dimensionality. In bandwidth-constrained or traffic-billed edge network environments, this high communication cost not only leads to extremely high training latency but can also cause training failures due to network congestion. Second, the strong dependence on model architecture homogeneity limits the system's flexibility and scalability. All devices must accommodate the lowest-configuration nodes, severely wasting the computational potential of high-performance devices and hindering collaboration between different generations of devices. Third, knowledge distillation-based methods have limitations in computation and data dependence. Some data-free distillation methods require training complex generator networks on a central server or demand extensive inference computations from clients to align the outputs. Finally, the lack of an effective semantic alignment mechanism leads to chaotic heterogeneous feature representations. In cases of severely skewed data distribution (Non-IID), different clients may learn drastically different feature representations for samples of the same class.

[0005] Therefore, traditional heterogeneous federated learning methods often suffer from the problem of difficulty in balancing high communication efficiency and global model accuracy in resource-constrained edge networks due to huge communication overhead, strong dependence on model isomorphism, and lack of effective semantic alignment mechanisms. Summary of the Invention

[0006] In order to solve the above technical problems, a heterogeneous federated learning method and system based on frozen backbone and dual loss collaboration is provided, which can realize efficient collaboration and semantic alignment between heterogeneous devices, improve communication efficiency and global model accuracy.

[0007] A heterogeneous federated learning method based on frozen backbone and dual-loss synergy, the method comprising:

[0008] The central server initializes the global semantic reference point, global projection head parameters, and global classifier parameters and distributes them to each client participating in federated learning; the global semantic reference point is used to represent each category in a unified latent semantic space;

[0009] Each client freezes the local heterogeneous backbone network parameters, receives the global semantic reference point, global projection head parameters, and global classifier parameters, and uses the completely frozen heterogeneous backbone network to extract heterogeneous features of local data. The heterogeneous features are then mapped to the unified latent semantic space through the architecture-aware projection head to obtain latent semantic features of a unified dimension.

[0010] Each client calculates a composite loss function containing alignment loss and contrastive learning loss based on local data labels, the latent semantic features, and global semantic reference points, and uses the composite loss function to update only the local projection head parameters and classifier parameters.

[0011] Each client uploads the updated projector parameters and classifier parameters to the central server, but does not upload any parameters of the completely frozen heterogeneous backbone network; the central server aggregates the updated projector parameters and classifier parameters to obtain the updated global projector parameters and global classifier parameters.

[0012] The central server performs momentum updates on the global semantic reference points based on the updated global projector head parameters and global classifier parameters, obtains the updated global semantic reference points, and enters the next iteration to realize heterogeneous federated learning.

[0013] In one embodiment, each client freezes its local heterogeneous backbone network parameters, so that the backbone network parameters of each client are subject to a complete freeze constraint throughout the federated learning process, and do not participate in gradient calculation, backpropagation updates, or communication transmission with the central server.

[0014] In one embodiment, the architecture-aware projection head is a multilayer perceptron structure, which includes adaptive branches for different backbone network output dimensions, used to map heterogeneous features with different dimensions to a preset unified dimension latent semantic space.

[0015] In one embodiment, each client receives the global semantic reference point, global projection head parameters, and global classifier parameters, and extracts heterogeneous features from local data using a fully frozen heterogeneous backbone network. These heterogeneous features are then mapped to the unified latent semantic space using an architecture-aware projection head to obtain latent semantic features of a unified dimension, including:

[0016] Each client has a local dataset containing input samples and corresponding labels for those input samples;

[0017] Each client receives the global semantic reference point, global projection head parameters, and global classifier parameters sent by the central server;

[0018] For each sample in the local dataset, each client extracts heterogeneous features using a fully frozen backbone network;

[0019] Each client uses the architecture-aware projection head corresponding to the global projection head parameters to map the heterogeneous features to a unified latent semantic space, thereby obtaining latent semantic features of a unified dimension.

[0020] Each client uses the classifier corresponding to the parameters of the global classifier to output the predicted probability based on the latent semantic features.

[0021] In one embodiment, each client calculates a composite loss function comprising alignment loss and contrastive learning loss based on local data labels, the latent semantic features, and global semantic reference points, including:

[0022] Each client calculates the cross-entropy classification loss based on the predicted probability and the label corresponding to the input sample;

[0023] Each client uses L2 distance to calculate the Euclidean distance between the latent semantic features and the corresponding category global semantic reference point, and obtains the alignment loss, which is used to cluster similar latent semantic features toward the global semantic reference point;

[0024] Each client uses the InfoNCE method to calculate the contrast relationship between the latent semantic features and the global semantic reference points of all categories, and obtains the contrastive learning loss, which is used to estimate the distance between the latent semantic features of different categories and the global semantic reference points.

[0025] Each client constructs a composite loss function that includes the cross-entropy classification loss, alignment loss, and contrastive learning loss.

[0026] In one embodiment, the composite loss function is used to update only the local projection head parameters and classifier parameters, including:

[0027] Each client calculates the gradient data of the composite loss function with respect to the local projector head parameters and the local classifier parameters through backpropagation;

[0028] Each client updates its local projector head parameters and local classifier parameters using the gradient descent algorithm based on the gradient data.

[0029] In one embodiment, the central server aggregates the updated projection head parameters and classifier parameters to obtain updated global projection head parameters and global classifier parameters, including:

[0030] The central server receives the number of local samples from each client and calculates the total number of samples.

[0031] The central server receives updated projection head parameters and classifier parameters uploaded by each client;

[0032] The central server aggregates the updated projector parameters and classifier parameters using a weighted average method based on the total number of samples to obtain the updated global projector parameters and global classifier parameters.

[0033] In one embodiment, the central server performs momentum updates on the global semantic reference points based on the updated global projection head parameters and global classifier parameters to obtain updated global semantic reference points, including:

[0034] The central server contains various types of reference features. The updated global projection head parameters and global classifier parameters are used to reproject the reference features and calculate the correction direction of the semantic reference points.

[0035] The central server uses consensus information from the updated global projection head parameters to progressively update the semantic reference point using a momentum update formula.

[0036] The central server obtains the updated global semantic reference point based on the correction direction and the incremental update result, and then enters the next iteration.

[0037] A heterogeneous federated learning system based on frozen backbone and dual-loss synergy, the system comprising:

[0038] The central server is used to initialize global semantic reference points, global projection head parameters, and global classifier parameters, and distribute them to each client participating in federated learning; the global semantic reference points are used to represent each category in a unified latent semantic space.

[0039] Each client is used to freeze the local heterogeneous backbone network parameters, receive the global semantic reference point, global projection head parameters, and global classifier parameters, and use the fully frozen heterogeneous backbone network to extract the heterogeneous features of the local data. The heterogeneous features are then mapped to the unified latent semantic space through the architecture-aware projection head to obtain latent semantic features of a unified dimension.

[0040] Each of the aforementioned clients is also configured to calculate a composite loss function containing alignment loss and contrastive learning loss based on local data labels, the latent semantic features, and global semantic reference points, and to use the composite loss function to update only the local projection head parameters and classifier parameters.

[0041] Each of the aforementioned clients is also used to upload the updated projector parameters and classifier parameters to the central server, without uploading any parameters of the completely frozen heterogeneous backbone network; the central server aggregates the updated projector parameters and classifier parameters to obtain the updated global projector parameters and global classifier parameters;

[0042] The central server is also used to perform momentum updates on the global semantic reference points based on the updated global projection head parameters and global classifier parameters, to obtain updated global semantic reference points, and then enter the next iteration to realize heterogeneous federated learning.

[0043] The aforementioned heterogeneous federated learning method and system based on frozen backbone and dual-loss collaboration reduces communication overhead by freezing the local heterogeneous backbone network and training only the lightweight projection head and classifier parameters. By mapping heterogeneous features to a unified latent semantic space and optimizing it using a composite loss function composed of alignment loss and contrastive learning loss, the system effectively addresses the feature-semantic misalignment problem in heterogeneous models, improving feature discriminativity and classification accuracy. The client uploads only lightweight parameters, not backbone network parameters, significantly reducing communication data volume while protecting the local model. The central server updates global parameters through aggregation and performs momentum updates on global semantic reference points, enabling continuous self-reinforcing convergence of the semantic space, accelerating model iteration, and enhancing the stability and generalization ability of the global model under non-independent and identically distributed data. This achieves efficient collaboration and semantic alignment between heterogeneous devices, improving communication efficiency and global model accuracy. Attached Figure Description

[0044] Figure 1 This is a diagram illustrating the application environment of a heterogeneous federated learning method based on frozen backbone and dual-loss synergy in one embodiment.

[0045] Figure 2 This is a flowchart illustrating a heterogeneous federated learning method based on frozen backbone and dual-loss synergy in one embodiment.

[0046] Figure 3 This is a schematic diagram of the FeLSC framework for applying a heterogeneous federated learning method based on frozen backbone and dual loss synergy in one embodiment.

[0047] Figure 4 This is a schematic diagram of an ultra-lightweight decoupling architecture based on a frozen backbone in one embodiment.

[0048] Figure 5 This is a schematic diagram of a semantic space optimization mechanism based on contrast-alignment dual loss in one embodiment;

[0049] Figure 6 This is a schematic diagram of a semantic space self-reinforcing mechanism based on momentum update in one embodiment;

[0050] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] The heterogeneous federated learning method based on frozen backbone and dual-loss synergy provided in this application can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1 As shown, the application environment includes an interconnected central server 110 and various clients 120. The central server 110 initializes global semantic reference points, global projection head parameters, and global classifier parameters, and distributes them to each client 120 participating in federated learning. The global semantic reference points are used to represent each category in a unified latent semantic space. Each client 120 freezes its local heterogeneous backbone network parameters, receives the global semantic reference points, global projection head parameters, and global classifier parameters, and uses the fully frozen heterogeneous backbone network to extract heterogeneous features from its local data. It then maps these heterogeneous features to a unified latent semantic space using an architecture-aware projection head, obtaining latent semantic features of a unified dimension. Each client 120 calculates a set of parameters based on its local data labels, latent semantic features, and global semantic reference points. A composite loss function combining alignment loss and contrastive learning loss is used to update only the local projector head parameters and classifier parameters. Each client 120 uploads the updated projector head parameters and classifier parameters to the central server 110, without uploading any parameters of the completely frozen heterogeneous backbone network. The central server 110 aggregates the updated projector head parameters and classifier parameters to obtain the updated global projector head parameters and global classifier parameters. Based on the updated global projector head parameters and global classifier parameters, the central server 110 performs momentum updates on the global semantic reference points to obtain the updated global semantic reference points, and then enters the next iteration to achieve heterogeneous federated learning.

[0053] In one embodiment, such as Figure 2 As shown, a heterogeneous federated learning method based on frozen backbone and dual loss synergy is provided, including the following steps:

[0054] Step 202: The central server initializes the global semantic reference point, global projection head parameters, and global classifier parameters and distributes them to each client participating in federated learning; the global semantic reference point is used to represent each category in a unified latent semantic space.

[0055] The central server first performs a global initialization operation. Based on the total number of categories in the federated learning task and the preset unified latent semantic space dimension, it randomly initializes global semantic reference points to accurately represent the semantic features of the category in the unified latent semantic space. At the same time, the central server also needs to initialize the global projection head parameters adapted to multiple heterogeneous backbone networks. The projection head is a multilayer perceptron structure that can uniformly map local features of different dimensions to the preset dimension of the latent semantic space. In addition, the central server also needs to initialize the global classifier parameters to output the category prediction probability based on the latent semantic features.

[0056] Specifically, the central server maintains a set of global semantic reference points. ,in The total number of categories, Indicates the first The semantic center (prototype) of a class in a unified latent space. During the training initialization phase, the central server determines the latent space dimensions based on the preset dimensions. Initialize the reference point. The preferred initialization method is random initialization based on a standard normal distribution, or initialization based on prior knowledge. ;in, This represents a standard normal distribution. These semantic reference points guide each client in assigning a category during training. Sample features Mapped to nearby.

[0057] After initializing the three types of global parameters, the central server can broadcast the initialized parameters to all online clients participating in federated learning, without transmitting any client-side backbone network parameters, in order to protect the intellectual property rights of the client models and reduce communication overhead.

[0058] Step 204: Each client freezes the local heterogeneous backbone network parameters, receives the global semantic reference point, global projection head parameters, and global classifier parameters, and uses the fully frozen heterogeneous backbone network to extract the heterogeneous features of the local data. The heterogeneous features are then mapped to a unified latent semantic space through the architecture-aware projection head to obtain latent semantic features of a unified dimension.

[0059] The client needs to completely freeze its local heterogeneous backbone network parameters, training and transmitting only a very small projection head to map local features of different dimensions to a unified latent semantic space. The backbone network parameters remain unchanged throughout the federated training process, not participating in gradient calculations or parameter transmission, thereby reducing communication overhead per round, breaking the homogeneity constraint of the model architecture, and enabling federated learning to break its dependence on model architecture homogeneity while ensuring model performance, thus minimizing communication costs.

[0060] In one embodiment, each client freezes its local heterogeneous backbone network parameters, thereby imposing a complete freeze constraint on the backbone network parameters of each client throughout the federated learning process. These parameters do not participate in gradient calculation, backpropagation updates, or communication transmission with the central server.

[0061] In one embodiment, the architecture-aware projection head is a multilayer perceptron structure, including adaptive branches for different backbone network output dimensions, used to map heterogeneous features with different dimensions to a preset unified dimension latent semantic space.

[0062] In this embodiment, by decoupling the deep neural network into a completely frozen feature extractor and a trainable lightweight projection head, the client is allowed to train and transmit only a very small number of projection parameters while retaining the local heterogeneous backbone network. This reduces communication overhead and natively supports model heterogeneity. The backbone network parameters are completely frozen throughout the training process and do not participate in gradient calculation and transmission.

[0063] In one embodiment, a heterogeneous federated learning method based on frozen backbone and dual-loss collaboration may further include a process of completely freezing the backbone network and extracting heterogeneous features. Specifically, the process includes: each client possessing a local dataset containing input samples and corresponding labels; each client receiving global semantic reference points, global projection head parameters, and global classifier parameters from a central server; for each sample in the local dataset, each client extracting heterogeneous features using a completely frozen backbone network; each client mapping the heterogeneous features to a unified latent semantic space using an architecture-aware projection head corresponding to the global projection head parameters, obtaining latent semantic features of a unified dimension; and each client outputting a predicted probability based on the latent semantic features using a classifier corresponding to the global classifier parameters.

[0064] In traditional federated learning, the client needs to train and upload all parameters of the entire model, resulting in huge communication overhead. This embodiment proposes a strategy of completely freezing the backbone network, which completely eliminates the need to transmit backbone network parameters.

[0065] Specifically, assuming the federated learning system contains One client, each client Having a private dataset ,in For the input sample, For the corresponding labels. Due to differences in device hardware, each client deploys a specific feature extraction network. ,in This represents the parameters of the backbone network (e.g., parameters of different architectures such as ResNet and MobileNet). These backbone network parameters are adjusted throughout the federated training process. Apply a complete freeze constraint, i.e.: This means that the backbone network parameters do not participate in gradient calculation and backpropagation, and always maintain their pre-trained state.

[0066] For input samples The client first uses the frozen backbone network to compute its high-dimensional feature representation: ;in, Indicates sample In the client The representation in the local feature space, This represents the output dimension of the client's backbone network. Due to architectural heterogeneity, the feature dimensions differ between different clients. The dimensions may differ; for example, ResNet-10 outputs 512 dimensions, while MobileNetV2 outputs 1280 dimensions. Since backbone network parameters do not need to be transmitted, communication overhead can be significantly reduced; this avoids degrading the performance of the local model and reduces the computational resource consumption on the client side.

[0067] To address the different client feature dimensions To address the inconsistency issue, this embodiment introduces a lightweight projection head with architecture awareness. Heterogeneous local features This is mapped to a unified shared latent space. The projector head contains adapted branches for different backbone network output dimensions. Specifically, the projector head's input layer is designed as follows: ;in, For client Specific projection branches of the backbone network architecture. For example, for a ResNet-10 backbone (output 512 dimensions), the following is adopted: The mapping branch; for the MobileNetV2 backbone (output 1280 dimensions), the following is adopted. The mapping branches. This architecture-aware design allows clients with different hardware configurations to seamlessly access the federated learning system.

[0068] In this embodiment, the projection head A multilayer perceptron (MLP) structure containing nonlinear activation functions (such as ReLU) can be used, with parameters as follows: The mapping process can be represented as: ;in, For the latent semantic features after mapping, This is a pre-defined unified latent space dimension. Through this mapping, regardless of the client's original feature dimension... Ultimately, they will all be converted into vectors of a uniform dimension. This makes the features of different clients comparable in semantic space. The client also includes a classifier. Used based on latent features Output predicted probability: ;in, These are the classifier parameters.

[0069] During federated training, the client only needs to update and upload the projection head parameters. and classifier parameters However, no backbone parameters are transmitted at all. Taking ResNet-10 as an example, the backbone network has approximately 4.9M parameters (about 19.6MB), while the projection head has only about 0.33M parameters (about 1.32MB), and the classifier has even fewer parameters (about 0.01MB). Therefore, the communication overhead per round is reduced from 16.0MB in traditional methods to 1.33MB, a reduction of 91.7%; in a complete 100-round training run, the total communication overhead is reduced from 8010.9MB to 573.3MB, a reduction of 92.8%.

[0070] Step 206: Each client calculates a composite loss function that includes alignment loss and contrastive learning loss based on local data labels, latent semantic features, and global semantic reference points, and uses the composite loss function to update only the local projection head parameters and classifier parameters.

[0071] Because the client's backbone network is heterogeneous and frozen, directly aggregating projection head parameters may lead to semantic space chaos. To address this issue, this embodiment proposes a contrast-alignment dual-loss collaborative optimization mechanism. This mechanism maintains a shared set of semantic reference points on a central server and combines L2 alignment loss and InfoNCE (Noise Contrastive Estimation) contrastive learning loss to form a "clustering-separation" dual constraint. On one hand, the alignment loss function in the form of L2 distance pulls projected features toward the corresponding category's semantic reference point; on the other hand, the contrastive learning loss function in the form of InfoNCE explicitly pushes away dissimilar feature representations. This dual constraint of "bringing closer to similar features and pushing away dissimilar features" creates stronger semantic separation capabilities, enabling the model to learn more discriminative feature representations and effectively solving the semantic misalignment problem of heterogeneous features.

[0072] In one embodiment, a heterogeneous federated learning method based on frozen backbone and dual-loss synergy may further include a process of calculating a composite loss function. Specifically, this process includes: each client calculating a cross-entropy classification loss based on predicted probabilities and corresponding labels of the input samples; each client calculating the Euclidean distance between latent semantic features and corresponding global semantic reference points using L2 distance to obtain an alignment loss, used to cluster similar latent semantic features towards the global semantic reference points; each client calculating the contrastive relationship between latent semantic features and all categories of global semantic reference points using InfoNCE to obtain a contrastive learning loss, used to widen the distance between dissimilar latent semantic features and global semantic reference points; and each client constructing a composite loss function comprising cross-entropy classification loss, alignment loss, and contrastive learning loss.

[0073] In this embodiment, to enforce semantic alignment of heterogeneous features, a synergistic combination of three loss functions is introduced into the client's local training objective function, forming a triple constraint of "classification-alignment-contrast". For the client... Each sample Cross-entropy classification loss This is used to ensure classification accuracy. ;in, The logits vector output by the classifier; reference point alignment loss. The method uses L2 distance (mean squared error, MSE) to represent the projected latent features. Pull to the global semantic reference point of its corresponding category (Agglomeration effect): ;in, Representing the L2 norm, by minimizing the alignment loss, clients with different architectures are forced to align their feature spaces around a unified global semantic reference point, thereby achieving cross-architecture semantic consistency; InfoNCE contrastive learning loss. Using the InfoNCE (Noise Contrastive Estimation) approach, we explicitly push away out-of-class features (separation effect) by comparing positive and negative reference point pairs, thereby enhancing representation quality and promoting better semantic clustering in the latent space. ;in, This is the projected feature representation. This is a temperature parameter (usually set to 0.1) used to control the degree of concentration of the distribution.

[0074] Among them, the contrastive learning loss enables the model to learn more discriminative features by explicitly bringing similar representations closer together and pushing dissimilar representations further away; the introduced InfoNCE contrastive loss can form clearer category boundaries in the latent space. The alignment loss is responsible for "clustering" together features of the same type and pulling them toward the reference point; the contrastive loss is responsible for "separating" features of different types and pushing them further away. The two form a complementary dual constraint, achieving a stronger semantic alignment capability than a single loss.

[0075] Combining the above three losses, the total local composite loss function is defined as: ;in, and This is a balancing coefficient, typically set to 0.5, used to control the weight balance among the three loss terms. The client minimizes the composite loss function using the gradient descent algorithm, thereby ensuring classification performance while achieving alignment with the global semantic reference point and semantic consistency across clients.

[0076] In one embodiment, a heterogeneous federated learning method based on frozen backbone and dual loss synergy may further include a process of updating local parameters. Specifically, the process includes: each client calculating the gradient data of the composite loss function with respect to the local projector head parameters and local classifier parameters through backpropagation; and each client updating the local projector head parameters and local classifier parameters using a gradient descent algorithm based on the gradient data.

[0077] Each client calculates the gradient data of the composite loss function on the local projector head parameters and local classifier parameters through backpropagation. The backbone network parameters do not participate in any gradient calculation or parameter update and always maintain their pre-trained state.

[0078] Each client uses stochastic gradient descent (SGD) or its variants (such as Adam) to update the projection head and classifier parameters. By performing multiple rounds of iterative training on the local dataset until the preset number of local training rounds is completed, the updated projection head parameters and classifier parameters are obtained.

[0079] In step 208, each client uploads the updated projector parameters and classifier parameters to the central server, but does not upload any parameters of the completely frozen heterogeneous backbone network; the central server aggregates the updated projector parameters and classifier parameters to obtain the updated global projector parameters and global classifier parameters.

[0080] In this embodiment, a gradual and stable convergence of the semantic space can be achieved through a closed-loop feedback mechanism. The central server first aggregates the projection head parameters uploaded by each client to obtain global consensus. Then, it uses the global projection head to perform momentum updates on the semantic reference points, enabling the reference point positions to adaptively adjust with the evolution of the global model. The smoothness of the update is controlled by the momentum coefficient to avoid drastic oscillations in the reference point positions, ultimately forming a stable semantic space with a clear structure and separable classes.

[0081] In one embodiment, a heterogeneous federated learning method based on frozen backbone and dual loss collaboration may further include a parameter aggregation process, specifically including: a central server receiving the number of local samples from each client and calculating the total number of samples; the central server receiving the updated projection head parameters and classifier parameters uploaded by each client; and the central server aggregating the updated projection head parameters and classifier parameters respectively using a weighted average method based on the total number of samples to obtain the updated global projection head parameters and global classifier parameters.

[0082] In each round of communication, the central server receives data from the set of online clients. Projector head parameters and classifier parameters The central server updates the global projection head using a weighted average method. and global classifier : in, For the client The number of samples, The total number of samples in this round ensures that the global model can absorb the general feature mapping capabilities learned by each client, forming a global consensus on the latent semantic space.

[0083] Step 210: The central server performs momentum update on the global semantic reference points based on the updated global projection head parameters and global classifier parameters, obtains the updated global semantic reference points, and enters the next iteration to realize heterogeneous federated learning.

[0084] The central server uses the aggregated global projection head to perform closed-loop momentum updates on semantic reference points. Through gradual position adjustments, it continuously optimizes the reference point positions, achieving stable convergence of the semantic space and further improving the generalization ability of the global model. The aggregated global projection head contains the consensus of all clients on the latent space in the current round. To utilize this consensus to optimize the semantic reference point positions, the central server performs momentum-based gradual updates on the global semantic reference points.

[0085] In one embodiment, a heterogeneous federated learning method based on frozen backbone and dual-loss collaboration may further include a momentum update process. Specifically, the process includes: a central server containing reference features of various categories; reprojecting the reference features using updated global projector parameters and global classifier parameters; calculating the correction direction of the semantic reference points; the central server using consensus information from the updated global projector parameters to progressively update the semantic reference points using a momentum update formula; and the central server obtaining the updated global semantic reference points based on the correction direction and the progressive update results, and proceeding to the next iteration.

[0086] Specifically, the updating of reference points follows these rules: ;in, This is the momentum coefficient, used to maintain the smoothness of the reference point's movement and avoid drastic positional oscillations; This represents a correction term based on the current global state. In the embodiment, the correction term... Preferably, a random perturbation term is used to simulate the exploration process to avoid the reference point getting trapped in a local optimum: Alternatively, the correction direction can be obtained by reprojecting the reference features using a global projection head. ;in, This is the aggregated global projection head. For the pre-stored first Class reference features.

[0087] In this embodiment, through this closed-loop update mechanism, the position of the semantic reference point can be adaptively adjusted as the global model converges; momentum coefficient In the early stages of training, a smaller value (e.g., 0.5) can be set to accelerate exploration, and then gradually increased (e.g., 0.95) can be set to ensure stability. Experiments show that using this self-reinforcing mechanism can reduce the number of convergence rounds from 98 rounds (random fixed reference point) to 67 rounds, accelerating the process by 31.6%, and ultimately forming a stable semantic space with a clear structure and separability between classes.

[0088] In one embodiment, a heterogeneous federated learning method based on frozen backbone and dual-loss collaboration is provided, which includes the interaction process between a central server and heterogeneous clients during the federated learning process. The specific process includes:

[0089] Initialization phase: The central server initializes the global semantic reference point, projection head, and classifier parameters;

[0090] Distribution phase: The central server sends the global parameters and semantic reference points for the current round to the selected online clients, without transmitting any backbone network parameters;

[0091] Local training phase: After receiving the parameters, the client freezes the local backbone network, and the parameters do not participate in gradient updates at all. Forward propagation is performed based on local data; cross-entropy loss, alignment loss (L2), and contrastive learning loss (InfoNCE) are calculated; the projection head and classifier parameters are updated through backpropagation.

[0092] Upload and aggregation phase: The client uploads the updated projector and classifier parameters to the central server; the central server performs weighted aggregation of the parameters and updates the global model;

[0093] Refinement phase: The central server performs momentum updates on the semantic reference points based on the new global projection head, completes this round of training, and enters the next round of iteration.

[0094] In one embodiment, a heterogeneous federated learning method based on frozen backbone and dual-loss synergy can be applied to, for example... Figure 3In the FeLSC framework shown, two heterogeneous clients extract features using different frozen backbone networks (such as ResNet-10 and MobileNetV2), and map these features to a unified latent space through an architecture-aware lightweight projection head. A central server maintains global semantic reference points, which serve as class-level prototypes to guide feature alignment in the clients. During client training, both contrastive learning loss (InfoNCE) and alignment loss (L2) are used for optimization, and the reference points are continuously updated through a momentum update mechanism, thereby achieving semantic consensus in a heterogeneous environment.

[0095] This application presents a heterogeneous federated learning method based on frozen backbone and dual-loss collaboration. By introducing InfoNCE contrastive learning loss and L2 alignment loss for collaborative optimization, it effectively solves the semantic alignment problem of heterogeneous features and significantly improves the classification performance of the global model. By adopting a strategy of completely freezing the backbone network, the client only needs to transmit a very small number of projection head parameters, thereby reducing communication costs by orders of magnitude and adapting to weak network environments with extremely limited bandwidth. Through architecture-aware projection head design, it achieves seamless compatibility with heterogeneous client model architectures, breaking the limitation of model architecture homogeneity in traditional federated learning. The projection head includes adaptive branches for different backbone network output dimensions, supporting efficient collaboration between clients with different architectures (such as ResNet-10 and MobileNetV2). It also enhances the robustness of the model under non-independent identically distributed (Non-IID) data. For scenarios with severely skewed data distribution, the contrast-alignment dual-loss mechanism of this invention provides a strong supervisory signal, ensuring the stability of the model under data heterogeneity. The semantic space self-reinforcing mechanism through momentum update significantly accelerates the convergence speed of the global model, and the global semantic reference point can quickly adapt to the evolution of the feature space, accelerating model convergence. Through the dual constraint of "clustering-separation" (alignment loss brings similar features closer together, contrast loss pushes dissimilar features away), features from different clients are forced to cluster towards the global semantic reference point. Similar sample features from different clients gradually cluster in the latent space and coincide with the global reference point, achieving high-quality semantic consensus in a heterogeneous feature space. Each core component makes an independent and positive contribution to performance. Whether it is the reference point refinement mechanism (AR), alignment loss (AL), contrast learning loss (IL), or reference point initialization strategy (RA), none of them can be omitted.

[0096] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0097] In one embodiment, such as Figure 1 As shown, a heterogeneous federated learning system based on frozen backbone and dual-loss synergy is provided, including:

[0098] The central server 110 is used to initialize the global semantic reference point, global projection head parameters, and global classifier parameters and distribute them to each client 120 participating in federated learning; the global semantic reference point is used to represent each category in a unified latent semantic space;

[0099] Each client 120 is used to freeze the parameters of the local heterogeneous backbone network, receive the global semantic reference point, global projection head parameters, and global classifier parameters, and use the fully frozen heterogeneous backbone network to extract the heterogeneous features of the local data. The heterogeneous features are then mapped to a unified latent semantic space through the architecture-aware projection head to obtain latent semantic features of a unified dimension.

[0100] Each client 120 is also used to calculate a composite loss function containing alignment loss and contrastive learning loss based on local data labels, latent semantic features, and global semantic reference points, and to update only the local projection head parameters and classifier parameters using the composite loss function;

[0101] Each client 120 is also used to upload the updated projection head parameters and classifier parameters to the central server 110, but does not upload any parameters of the completely frozen heterogeneous backbone network; the central server 110 aggregates the updated projection head parameters and classifier parameters to obtain the updated global projection head parameters and global classifier parameters.

[0102] The central server 110 is also used to perform momentum updates on the global semantic reference points based on the updated global projection head parameters and global classifier parameters, to obtain the updated global semantic reference points, and then enter the next iteration to realize heterogeneous federated learning.

[0103] In one embodiment, the central server acts as the system's control center, responsible for maintaining global model parameters (including global projectors and global classifiers) and global semantic anchors. The central server includes a parameter aggregation module and an anchor refinement module. The parameter aggregation module receives projector and classifier parameters uploaded by each client and performs weighted aggregation. The anchor refinement module uses the aggregated global projectors to update the semantic anchors' momentum, thereby optimizing the semantic distribution in the latent space.

[0104] Heterogeneous clients act as data holders and computation executors, each possessing local private data and a specific model architecture (such as ResNet, MobileNet, etc.). The client side includes a frozen feature extractor, a lightweight projection head, and a local classifier. The frozen feature extractor extracts high-dimensional features from the raw data, and its parameters remain unchanged during training. The lightweight projection head maps heterogeneous features to a unified latent space. The local classifier makes predictions based on the latent features.

[0105] The central server sends global parameters and anchor points to the client; the client uses local data for training, aligns features to anchor points using alignment loss, and uploads the updated parameters to the central server.

[0106] In one embodiment, the client is configured with an extremely lightweight decoupling architecture based on a frozen backbone. This architecture employs a feature decoupling strategy using a frozen backbone, aiming to address model heterogeneity issues and achieve an order-of-magnitude reduction in communication overhead. For example... Figure 4 As shown, it mainly includes:

[0107] Freeze Feature Extractor: Uses pre-trained heterogeneous backbone networks (such as ResNet and MobileNet) to extract local features. The parameters are completely frozen during federated training and do not participate in gradient updates and parameter transfer, thereby achieving a 92.8% reduction in communication overhead.

[0108] Architecture-aware projection head: Composed of a multilayer perceptron (MLP), it includes adaptation branches for different backbone network output dimensions (such as 512 dimensions of ResNet and 1280 dimensions of MobileNetV2), and is responsible for mapping local features of different dimensions to a latent semantic space of a unified dimension, thereby achieving feature dimension alignment.

[0109] Dual loss calculation and update: Combining contrastive learning loss (InfoNCE, to achieve "clustering-separation" dual constraints) and alignment loss (L2, to ensure semantic consistency) and cross-entropy loss (to ensure classification accuracy), the parameters of the lightweight projector and classifier are updated only through the backpropagation algorithm.

[0110] In one embodiment, the principle of the semantic space optimization mechanism based on contrast-alignment dual loss is as follows: Figure 5 As shown, combining the InfoNCE contrastive learning loss and the L2 alignment loss forms a "clustering-separation" dual constraint. Figure 5 As shown, it mainly includes:

[0111] Reference point initialization: In the initial stage of training, the central server initializes semantic reference points (class-level prototypes) for each category based on the potential space dimension.

[0112] Reference point distribution: The central server broadcasts global semantic reference points to all clients participating in the training.

[0113] Dual-loss semantic alignment: During local training, the client simultaneously calculates two types of losses: (1) Alignment loss - calculates the L2 distance between latent features and corresponding category reference points, forcing features of the same category to move closer to the reference points; (2) Contrastive learning loss - calculates the contrast relationship between features and all reference points based on the InfoNCE formula, bringing positive sample pairs (of the same category) closer and pushing negative sample pairs (of different categories) further apart. This dual constraint overcomes the semantic misalignment caused by heterogeneous architecture.

[0114] In one embodiment, the principle of the semantic space self-reinforcing mechanism based on momentum update is as follows: Figure 6 As shown, the aggregated global model is used to dynamically update the semantic reference points, achieving self-reinforcing convergence of the semantic space. Figure 6 As shown, it mainly includes:

[0115] Parameter aggregation: The parameter aggregation module on the central server collects projection head parameters from each client and generates a global projection head through weighted averaging (FedAvg);

[0116] Reference point refinement: The reference point refinement module utilizes the consensus information contained in the global projection head, through the momentum update formula. The old reference point is updated smoothly. This process allows the reference point position to be dynamically adjusted as the global model evolves, ensuring that the reference point always represents the current optimal class-level semantic center, which can accelerate the convergence speed by 31.6% compared to using a fixed reference point.

[0117] In one embodiment, each client freezes its local heterogeneous backbone network parameters, thereby imposing a complete freeze constraint on the backbone network parameters of each client throughout the federated learning process. These parameters do not participate in gradient calculation, backpropagation updates, or communication transmission with the central server.

[0118] In one embodiment, the architecture-aware projection head is a multilayer perceptron structure, including adaptive branches for different backbone network output dimensions, used to map heterogeneous features with different dimensions to a preset unified dimension latent semantic space.

[0119] In one embodiment, each client is also configured to possess a local dataset containing input samples and corresponding labels; receive global semantic reference points, global projection head parameters, and global classifier parameters from the central server; for each sample in the local dataset, each client extracts heterogeneous features using a fully frozen backbone network; each client uses the architecture-aware projection head corresponding to the global projection head parameters to map the heterogeneous features to a unified latent semantic space, obtaining latent semantic features of a unified dimension; and each client uses the classifier corresponding to the global classifier parameters to output predicted probabilities based on the latent semantic features.

[0120] In one embodiment, each client is further configured to calculate the cross-entropy classification loss based on the predicted probability and the label corresponding to the input sample; calculate the Euclidean distance between the latent semantic features and the corresponding global semantic reference point using the L2 distance form to obtain the alignment loss, which is used to cluster latent semantic features of the same class toward the global semantic reference point; calculate the contrastive relationship between the latent semantic features and the global semantic reference point of all classes using the InfoNCE form to obtain the contrastive learning loss, which is used to widen the distance between latent semantic features of different classes and the global semantic reference point; and construct a composite loss function that includes the cross-entropy classification loss, the alignment loss, and the contrastive learning loss.

[0121] In one embodiment, each client is also used to calculate the gradient data of the composite loss function with respect to the local projector head parameters and the local classifier parameters through backpropagation; and to update the local projector head parameters and the local classifier parameters using a gradient descent algorithm based on the gradient data.

[0122] In one embodiment, the central server is also used to receive the number of local samples from each client and calculate the total number of samples; receive the updated projection head parameters and classifier parameters uploaded by each client; and aggregate the updated projection head parameters and classifier parameters respectively using a weighted average method based on the total number of samples to obtain the updated global projection head parameters and global classifier parameters.

[0123] In one embodiment, the central server is further configured to: have reference features of various categories; reproject the reference features using updated global projector parameters and global classifier parameters; calculate the correction direction of the semantic reference points; use the consensus information of the updated global projector parameters to progressively update the semantic reference points using the momentum update formula; obtain the updated global semantic reference points based on the correction direction and the progressive update results, and proceed to the next iteration.

[0124] In one embodiment, a heterogeneous federated learning method and system based on frozen backbone and dual-loss collaboration is applied to a smart city traffic monitoring scenario. The urban traffic network deploys a large number of heterogeneous monitoring terminals (as clients), including high-definition checkpoint cameras, PTZ cameras, and bullet cameras. These devices have built-in computing chips of different architectures (such as FPGA, ASIC, GPU) and pre-trained models (such as ResNet-10, MobileNetV2, ShuffleNet). The goal of this embodiment is to collaboratively train a high-precision fine-grained vehicle classification model without uploading raw video data to protect privacy, without transmitting backbone network parameters to reduce communication overhead, and while maintaining compatibility with the computing power of heterogeneous terminals. The specific implementation steps for application in a smart city traffic monitoring scenario include:

[0125] Global semantic reference point initialization: The central server first defines the reference points according to the requirements of the vehicle recognition task. Categories (e.g., cars, SUVs, trucks, buses, motorcycles, etc.). A unified latent semantic space dimension is defined. (For example, 256 dimensions). The central server uses a standard normal distribution. Initialize the global semantic reference point for each category. These reference points constitute the baseline coordinate system in the semantic space.

[0126] Model and reference point distribution: The central server distributes the initialized global semantic reference points via wireless communication networks (4G / 5G / Wi-Fi). Global projection head parameters and global classifier parameters The broadcast is distributed to all online traffic monitoring terminals. Note: No backbone network parameters are transmitted.

[0127] Local heterogeneous feature extraction (backbone network completely frozen): After receiving global parameters, each monitoring terminal utilizes the locally fixed heterogeneous backbone network. Feature extraction is performed on the collected vehicle image data. Backbone network parameters. Completely frozen, it does not participate in gradient calculation, parameter updates, or communication transmission. This is the core strategy of this invention to achieve a 92.8% reduction in communication overhead. For example, high-performance camera mounts use ResNet-10 to extract 512-dimensional features, while low-power roadside PTZ cameras use MobileNetV2 to extract 1280-dimensional features.

[0128] Architecture-aware projection mapping: Heterogeneous feature vectors extracted by each terminal Input to a locally trainable architecture-aware projector The projection head includes adaptation branches for different backbone network output dimensions (such as a 512-dimensional to 256-dimensional branch, or a 1280-dimensional to 256-dimensional branch), mapping local features of different dimensions to latent semantic features of a unified dimension (256 dimensions). This eliminates architectural differences between devices.

[0129] Contrast-alignment dual-loss collaborative optimization and local update: The terminal uses local data for training. For each sample, three types of losses are calculated simultaneously: cross-entropy classification loss. : Ensure classification accuracy; L2 alignment loss Potential characteristics Reference points for corresponding categories The mean squared error between them forces similar features to converge towards the reference point; InfoNCE contrastive learning loss Based on the principle of contrastive learning, sample features are compared with reference points of all categories to achieve a dual constraint of "clustering-separation"—bringing positive sample pairs (of the same class) closer together and pushing negative sample pairs (of different classes) further apart. This is achieved by minimizing the total loss. The stochastic gradient descent (SGD) algorithm is used to update only the lightweight projector head parameters. and classifier parameters .

[0130] Lightweight parameter upload: After training, each terminal only uploads the updated projection head parameters. and classifier parameters The data is uploaded to the central server. Because it does not transmit large amounts of backbone network parameters, the uploaded data size is only 1.33MB (a 91.7% reduction compared to the traditional method's 16.0MB), greatly reducing the bandwidth consumption on the city's dedicated monitoring network.

[0131] Global consensus aggregation: The central server receives parameters uploaded by each terminal, performs weighted aggregation using the FedAvg algorithm, and generates a new global projection head. and classifier This step integrates vehicle feature mapping capabilities learned from different perspectives and devices.

[0132] Semantic reference point momentum update: The central server uses the aggregated global projection head to update the momentum of the semantic reference points. Specifically, according to the formula... (in (The momentum coefficient) is used to smoothly adjust the reference point position using global consensus information, making it more accurately represent the distribution center of the current global vehicle features, thus completing this iteration. Compared to using a fixed reference point, this mechanism can accelerate the convergence speed by 31.6%.

[0133] In one embodiment, a heterogeneous federated learning method and system based on frozen backbone and contrast-alignment dual-loss synergy is applied in a multi-center medical image-assisted diagnosis scenario. Multiple hospitals (clients) possess medical imaging equipment from different brands (such as CT / MRI equipment from GE, Siemens, and Philips) and locally pre-trained models. Due to data privacy regulations (such as HIPAA), hospitals cannot share raw patient image data. This embodiment aims to protect patient privacy and ensure compatibility with heterogeneous imaging equipment by jointly training a general disease-assisted diagnosis model (such as pneumonia detection) through a frozen backbone and contrast-alignment dual-loss synergy mechanism. Specific implementation steps in the multi-center medical image-assisted diagnosis scenario include:

[0134] Disease semantic reference point construction: The central server, acting as the coordination center, defines diagnostic categories (such as normal, bacterial pneumonia, viral pneumonia) and initializes global semantic reference points for each disease category. These reference points serve as the mathematical representation of the "standard case" in the latent space.

[0135] Privacy-preserving distribution: The central server sends the global semantic reference point, global projection head, and classifier parameters to the local central servers of each participating hospital. Note: Only a very lightweight set of reference point and projection head parameters are transmitted; no backbone network parameters are transmitted. The transmission process uses an encrypted channel to ensure the security of the model parameters.

[0136] Local Image Feature Extraction (Completely Frozen Backbone): Each hospital utilizes its existing local image analysis models (such as DenseNet121, InceptionV3, ResNet-50, etc.) as the backbone network to extract features from in-hospital image data. The backbone network is completely frozen (parameters do not participate in gradient updates and communication transmission), which not only protects the hospital's original intellectual property but also avoids damage to the performance of local models, achieving a 92.8% reduction in communication overhead.

[0137] Architecture-aware cross-domain feature projection: This technology uses an architecture-aware projection head to map image features from different hospitals and devices to a shared latent semantic space. The projection head includes adaptive branches for different backbone network output dimensions, enabling cross-domain and cross-device feature alignment and making image features from different sources semantically comparable.

[0138] Contrast-Alignment Dual Loss Guided Training: During local training, two types of losses are used simultaneously for optimization: L2 alignment loss: forces local image features to cluster towards the corresponding disease semantic reference points, effectively allowing models from different hospitals to learn a unified diagnostic standard without sharing data; InfoNCE contrastive learning loss: achieves a dual constraint of "clustering-separation," not only bringing positive sample pairs closer together but also actively pushing away features from different disease categories, enhancing inter-class separability. Ablation experiments show that the contrastive learning loss alone contributed 2.16 percentage points to the accuracy improvement.

[0139] Parameter encryption upload: Each hospital only uploads updated projector head parameters and classifier parameters (communication volume is only 1.33MB, a 91.7% reduction compared to traditional methods). Since no image data or backbone network parameters are uploaded, it strictly complies with medical data privacy protection requirements.

[0140] Global model fusion: The central server aggregates parameters uploaded by various hospitals, integrating diagnostic knowledge from multiple centers. This process effectively utilizes long-tail data from various hospitals, improving the model's generalization ability to rare cases.

[0141] Semantic reference point momentum evolution: The central server updates the momentum of disease semantic reference points based on the fused global model. As training progresses, the reference point will gradually move to the common center of the characteristic distribution of each hospital, achieving dynamic unification and optimization of diagnostic criteria. Compared with using a fixed reference point, this can accelerate the convergence speed by 31.6%.

[0142] In one embodiment, a computer device is provided, which may include a central server and various client computers, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a heterogeneous federated learning method based on frozen backbone and dual-loss collaboration. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0143] Those skilled in the art will understand that Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0144] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement steps of a heterogeneous federated learning method based on frozen backbone and dual-loss synergy.

[0145] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements steps of a heterogeneous federated learning method based on frozen backbone and dual-loss synergy.

[0146] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0147] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A heterogeneous federated learning method based on frozen backbone and dual-loss synergy, characterized in that, The method includes: The central server initializes the global semantic reference point, global projection head parameters, and global classifier parameters and distributes them to each client participating in federated learning; the global semantic reference point is used to represent each category in a unified latent semantic space; Each client freezes the local heterogeneous backbone network parameters, receives the global semantic reference point, global projection head parameters, and global classifier parameters, and uses the completely frozen heterogeneous backbone network to extract heterogeneous features of local data. The heterogeneous features are then mapped to the unified latent semantic space through the architecture-aware projection head to obtain latent semantic features of a unified dimension. Each client calculates a composite loss function containing alignment loss and contrastive learning loss based on local data labels, the latent semantic features, and global semantic reference points, and uses the composite loss function to update only the local projection head parameters and classifier parameters. Each client uploads the updated projector parameters and classifier parameters to the central server, but does not upload any parameters of the completely frozen heterogeneous backbone network; the central server aggregates the updated projector parameters and classifier parameters to obtain the updated global projector parameters and global classifier parameters. The central server performs momentum updates on the global semantic reference points based on the updated global projector head parameters and global classifier parameters, obtains the updated global semantic reference points, and enters the next iteration to realize heterogeneous federated learning.

2. The heterogeneous federated learning method based on frozen backbone and dual-loss synergy as described in claim 1, characterized in that, Each client freezes its local heterogeneous backbone network parameters, thus imposing a complete freeze constraint on the backbone network parameters of each client throughout the federated learning process. These parameters do not participate in gradient calculation, backpropagation updates, or communication transmission with the central server.

3. The heterogeneous federated learning method based on frozen backbone and dual-loss synergy as described in claim 1, characterized in that, The architecture-aware projection head is a multi-layer perceptron structure, which includes adaptive branches for different backbone network output dimensions, used to map heterogeneous features with different dimensions to a preset unified dimension latent semantic space.

4. The heterogeneous federated learning method based on frozen backbone and dual-loss synergy as described in claim 1, characterized in that, Each client receives the global semantic reference point, global projection head parameters, and global classifier parameters, and extracts heterogeneous features from local data using a fully frozen heterogeneous backbone network. These heterogeneous features are then mapped to the unified latent semantic space using an architecture-aware projection head, resulting in unified-dimensional latent semantic features, including: Each client has a local dataset containing input samples and corresponding labels for those input samples; Each client receives the global semantic reference point, global projection head parameters, and global classifier parameters sent by the central server; For each sample in the local dataset, each client extracts heterogeneous features using a fully frozen backbone network; Each client uses the architecture-aware projection head corresponding to the global projection head parameters to map the heterogeneous features to a unified latent semantic space, thereby obtaining latent semantic features of a unified dimension. Each client uses the classifier corresponding to the parameters of the global classifier to output the predicted probability based on the latent semantic features.

5. The heterogeneous federated learning method based on frozen backbone and dual-loss synergy as described in claim 4, characterized in that, Each client calculates a composite loss function, including alignment loss and contrastive learning loss, based on local data labels, the latent semantic features, and global semantic reference points, including: Each client calculates the cross-entropy classification loss based on the predicted probability and the label corresponding to the input sample; Each client uses L2 distance to calculate the Euclidean distance between the latent semantic features and the corresponding category global semantic reference point, and obtains the alignment loss, which is used to cluster similar latent semantic features toward the global semantic reference point; Each client uses the InfoNCE method to calculate the contrast relationship between the latent semantic features and the global semantic reference points of all categories, and obtains the contrastive learning loss, which is used to estimate the distance between the latent semantic features of different categories and the global semantic reference points. Each client constructs a composite loss function that includes the cross-entropy classification loss, alignment loss, and contrastive learning loss.

6. The heterogeneous federated learning method based on frozen backbone and dual-loss synergy as described in claim 5, characterized in that, The composite loss function is used to update only the local projection head parameters and classifier parameters, including: Each client calculates the gradient data of the composite loss function with respect to the local projector head parameters and the local classifier parameters through backpropagation; Each client updates its local projector head parameters and local classifier parameters using the gradient descent algorithm based on the gradient data.

7. The heterogeneous federated learning method based on frozen backbone and dual-loss synergy as described in claim 1, characterized in that, The central server aggregates the updated projection head parameters and classifier parameters to obtain updated global projection head parameters and global classifier parameters, including: The central server receives the number of local samples from each client and calculates the total number of samples. The central server receives updated projection head parameters and classifier parameters uploaded by each client; The central server aggregates the updated projector parameters and classifier parameters using a weighted average method based on the total number of samples to obtain the updated global projector parameters and global classifier parameters.

8. The heterogeneous federated learning method based on frozen backbone and dual-loss synergy as described in claim 1, characterized in that, The central server performs momentum updates on the global semantic reference points based on the updated global projection head parameters and global classifier parameters, obtaining updated global semantic reference points, including: The central server contains various types of reference features. The updated global projection head parameters and global classifier parameters are used to reproject the reference features and calculate the correction direction of the semantic reference points. The central server uses consensus information from the updated global projection head parameters to progressively update the semantic reference point using a momentum update formula. The central server obtains the updated global semantic reference point based on the correction direction and the incremental update result, and then proceeds to the next iteration.

9. A heterogeneous federated learning system based on frozen backbone and dual-loss collaboration, characterized in that, The system includes: The central server is used to initialize global semantic reference points, global projection head parameters, and global classifier parameters, and distribute them to each client participating in federated learning; the global semantic reference points are used to represent each category in a unified latent semantic space. Each client is used to freeze the local heterogeneous backbone network parameters, receive the global semantic reference point, global projection head parameters, and global classifier parameters, and use the fully frozen heterogeneous backbone network to extract the heterogeneous features of the local data. The heterogeneous features are then mapped to the unified latent semantic space through the architecture-aware projection head to obtain latent semantic features of a unified dimension. Each of the aforementioned clients is also configured to calculate a composite loss function containing alignment loss and contrastive learning loss based on local data labels, the latent semantic features, and global semantic reference points, and to use the composite loss function to update only the local projection head parameters and classifier parameters. Each of the aforementioned clients is also used to upload the updated projector parameters and classifier parameters to the central server, without uploading any parameters of the completely frozen heterogeneous backbone network; the central server aggregates the updated projector parameters and classifier parameters to obtain the updated global projector parameters and global classifier parameters; The central server is also used to perform momentum updates on the global semantic reference points based on the updated global projection head parameters and global classifier parameters, to obtain updated global semantic reference points, and then enter the next iteration to realize heterogeneous federated learning.

Images