A heterogeneous federated learning method and system based on a double-anchor mechanism
The heterogeneous parsing federated learning method with dual anchor points solves the problems of feature rigidity and update rigidity in heterogeneous data scenarios by using global and local anchor points to constrain model parameters, thereby achieving efficient personalized model training and improving performance and training efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN INST OF TECH AT WEIHAI
- Filing Date
- 2026-03-27
- Publication Date
- 2026-06-02
AI Technical Summary
Existing federated learning methods face the problems of feature rigidity and update rigidity in heterogeneous data scenarios, resulting in performance degradation and low training efficiency on some clients, making them unable to adapt to personalized needs.
We adopt a heterogeneous parsing federated learning method based on a dual-anchor mechanism. By using a projection adaptation mechanism and a dual-anchor regularization term, we break the rigidity of features and updates, and use global anchors and local anchors to constrain model parameters, thereby achieving a balance between personalization and generalization ability.
While maintaining efficient communication, it significantly improves the model's performance and personalization capabilities in heterogeneous data scenarios, reduces the risk of gradient vanishing or exploding, and is suitable for scenarios with limited network conditions.
Smart Images
Figure CN121935707B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and distributed computing technology, and specifically relates to a heterogeneous analytical federated learning method and system based on a dual-anchor mechanism. Background Technology
[0002] With the rapid development of big data and artificial intelligence technologies, data privacy protection has become an increasingly important issue. Federated Learning (FL), as an emerging distributed machine learning paradigm, allows clients to collaboratively train models without sharing raw data. It has broad application prospects in fields such as smart healthcare, smart finance, smart IoT, and mobile edge computing. For example, in smart healthcare, multiple hospitals want to jointly train disease prediction models but cannot share patient data; in smart finance, different financial institutions need to collaboratively build credit assessment models but are constrained by privacy regulations; in smart IoT scenarios, edge devices need to collaboratively learn user behavior patterns, but the data cannot be centrally processed. However, existing federated learning methods generally face the following technical problems in these scenarios. First, in practical applications, the data distribution of different clients often exhibits non-independent and identically distributed characteristics. For example, the patient groups treated by different hospitals differ significantly, the customer profiles of different financial institutions are different, and the distribution of sensing data from different IoT devices varies. This data heterogeneity leads to slow convergence of traditional gradient-based iterative federated learning methods (such as FedAvg), and the performance of the global model degrades significantly on some clients. Second, traditional gradient basis federated learning methods require multiple rounds of communication to converge. In scenarios with limited network conditions, such as mobile edge computing, a large number of communication rounds lead to low training efficiency and make it difficult to deploy in practice.
[0003] Analytic Federated Learning (AFL), as an emerging paradigm, directly computes closed-form solutions using the least squares method, requiring only one round of communication to complete training, theoretically solving the communication efficiency problem. However, existing AFL methods face a rigidity dilemma. All clients use the same frozen backbone network to extract features, which, in highly heterogeneous scenarios such as smart healthcare, fails to effectively extract specific features adapted to the data distribution of each client, resulting in limited feature representation capabilities. Existing single-round fixed aggregation rules cannot adapt to the heterogeneity of client data, and in scenarios with strong personalized needs such as smart finance, they cannot eliminate parameter drift during local updates, causing the global model to fail to balance generalization ability and personalized requirements.
[0004] Therefore, how to break the rigidity of features and updates while maintaining the efficiency of parsing federated learning, and achieve robust federated learning that adapts to heterogeneous data scenarios, is a technical problem that urgently needs to be solved in this field. Summary of the Invention
[0005] To address the aforementioned technical challenges, this invention proposes a heterogeneous parsing federated learning method and system based on a dual-anchor mechanism. By breaking feature rigidity through a projection adaptation mechanism and eliminating update rigidity through a dual-anchor mechanism, the method maintains the efficiency of parsing federated learning while improving the model's performance and personalization capabilities in heterogeneous data scenarios.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] Firstly, this invention proposes a heterogeneous parsing federated learning method based on a dual-anchor mechanism, applied to a server, comprising the following steps:
[0008] Distribute the global backbone network and random projection matrix to multiple clients;
[0009] The system receives local classifier parameters and local autocorrelation matrices uploaded by each client. These parameters and autocorrelation matrices are obtained by the client based on enhanced features obtained through adaptive projection processing of local private data, and then calculated using different methods: the local classifier parameters are calculated through parsing learning based on the enhanced features and local labels; the local autocorrelation matrix is obtained by performing matrix autocorrelation operations on the enhanced features and adding a regularization term.
[0010] The aggregated autocorrelation matrix is obtained by aggregating all local autocorrelation matrices. Based on the local classifier parameters uploaded by all clients, the mask representation matrix is constructed by iteratively integrating the information of each client. Then, the global classifier parameters are calculated based on the aggregated autocorrelation matrix and the mask representation matrix.
[0011] Broadcast the global classifier parameters to all clients, enabling clients to optimize their local models based on the global classifier parameters.
[0012] Furthermore, all local autocorrelation matrices are aggregated to obtain an aggregated autocorrelation matrix. Based on the local classifier parameters uploaded by all clients, a mask representation matrix is constructed by iteratively integrating the information from each client. Specifically:
[0013] ;
[0014] in, Represents the aggregated autocorrelation matrix; Indicates the number of clients; Indicates the first Autocorrelation matrix of each client;
[0015] Set the initial matrix This serves as the starting point for iteratively constructing the mask representation matrix; iterates through each client sequentially, for the... Local classifier parameters for each client The mask representation matrix is iteratively updated using the following formula;
[0016] ;
[0017] in, Indicates that the first step has been completed. The mask representation matrix constructed after each client; This represents the first weighted coefficient matrix; This represents the second weighted coefficient matrix; Indicates that the first step has been completed. The mask representation matrix constructed after each client;
[0018] In all After all clients have been traversed, the final mask representation matrix is obtained. .
[0019] Furthermore, the global classifier parameters are calculated based on the aggregated autocorrelation matrix and the mask representation matrix, specifically as follows:
[0020] ;
[0021] in, Indicates the parameters of the global classifier; Represents the regularization coefficient; Represents the identity matrix; This represents the final mask representation matrix.
[0022] Secondly, this invention proposes a heterogeneous parsing federated learning method based on a dual-anchor mechanism, applied to the client side, comprising the following steps:
[0023] Receive the global backbone network and random projection matrix distributed by the server;
[0024] Using local private data, initial features are extracted through a global backbone network, and then the initial features are subjected to projection adaptive processing using a random projection matrix and a nonlinear activation function to obtain enhanced features;
[0025] Based on the enhanced features and the corresponding local labels, the local classifier parameters are calculated through parsing learning; based on the enhanced features, the local autocorrelation matrix is calculated through matrix autocorrelation operation and superimposed regularization terms.
[0026] Upload the local classifier parameters and local autocorrelation matrix to the server;
[0027] Receive the global classifier parameters broadcast by the server; construct a dual-anchor regularization term by using the global classifier parameters as global anchors and the local classifier parameters as local anchors; wherein, the dual-anchor regularization term is used to constrain the distances between the individual classifier parameters to be solved and the global anchors and the local anchors respectively;
[0028] The local data is optimized using a dual-anchor regularization term to calculate personalized classifier parameters, which are then used to update the local model.
[0029] Furthermore, using local private data, initial features are extracted through a global backbone network, and then the initial features are subjected to adaptive projection processing using a random projection matrix and a nonlinear activation function to obtain enhanced features, specifically:
[0030] Input local private data into the frozen global backbone network. Output initial features ; Indicates the frozen parameters of the global backbone network;
[0031] Combine the initial features with the random projection matrix Multiplication expands or transforms the feature dimensions;
[0032] The transformed features are input into a nonlinear activation function. Enhanced features are obtained Specifically:
[0033]
[0034] in, Represents the projection matrix; This represents local private data; This represents a non-linear activation function.
[0035] Furthermore, based on the enhanced features and corresponding local labels, the parameters of the local classifier are calculated through parsing learning, specifically:
[0036] ;
[0037] ;
[0038] in, Indicates enhanced features; Indicates a local tag; Represents the regularization coefficient; Represents the identity matrix; Indicates the first Autocorrelation matrix of each client.
[0039] Furthermore, the double-anchor regularization term is expressed as:
[0040] ;
[0041] in, This represents the parameters of the personalized classifier to be solved; Indicates the global anchor point; Denotes the Frobenius norm;
[0042] The personalized classifier parameters are calculated by optimizing the solution on local data based on the dual-anchor regularization term, specifically:
[0043] ;
[0044] in, Indicates the parameters of the personalized classifier; This represents the regularization strength coefficient; Indicates a uniquely encoded tag; This represents the regularization coefficient.
[0045] Further updates to the local model include:
[0046] Maintain the frozen parameters of the global backbone network Keep the personalized classifier parameters unchanged. Deploy a classifier locally for predicting or classifying future local data.
[0047] Furthermore, the global backbone network is a deep neural network model pre-trained on a public dataset, and its parameters are kept frozen during the federated learning training process;
[0048] The random projection matrix is distributed by the server to all clients and remains the same and fixed in each client, used to map the features extracted by different clients to the same common latent space.
[0049] Thirdly, this invention proposes a heterogeneous parsing federated learning system based on a dual-anchor mechanism, comprising at least one server and at least one client.
[0050] The server is used to execute the aforementioned server-side federated learning method;
[0051] The client is used to execute the aforementioned client-side federated learning method.
[0052] The effects described in the invention are merely those of the embodiments, and not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects:
[0053] This invention proposes a heterogeneous analytical federated learning method and system based on a dual-anchor mechanism, belonging to the field of artificial intelligence and distributed computing technology. When configured on a server, the method: distributes a global backbone network and random projection matrices to multiple clients; receives local classifier parameters and local autocorrelation matrices uploaded by each client; aggregates all local autocorrelation matrices to obtain an aggregated autocorrelation matrix, and based on the local classifier parameters uploaded by all clients, iteratively integrates the information from each client to construct a mask representation matrix, and then calculates the global classifier parameters based on the aggregated autocorrelation matrix and the mask representation matrix; broadcasts the global classifier parameters to all clients, enabling clients to optimize their local models based on the global classifier parameters. When configured on the client: The system receives the global backbone network and random projection matrix distributed by the server; using local private data, it extracts initial features through the global backbone network, and then performs adaptive projection processing on the initial features using the random projection matrix and a nonlinear activation function to obtain enhanced features; based on the enhanced features and corresponding local labels, it calculates local classifier parameters through parsing learning; based on the enhanced features, it calculates the local autocorrelation matrix through matrix autocorrelation operations and superimposed regularization terms; it uploads the local classifier parameters and the local autocorrelation matrix to the server; it receives the global classifier parameters broadcast by the server; it constructs a dual-anchor regularization term using the global classifier parameters as global anchors and the local classifier parameters as local anchors; wherein, the dual-anchor regularization term is used to constrain the distances between the personalized classifier parameters to be solved and the global anchors and local anchors, respectively; based on the dual-anchor regularization term, it performs optimization on local data to calculate personalized classifier parameters, which are used to update the local model. Based on this method, a corresponding system is also proposed. This invention significantly reduces the feature correlation between different categories by using a projection adaptive mechanism, random projection and nonlinear transformation, and enhances the linear separability of the frozen backbone network output features. It can effectively extract the specific features of each client data in highly heterogeneous scenarios.
[0054] This invention employs a dual-anchor mechanism, which effectively solves the parameter drift problem in analytical federated learning. On the one hand, it utilizes global anchors to ensure the model's generalization ability; on the other hand, it uses local anchors to preserve personalized features, exhibiting excellent robustness in non-independent and identically distributed (Non-IID) data scenarios.
[0055] This invention is entirely based on analytical solutions, eliminating the need for gradient descent iterations and avoiding the vanishing or exploding gradient problems. It converges with only a minimal number of communication rounds, making it particularly suitable for scenarios with limited network conditions, such as mobile edge computing. Experiments on multiple public datasets demonstrate that the accuracy of this invention significantly outperforms existing analytical federated learning and mainstream gradient basis federated learning methods, with even more pronounced performance improvements in heavily heterogeneous scenarios. Attached Figure Description
[0056] Figure 1 This is a flowchart illustrating the application of a heterogeneous analytical federated learning method based on a dual-anchor mechanism proposed in Embodiment 1 of the present invention to a server.
[0057] Figure 2 This is a flowchart illustrating the application of a heterogeneous analytical federated learning method based on a dual-anchor mechanism proposed in Embodiment 1 of the present invention to a client.
[0058] Figure 3 This is a flowchart of the server-client interaction according to Embodiment 1 of the present invention;
[0059] Figure 4 This is a diagram illustrating the overall framework of a heterogeneous analytical federated learning method based on a dual-anchor mechanism proposed in Embodiment 1 of the present invention.
[0060] Figure 5 This is a schematic diagram illustrating the principle of the dual-anchor point mechanism proposed in Embodiment 1 of the present invention;
[0061] Figure 6 This is a diagram showing the feature extraction effect of the original backbone in Embodiment 1 of the present invention;
[0062] Figure 7 This is a schematic diagram illustrating the improvement effect of the projection adaptive mechanism proposed in Embodiment 1 of the present invention on feature separability;
[0063] Figure 8 This is a schematic diagram of a heterogeneous analytical federated learning system based on a dual-anchor mechanism proposed in Embodiment 2 of the present invention. Detailed Implementation
[0064] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the invention. To simplify the disclosure of the invention, components and arrangements of specific examples are described below. Furthermore, reference numerals and / or letters may be repeated in different examples. This repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. It should be noted that the components illustrated in the drawings are not necessarily drawn to scale. Descriptions of well-known components, processing techniques, and processes are omitted in this invention to avoid unnecessarily limiting the invention.
[0065] Example 1
[0066] Embodiment 1 of this invention proposes a heterogeneous parsing federated learning method based on a dual-anchor mechanism. By introducing a projection adaptation mechanism and a dual-anchor mechanism, it breaks the rigidity of features and updates while retaining the efficiency of parsing federated learning. This method mainly consists of two parts: a server-side component and a client-side component.
[0067] Figure 1 This is a flowchart illustrating the application of a heterogeneous analytical federated learning method based on a dual-anchor mechanism proposed in Embodiment 1 of the present invention to a server.
[0068] S11: Distribute the global backbone network and random projection matrix;
[0069] The server distributes a global backbone network and random projection matrix to multiple clients for local feature extraction and projection adaptation processing.
[0070] S12: Receive the local classifier parameters and autocorrelation matrix uploaded by the client;
[0071] The server receives local classifier parameters and local autocorrelation matrices uploaded by each client. These local classifier parameters and local autocorrelation matrices are calculated by the client using different methods based on the enhanced features obtained after projective adaptive processing of local private data.
[0072] The local classifier parameters are calculated through parsing learning based on the enhanced features and local labels; the local autocorrelation matrix is obtained by performing matrix autocorrelation operations on the enhanced features and adding regularization terms.
[0073] S13: Aggregate all autocorrelation matrices;
[0074] The server aggregates the local autocorrelation matrices uploaded by all clients to obtain the aggregated autocorrelation matrix, which is as follows: ;
[0075] in, Represents the aggregated autocorrelation matrix; Indicates the number of clients; Indicates the first Autocorrelation matrix of each client.
[0076] S14: Initialize the mask representation matrix as the starting point for iteration;
[0077] The server sets an initial matrix as the starting point for subsequent iterations to construct the mask representation matrix; setting the initial matrix The initial matrix serves as the starting point for iteratively constructing the mask representation matrix, where the initial matrix is preferably the zero matrix.
[0078] S15: Set up the first client server to prepare for iterative processing starting from the first client;
[0079] Iterative processing phase (S16-1 to S16-4).
[0080] S16-1: Obtain current client parameters:
[0081] The server obtains the local classifier parameters and local autocorrelation matrix of the current client.
[0082] S16-2: Calculate the weighting coefficients:
[0083] Based on the local autocorrelation matrix and aggregated autocorrelation matrix of the current client, the server calculates the first weighted coefficient matrix and the second weighted coefficient matrix.
[0084] S16-3: Update the mask representation matrix:
[0085] Iterate through each client in order, for the first... Local classifier parameters for each client The mask representation matrix is iteratively updated using the following formula;
[0086] ;
[0087] in, Indicates that the first step has been completed. The mask representation matrix constructed after each client; This represents the first weighting coefficient matrix, used to control the degree of retention of historical integrated information; This represents the second weighting coefficient matrix, used to control the degree to which current client information is introduced; Indicates that the first step has been completed. The mask representation matrix constructed after each client.
[0088] The server uses a formula to update the mask representation matrix, integrating historical integration information with current client information.
[0089] S16-4: Next Client:
[0090] The server moves to the next client and continues iterative processing; it then checks if all clients have been processed. If there are still unprocessed clients, it returns to step S16-1 to continue iterating; if all clients have been processed, it proceeds to the next step.
[0091] S17-1: Obtain the final mask representation matrix;
[0092] Servers are all After all clients have been traversed, the final mask representation matrix is obtained. .
[0093] S17-2: Calculate the parameters of the global classifier;
[0094] The server calculates the global classifier parameters based on the aggregated autocorrelation matrix and the final mask representation matrix, specifically:
[0095] ;
[0096] in, Indicates the parameters of the global classifier; Represents the regularization coefficient; Represents the identity matrix; This represents the final mask representation matrix.
[0097] S17-3: Broadcast global parameters as anchor points;
[0098] The server broadcasts the calculated global classifier parameters to all clients, which serve as the global anchor point for subsequent personalized updates on the clients.
[0099] Through the above steps, the server completes the aggregation of local information from each client and the construction of a global model with only one round of communication, providing a global anchor point for subsequent personalized updates to the clients.
[0100] Figure 2 This is a flowchart illustrating the application of a heterogeneous analytical federated learning method based on a dual-anchor mechanism proposed in Embodiment 1 of the present invention to a client.
[0101] S21: Receive backbone network and projection matrix;
[0102] The client receives the global backbone network and random projection matrix distributed by the server. The global backbone network is a pre-trained deep neural network model whose parameters are frozen during the federated learning process; the random projection matrix is uniformly distributed by the server and remains the same and unchanged in all clients.
[0103] S22: Retrieve local data;
[0104] The client obtains local private data and corresponding local tags; the local private data exhibits heterogeneous characteristics of non-independent and identically distributed across different clients.
[0105] S23: Extracting initial features from the backbone network;
[0106] The client inputs local private data into the frozen global backbone network, and extracts initial features through the backbone network.
[0107] S24: Enhanced features are obtained based on random projection and nonlinear activation;
[0108] The client inputs local private data into the frozen global backbone network. Output initial features ; This represents the frozen parameters of the global backbone network; the initial features are then compared with the random projection matrix. Multiplication expands or transforms the feature dimension; the transformed features are then input into a nonlinear activation function. Enhanced features are obtained Specifically:
[0109]
[0110] in, Represents the projection matrix; This represents local private data; This represents a non-linear activation function.
[0111] S25: Calculate the parameters of the local classifier;
[0112] The client computes local classifier parameters through analytical learning based on enhanced features and corresponding local labels. Specifically, the local classifier parameters are obtained using a ridge regression analytical calculation formula.
[0113] S26: Calculate the local autocorrelation matrix;
[0114] The client calculates the local autocorrelation matrix based on enhanced features, through matrix autocorrelation operations and the addition of regularization terms; specifically:
[0115] ;
[0116] ;
[0117] in, Indicates enhanced features; Indicates a local tag; Represents the regularization coefficient; Represents the identity matrix; Indicates the first Autocorrelation matrix of each client.
[0118] S27: Upload parameters to the server;
[0119] The client uploads the calculated local classifier parameters and local autocorrelation matrix to the server for aggregation and global model construction.
[0120] S28: Receive global classifier parameters;
[0121] The client receives the global classifier parameters broadcast by the server, which serve as the global anchor for subsequent personalized updates.
[0122] S29: Global anchor point ← global parameters, local anchor point ← local parameters;
[0123] The client uses the received global classifier parameters as global anchors and the locally computed local classifier parameters as local anchors to prepare for the construction of the dual-anchor regularization term.
[0124] S30: Construct a dual-anchor regularization term;
[0125] The client constructs a dual-anchor regularization term to constrain the distances between the parameters of the personalized classifier to be solved and both the global and local anchors. This regularization term consists of two parts: the first part constrains the distance between the model parameters and the local anchor, preserving the beneficial parts from local updates; the second part constrains the distance between the model parameters and the global anchor, using global generalization capabilities to correct parameter drift caused by data heterogeneity.
[0126] The double-anchor regularization term is represented as:
[0127] ;
[0128] in, This represents the parameters of the personalized classifier to be solved; Indicates the global anchor point; Denotes the Frobenius norm; These are the local classifier parameters (and are also equal to the local anchor point).
[0129] S31: Optimization solution yields personalized parameters;
[0130] The client performs optimization on local data based on a dual-anchor regularization term to calculate the personalized classifier parameters. This optimization objective has a closed-form solution and can be directly calculated.
[0131] The personalized classifier parameters are calculated by optimizing the solution on local data based on the dual-anchor regularization term, specifically:
[0132] ;
[0133] in, Indicates the parameters of the personalized classifier; This represents the regularization strength coefficient; Indicates a uniquely encoded tag; This represents the regularization coefficient.
[0134] Its function is to control the strength of the basic ridge regression regularization term, to prevent the model from overfitting, and to penalize the complexity of the model parameter W. Its function is to control the strength of the dual-anchor regularization term, which is used to balance the constraint strength of local anchors and global anchors, that is, to control the trade-off between the degree of personalization and the ability to generalize.
[0135] The optimization objective has a closed-form solution:
[0136] ;
[0137] In this mechanism: Local anchors (implicit in the regression term): Preserve the beneficial parts of local updates and prevent the model from deviating excessively from the local data distribution. Global anchors. : Drag and drop the local model onto the global model to take advantage of global generalization capabilities and correct parameter drift caused by data heterogeneity.
[0138] S32: Update local classifier parameters;
[0139] The client keeps the global backbone network parameters frozen and only updates the local classifier parameters to the calculated personalized classifier parameters, thus updating the local model. The updated model can then be used for prediction or classification tasks on new local data.
[0140] Maintain the frozen parameters of the global backbone network Keep the personalized classifier parameters unchanged. Deploy a classifier locally for predicting or classifying future local data.
[0141] Through the above steps, after receiving the global backbone network and projection matrix distributed by the server, the client sequentially completes the projection adaptation processing, local parsing calculation, parameter uploading, global anchor point reception, dual anchor point regularization construction, personalized parameter solving, and local model update, thereby realizing personalized model training that integrates global knowledge and local characteristics while protecting local data privacy.
[0142] Figure 3 This is a flowchart of the server-client interaction according to Embodiment 1 of the present invention;
[0143] System initialization phase:
[0144] Server action (S11): The server distributes the global backbone network and random projection matrix to all clients;
[0145] Client actions: Each client receives the backbone network and projection matrix distributed by the server and prepares to perform local processing.
[0146] Client-side local processing stage:
[0147] S21-S24: Projection adaptive processing; S25-S26: Local analytical calculation.
[0148] Client action (S27: Upload local parameters); The client will upload the calculated local classifier parameters. and local autocorrelation matrix Uploaded to the server.
[0149] Server aggregation processing phase:
[0150] S13: Aggregate autocorrelation matrix;
[0151] Server actions (S14-S17: Iteratively construct the mask representation matrix and calculate global parameters)
[0152] Server action (S17-3: Broadcast global parameters): The server uses the calculated global classifier parameters as a global anchor point and broadcasts them to all clients.
[0153] Client personalization update phase:
[0154] Client-side actions (S28-S32: Dual-anchor point personalized update).
[0155] Complete the local model update.
[0156] Figure 4 This is a diagram illustrating the overall framework of a heterogeneous analytical federated learning method based on a dual-anchor mechanism proposed in Embodiment 1 of the present invention. It employs a classic centralized federated learning architecture, comprising one server and multiple clients (client 1, client 2... client K). The components and their interrelationships are described in detail below.
[0157] The left side of the diagram shows multiple clients participating in federated learning, including client 1, client 2... client K, representing multiple participants.
[0158] In a healthcare network platform, these clients correspond to different hospitals, each possessing local data (such as medical images and medical records). Due to the different specialties and service populations of each hospital, this local data exhibits significant heterogeneous characteristics with non-independent distribution. For example, specialized hospitals mainly treat patients with specific diseases, general hospitals have departments evenly distributed across different departments, and community hospitals primarily serve elderly patients with chronic diseases. Under privacy protection requirements, hospitals cannot directly share raw data and must complete model training and parameter calculation locally.
[0159] The scope of protection of this invention is not limited to the specific application scenarios listed in Example 1. Those skilled in the art can make reasonable selections and adjustments based on the actual situation.
[0160] Generated by the server and distributed to all clients, it remains identical and fixed across all clients. Its purpose is to map features extracted by different clients to the same common latent space, ensuring uniform feature dimensions and facilitating subsequent aggregation by the server.
[0161] After receiving the random projection matrix, the client processes the local data as follows: extracts initial features through the global backbone network, multiplies the initial features by the random projection matrix to expand the dimensions, and processes them through a nonlinear activation function to obtain enhanced features.
[0162] This mechanism effectively breaks feature rigidity, enabling clients to extract specific features adapted to local data distribution even when the backbone network is frozen, thereby enhancing the linear separability of features.
[0163] Figure 5 This is a schematic diagram illustrating the principle of the dual-anchor mechanism proposed in Embodiment 1 of the present invention; the mechanism includes two anchor points:
[0164] Local anchor: Classifier parameters calculated locally on the client side, retaining the specific features of local data; Global anchor: Global classifier parameters broadcast by the server, containing global generalization knowledge.
[0165] The client constructs a dual-anchor regularization term, constraining the parameters of the personalized classifier to move closer to two anchors simultaneously: "dragging" towards the local anchor preserves useful local information and prevents the model from deviating excessively from the local distribution; "dragging" towards the global anchor utilizes global knowledge to correct parameter drift caused by data heterogeneity. This mechanism effectively eliminates update rigidity and achieves a balance between personalization and generalization.
[0166] Anchor-free update: Presented as a contrasting term in the diagram, this represents the update method of traditional federated learning. This type of method relies solely on the global model for updates, making it unsuitable for heterogeneous data and resulting in significant performance degradation in scenarios with non-independent and identically distributed data. By juxtaposing it with the dual-anchor mechanism, the technical advantages of this invention are highlighted.
[0167] The server is responsible for task initialization, parameter aggregation, and global model distribution. It never touches any client's raw data and only receives preprocessed model parameters. Figure 4 The text demonstrates two key computations performed by the server:
[0168] This indicates that the server receives and processes the local autocorrelation matrix uploaded by the client. : Represents the global classifier parameters obtained after server aggregation calculation, which are used as global anchors for subsequent distribution to clients.
[0169] This invention enables the parsing of heterogeneous data and optimization of training anchors for distributed models in a federated environment, thereby achieving global generalization of the model, overcoming the performance limitations of distributed storage and privacy protection, and obtaining a better federated learning model.
[0170] To fully illustrate the implementation process of the heterogeneous parsing federated learning method based on the dual-anchor mechanism proposed in Embodiment 1 of this invention, a specific example is used: taking the image classification task in a federated environment as an example, the computational steps and final effect of this technology in processing distributed storage privacy heterogeneous data (Non-IID) for model training are specifically demonstrated.
[0171] 1. Basic Scenarios and Input
[0172] The federated learning system consists of 100 clients and is trained using the CIFAR-10 dataset. To simulate real-world heterogeneous scenarios, a latent Dirichlet partitioning strategy is employed, resulting in a highly uneven distribution of image categories across different clients. For example, client A primarily possesses "animal" images, while client B primarily possesses "truck" images.
[0173] Input data: per client Local input batch image data, with a single image size of [size missing]. .
[0174] Pre-trained backbone: ResNet pre-trained on ImageNet is used as the feature extractor. Its parameters The entire process is frozen, and the original output feature dimension is 512. Taking stray dogs and trucks in surveillance footage as an example... Figure 6 This is a diagram showing the feature extraction effect of the original backbone in Embodiment 1 of the present invention.
[0175] 2. Projection Adaptive Feature Processing
[0176] For the client The input image is first subjected to feature extraction and projection transformation.
[0177] Processing procedure: The system initializes an aggregate mask representation matrix, whose dimensions are set to... This matrix is shared and fixed across all clients.
[0178] Calculation steps:
[0179] Input image Inputting the backbone network yields the original feature vector, with dimension [missing information]. ; Combine the original features with the projection matrix Multiplication expands the dimension; non-linear activation functions are used. Processing; finally, the enhanced feature matrix is obtained. The calculation formula is: At this point, the feature dimension is mapped to .
[0180] Technical effects: Figure 7 This is a schematic diagram illustrating the improvement effect of the projection adaptive mechanism proposed in Embodiment 1 of the present invention on feature separability; by introducing and This makes the generated features more linearly separable in the feature space even if the backbone network is frozen.
[0181] 3. Local resolution and correlation calculation
[0182] The client utilizes the enhanced feature matrix and corresponding one-hot encoded tags Perform analytical calculations.
[0183] Server-side mask aggregation generates global anchor points
[0184] The server receives upload parameters from each client. After generating anchor points, it distributes them to each client.
[0185] Personalized output based on dual anchor points
[0186] The client received Then, the final training of the federal environment model is performed.
[0187] Based on the above illustration, it can be shown that the heterogeneous analytical federated learning method based on the dual-anchor mechanism proposed in Embodiment 1 of the present invention significantly reduces the feature correlation between different categories by using a projection adaptive mechanism, random projection and nonlinear transformation, and enhances the linear separability of the frozen backbone network output features. It can effectively extract the specific features of each client data in highly heterogeneous scenarios.
[0188] Table 1 below shows a comparison of the accuracy of the present invention with existing methods (FedAvg, FedProx, AFL, etc.).
[0189] Table 1: Accuracy Comparison of the Invention with Existing Methods (FedAvg, FedProx, AFL, etc.):
[0190]
[0191] pass Figure 1 As can be seen, under various heterogeneous settings of the CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets, the accuracy of this invention is significantly better than existing AFL and mainstream gradient basis federated learning methods, with a significant performance improvement.
[0192] Example 2
[0193] Based on the heterogeneous analytical federated learning method based on the dual-anchor mechanism proposed in Embodiment 1 of this invention, Embodiment 2 of this invention also proposes a heterogeneous analytical federated learning system based on the dual-anchor mechanism. Figure 8 This is a schematic diagram of a heterogeneous parsing federated learning system based on a dual-anchor mechanism proposed in Embodiment 2 of the present invention. The system includes a server and multiple clients.
[0194] The server is used to execute the aforementioned server-side federated learning method; that is... Figure 1 The process shown;
[0195] S11: Distribute the global backbone network and random projection matrix;
[0196] S12: Receive the local classifier parameters and autocorrelation matrix uploaded by the client;
[0197] S13: Aggregate all autocorrelation matrices;
[0198] S14: Initialize the mask representation matrix as the starting point for iteration;
[0199] S15: Set up the first client server to prepare for iterative processing starting from the first client;
[0200] S16-1: Obtain current client parameters:
[0201] S16-2: Calculate the weighting coefficients:
[0202] S16-3: Update the mask representation matrix:
[0203] S16-4: Next Client:
[0204] S17-1: Obtain the final mask representation matrix;
[0205] S17-2: Calculate the parameters of the global classifier;
[0206] S17-3: Broadcast global parameters as anchor points;
[0207] The client is used to execute the aforementioned client-side federated learning method, i.e. Figure 2 The process described above.
[0208] S21: Receive backbone network and projection matrix;
[0209] S22: Retrieve local data;
[0210] S23: Extracting initial features from the backbone network;
[0211] S24: Enhanced features are obtained based on random projection and nonlinear activation;
[0212] S25: Calculate the parameters of the local classifier;
[0213] S26: Calculate the local autocorrelation matrix;
[0214] S27: Upload parameters to the server;
[0215] S28: Receive global classifier parameters;
[0216] S29: Use the received global classifier parameters as the global anchor point and the locally calculated local classifier parameters as the local anchor point;
[0217] S30: Construct a dual-anchor regularization term;
[0218] S31: Optimization solution yields personalized parameters;
[0219] S32: Update local classifier parameters;
[0220] Embodiment 2 of this invention provides a heterogeneous parsing federated learning system based on a dual-anchor mechanism. Through a projection adaptive mechanism, random projection and nonlinear transformation are used to significantly reduce the feature correlation between different categories, enhance the linear separability of the frozen backbone network output features, and effectively extract the specific features of each client data in highly heterogeneous scenarios.
[0221] For a description of the relevant part of the heterogeneous parsing federated learning system based on the dual-anchor mechanism provided in Embodiment 2 of this application, please refer to the detailed description of the corresponding part of the heterogeneous parsing federated learning system based on the dual-anchor mechanism provided in Embodiment 1 of this application, and will not be repeated here.
[0222] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that the elements inherent in a process, method, article, or apparatus that includes a list of elements are included. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Additionally, portions of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.
[0223] While specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art can make other modifications or variations based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.
Claims
1. A heterogeneous resolution federated learning method based on a dual anchor point mechanism, applied to a server, characterized in that, Includes the following steps: Distribute the global backbone network and random projection matrix to multiple clients; The system receives local classifier parameters and local autocorrelation matrices uploaded by each client. These parameters and matrices are calculated by the client using different methods after performing projection adaptive processing on local private data to obtain enhanced features: the local classifier parameters are calculated through parsing learning based on the enhanced features and local labels; the local autocorrelation matrix is obtained by performing matrix autocorrelation operations on the enhanced features and adding a regularization term. The enhanced features are obtained by: extracting initial features using local private data through a global backbone network, and then performing projection adaptive processing on the initial features using a random projection matrix and a nonlinear activation function. Specifically, the local private data is input into a frozen global backbone network. Output initial features ; Indicates the frozen parameters of the global backbone network; Combine the initial features with the random projection matrix Multiplication expands or transforms the feature dimensions; The transformed features are input into a nonlinear activation function. Enhanced features are obtained Specifically: in, Represents the projection matrix; This represents local private data; Represents a nonlinear activation function; The aggregated autocorrelation matrix is obtained by aggregating all local autocorrelation matrices. Based on the local classifier parameters uploaded by all clients, the mask representation matrix is constructed by iteratively integrating the information of each client. Then, the global classifier parameters are calculated based on the aggregated autocorrelation matrix and the mask representation matrix. Broadcast the global classifier parameters to all clients, enabling clients to optimize their local models based on the global classifier parameters.
2. The heterogeneous analytical federated learning method based on a dual-anchor mechanism according to claim 1, characterized in that, The aggregated autocorrelation matrix is obtained by aggregating all local autocorrelation matrices. Then, based on the local classifier parameters uploaded by all clients, a mask representation matrix is constructed by iteratively integrating information from each client. Specifically: ； in, Represents the aggregated autocorrelation matrix; Indicates the number of clients; Indicates the first Autocorrelation matrix of each client; Set the initial matrix This serves as the starting point for iteratively constructing the mask representation matrix; it iterates through each client in a preset order, for the... Local classifier parameters for each client The mask representation matrix is iteratively updated using the following formula; ； in, Indicates that the first step has been completed. The mask representation matrix constructed after each client; This represents the first weighted coefficient matrix; This represents the second weighted coefficient matrix; Indicates that the first step has been completed. The mask representation matrix constructed after each client; In all After all clients have been traversed, the final mask representation matrix is obtained. .
3. The heterogeneous analytical federated learning method based on a dual-anchor mechanism according to claim 2, characterized in that, The global classifier parameters are calculated based on the aggregated autocorrelation matrix and the mask representation matrix, specifically as follows: ； in, Indicates the parameters of the global classifier; Represents the regularization coefficient; Represents the identity matrix; This represents the final mask representation matrix.
4. A heterogeneous analytical federated learning method based on a dual-anchor mechanism, applied to a client, characterized in that, Includes the following steps: Receive the global backbone network and random projection matrix distributed by the server; Using local private data, initial features are extracted through a global backbone network, and then the initial features are subjected to projection adaptive processing using a random projection matrix and a nonlinear activation function to obtain enhanced features; Specifically: Input local private data into the frozen global backbone network. Output initial features ; Indicates the frozen parameters of the global backbone network; Combine the initial features with the random projection matrix Multiplication expands or transforms the feature dimensions; The transformed features are input into a nonlinear activation function. Enhanced features are obtained Specifically: in, Represents the projection matrix; This represents local private data; Represents a nonlinear activation function; Based on the enhanced features and the corresponding local labels, the local classifier parameters are calculated through parsing learning; based on the enhanced features, the local autocorrelation matrix is calculated through matrix autocorrelation operation and superimposed regularization terms. Upload the local classifier parameters and local autocorrelation matrix to the server; Receive the global classifier parameters broadcast by the server; construct a dual-anchor regularization term by using the global classifier parameters as global anchors and the local classifier parameters as local anchors; wherein, the dual-anchor regularization term is used to constrain the distances between the individual classifier parameters to be solved and the global anchors and the local anchors respectively; The local data is optimized using a dual-anchor regularization term to calculate personalized classifier parameters, which are then used to update the local model.
5. The heterogeneous analytical federated learning method based on a dual-anchor mechanism according to claim 4, characterized in that, Based on the enhanced features and their corresponding local labels, the parameters of the local classifier are calculated through parsing learning, specifically: ； ； in, Indicates enhanced features; Indicates a local tag; Represents the regularization coefficient; Represents the identity matrix; Indicates the first Autocorrelation matrix of each client.
6. The heterogeneous analytical federated learning method based on a dual-anchor mechanism according to claim 5, characterized in that, The double-anchor regularization term is represented as: ； in, This represents the parameters of the personalized classifier to be solved; Indicates the global anchor point; Denotes the Frobenius norm; The personalized classifier parameters are calculated by optimizing the solution on local data based on the dual-anchor regularization term, as follows: ； in, Indicates the parameters of the personalized classifier; This represents the regularization strength coefficient; Indicates a uniquely encoded tag; This represents the regularization coefficient.
7. The heterogeneous analytical federated learning method based on a dual-anchor mechanism according to claim 6, characterized in that, Updating the local model includes: Maintain the frozen parameters of the global backbone network Keep the personalized classifier parameters unchanged. Deploy a classifier locally for predicting or classifying future local data.
8. The heterogeneous analytical federated learning method based on a dual-anchor mechanism according to claim 1 or 4, characterized in that, The global backbone network is a deep neural network model pre-trained on a public dataset, and its parameters are kept frozen during the federated learning training process; The random projection matrix is distributed by the server to all clients and remains the same and fixed in each client, used to map the features extracted by different clients to the same common latent space.
9. A heterogeneous analytical federated learning system based on a dual-anchor mechanism, characterized in that, Includes at least one server and at least one client; The server is configured to perform the method according to any one of claims 1 to 3; The client is configured to perform the method of any one of claims 4 to 8.