Clustering-based heterogeneous federal basic model adaptive fine tuning method and computer device

Abstract

The invention relates to the field of heterogeneous federal basic model adjustment, in particular to a clustering-based heterogeneous federal basic model adaptive fine adjustment method and a computer device. According to the scheme, a representative client is selected for each cluster through a multi-factor heterogeneous sensing clustering module, and the representative client selects a corresponding model as a cluster model according to own computing power limitation; searching an optimal sub-model based on the cluster model for all the clients in each cluster through a knowledge perception model architecture search algorithm, and deploying the optimal sub-model at the client; the method comprises the following steps of: uploading parameters to representative clients, aggregating corresponding parameters on the representative clients, issuing the aggregated parameters to clients in clusters, transmitting knowledge of each cluster to a server model through a knowledge transfer module perceived by the clusters, and returning the knowledge of the server model and updating the representative clients of each cluster through reverse knowledge distillation. The method is suitable for self-adaptive fine tuning of the heterogeneous federal basic model.

Description

Technical Field

[0001] The present invention relates to the field of heterogeneous federated basic model adjustment, and in particular to a clustering-based heterogeneous federated basic model adaptive fine-tuning method and a computer device. Background Art

[0002] The large model is pre-trained on a large number of data sets, making it suitable for a wide range of application scenarios and having a wide range of generalization. Therefore, by fine-tuning the large model on a specific data set, the large model can be adapted to various specific tasks. However, in reality, fine-tuning of large models faces many problems, especially the leakage of private data. Most of the data is distributed locally and is private, which limits the data range of large model fine-tuning. In the cloud-edge collaborative training scenario, federated learning has emerged as a promising method. It can realize the collaborative training of models between multiple clients without directly exchanging private data. However, due to the increasing number of parameters of large models, which usually have billions of parameters or even larger, many edge clients cannot deploy or fine-tune these large models. However, traditional federation, such as fedavg, requires the client and server to share the same model for parameter aggregation, so traditional federation is not applicable. How to optimize the federation of heterogeneous models has become a top priority.

[0003] Current research has proposed several strategies to achieve heterogeneous federation, including methods based on knowledge distillation and partial training. For example, knowledge distillation-based methods such as FedDF and DS-FL implement the deployment of small models on the client and large models on the server, and transfer logits (maximum likelihood estimates). Logits (logical values) usually refer to the original unprocessed scores or scores of the output layer of the model, and use the small model on the client as a teacher to guide the training of the model on the server, thereby achieving heterogeneous model federation.

[0004] However, these methods require each client to interact with the server. The knowledge distillation method is very time-consuming. In addition, for clients with limited computing resources, they may not be able to find a suitable smaller version of the FM (Foundation model) to deploy, or there may not be enough versions of the FM to match the highly heterogeneous client resource constraints, resulting in resource waste. Methods based on partial training, such as HeteroFL and FedRolex, deploy sub-models of the server model on the client and upload the corresponding parameters to the server for aggregation, thereby achieving heterogeneous model federation. However, HeteroFL can only fine-tune the first part of the parameters of each layer of the model, and cannot fine-tune the entire model. Although FedRolex adopts the method of rolling extraction of sub-models so that all parameters can be fine-tuned, it can only be an irregular rolling extraction and cannot highlight the importance of certain layers.

[0005] Therefore, the existing methods have the following problems:

[0006] 1.FM deployment issues;

[0007] 2. Collaborative training of FMs brings huge computational and communication overheads. In federated learning, frequent exchanges of large model parameters or gradients will lead to significant communication and computational overheads, especially in network connections with limited bandwidth;

[0008] 3. Heterogeneous data and resource distribution lead to unbalanced training, slow convergence and poor performance. FMs puts higher requirements on the quality of training data, so the actual data and resource heterogeneity will have a serious impact. Summary of the invention

[0009] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a clustering-based heterogeneous federated basic model adaptive fine-tuning method and computer device, which effectively solves the problem of fine-tuning FMs in highly resource heterogeneous scenarios, so that the model deployed on each client meets its resource constraints and greatly reduces the computing overhead and communication overhead.

[0010] The present invention adopts the following technical solutions to achieve the above-mentioned purpose. In a first aspect, the present invention provides a clustering-based heterogeneous federated basic model adaptive fine-tuning method, comprising:

[0011] S1. Through the multi-factor heterogeneous perception clustering module, the computing resource limitations and data distribution of each client are comprehensively considered for clustering, and a representative client is selected for each cluster. The representative client will select a corresponding model as the cluster model according to its own computing power limitations;

[0012] S2. Through the knowledge-aware model architecture search algorithm, according to the heterogeneous computing power limit of each client, the optimal sub-model based on the cluster model is searched for all clients in each cluster, and the optimal sub-model is deployed on the client;

[0013] S3, perform local training on the clients in each cluster, then upload the parameters to the representative client, aggregate the corresponding parameters on the representative client, and send them to the clients in the cluster after aggregation, repeating step S3 until the training in the cluster is completed;

[0014] S4, through the cluster-aware knowledge transfer module, the knowledge of each cluster is transferred to the server model to realize the training of the server model;

[0015] S5. Through reverse knowledge distillation, the knowledge of the server model is transmitted back and the representative client of each cluster is updated. The updated representative client sends the corresponding parameters to each client in the cluster.

[0016] Furthermore, step S1 specifically includes:

[0017] The multi-factor heterogeneous perception clustering module uses the K-means algorithm to comprehensively consider the computing resource limitations and data distribution of each client, and divides clients with similar data distribution and computing resource limitations into one cluster;

[0018] When clustering, we use the differential privacy method to add Gaussian noise to the data of each client. For client i, its data distribution is P(D i ), after adding Gaussian noise, the data characteristics are:

[0019] Δ f represents the sensitivity of the function, ∈ represents a parameter that measures the strength of privacy protection;

[0020] For each cluster, according to the computing power limit of each client in the cluster, the client with the largest computing power is used as the representative client of this cluster. The representative client selects the corresponding basic model according to its own computing power limit, and the selected basic model is deployed on the representative client as the cluster model.

[0021] Furthermore, step S2 specifically includes:

[0022] According to different computing power limitations, the knowledge-aware model architecture search algorithm is used to search for the optimal sub-model of the cluster model for the client with insufficient computing power and deploy it on the client;

[0023] The knowledge-aware model architecture search algorithm is a deep pruning algorithm based on a genetic algorithm. It prunes the entire transformerblocks and calculates the fitness through two metrics, one of which is the NASWOT metric, which is calculated as follows:

[0024] S = log|K|, N A is the unit of the activation function, d H represents the Hamming distance, K represents the kernel matrix, and S represents the NASWOT metric;

[0025] The other is the KL divergence measure, which is calculated as follows:

[0026]

[0027] Where p is the logits of the original model, q is the logits of the sub-model, T is an adjustable hyperparameter to control the influence between logits, and d is a measure of KL divergence;

[0028] Then the fitness F, F = Sd;

[0029] The specific search of the knowledge-aware model architecture search algorithm includes:

[0030] Step 1: Generate the structure of multiple sub-models;

[0031] Step 2: Randomly select two sub-model structures and calculate the fitness of the two sub-model structures. If the fitness of the first sub-model structure is greater than the fitness of the second sub-model structure, the first sub-model structure is the winner and the second sub-model structure is the loser. Otherwise, the second sub-model structure is the winner and the first sub-model structure is the loser.

[0032] Step 3: Generate a random number. If it is less than the crossover rate, cross the sub-model structures corresponding to the winner and the loser to obtain a new structure. If the random number is less than the mutation rate, flip the sub-model structure corresponding to the loser to obtain a new structure.

[0033] Step 4: Calculate the fitness of the new structure. If the fitness of the new structure is greater than the fitness of the sub-model structure corresponding to the loser, replace the sub-model structure corresponding to the loser with the new structure.

[0034] Step5: Repeat Step1 to Step4 until the cycle is complete.

[0035] Furthermore, step S3 specifically includes:

[0036] S301, fine-tuning is performed using private data of the client, and only parameters are saved after fine-tuning;

[0037] S302, uploading the parameters of each client to the representative client in the corresponding cluster, and then performing parameter aggregation;

[0038] S303: Repeat steps S301 to S302 until the intra-cluster training is completed.

[0039] Furthermore, step S4 specifically includes:

[0040] S401. Knowledge is transferred through the representative clients in each cluster. The weight calculation formula of each representative client is as follows:

[0041] Among them, ω m represents the weight of each representative client, M is the number of clusters, N k represents the amount of data on the client side, x i is the original data, y i For labels;

[0042] S402: Use the unlabeled public data set for knowledge distillation, use the representative client in each cluster as the teacher model, generate pseudo labels for the unlabeled data, and calculate the cross-entropy loss through the pseudo labels and the predicted values ​​of the server model. The calculation formula is as follows:

[0043]

[0044] in, represents unlabeled data, yes After θ leader(m) Generated pseudo labels;

[0045] S403, the logits calculated by the teacher model using the public data set are transferred to the server, and the KL divergence is calculated with the logits calculated by the server model. The calculation formula is as follows:

[0046] D KL is the representation of KL divergence, σ represents the activation function;

[0047] S404. Combine cross-entropy loss and KL loss to obtain the total Loss. By minimizing the Loss, the server model is optimized to achieve fine-tuning of the server model. The total Loss calculation formula is as follows:

[0048] α represents a hyperparameter that controls the ratio of cross-entropy loss and KLloss.

[0049] Furthermore, step S4 specifically includes:

[0050] S501, after optimizing the server model, the representative model of each cluster is updated by using the knowledge distillation method through the unlabeled public data set, and the representative model also saves the parameters;

[0051] S502, after the representative model of each cluster is updated, the parameters are issued according to the sub-model structure of each client in the cluster, and the parameters corresponding to the sub-model structure are issued to update the model of each client;

[0052] S503, executing step S1 until the fine-tuning is completed.

[0053] In a second aspect, the present invention provides a computer device, including a memory, wherein the memory stores program instructions, and when the program instructions are executed, the above-mentioned clustering-based heterogeneous federated basic model adaptive fine-tuning method is executed.

[0054] The beneficial effects of the present invention are:

[0055] The present invention utilizes the partial training (PT) method and the knowledge distillation (KD) method to effectively solve the problem of fine-tuning FMs in highly resource heterogeneous scenarios, so that the model deployed on each client meets its resource constraints and significantly reduces the computational overhead and communication overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flow chart of a clustering-based adaptive fine-tuning method for a heterogeneous federated basic model provided by an embodiment of the present invention;

[0057] Figure 2 It is a schematic diagram of aggregation of sub-model parameters within a cluster provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0059] The present invention provides a clustering-based adaptive fine-tuning method for heterogeneous federated basic models, such as Figure 1 As shown, specifically including:

[0060] S1. Clustering is performed by comprehensively considering the computing resource limitations and data distribution of each client through the MHAC (Multi-Factor Heterogeneous Aware Clustering) module. A representative client (leader node) is selected for each cluster. The representative client will select a suitable model called cluster model based on its own computing power limitations.

[0061] S2. Through the KAMAS (Knowledge-Aware Model Architecture Search) algorithm, according to the heterogeneous computing power limitations of each client, the optimal sub-model based on the cluster model is searched for all clients in each cluster, and the optimal sub-model is deployed on the client.

[0062] S3. After local training (based on efficient fine-tuning of LoRa), each client in the cluster will upload the LoRa parameters to the representative client (leader node), aggregate the corresponding parameters on the representative client, and then send them to the clients in the cluster. Repeat the above steps until the training in the cluster is completed.

[0063] S4. Through the CAKT (Cluster-Aware Knowledge Transfer) module, the knowledge of each cluster is transferred to the server model to realize the training of the server model.

[0064] S5. Through reverse knowledge distillation, the knowledge of the server model is transmitted back and updated to the representative client (leader node) of each cluster. The updated leader node sends the corresponding lora parameters to each client in the cluster.

[0065] Specifically, step S1 specifically includes:

[0066] S101: The MHAC module uses the K-means algorithm for clustering, taking into account the computing power and data distribution on the clients. In order to protect the privacy of the data, we use the differential privacy method to add Gaussian noise to the data of each client. For client i, there is a data distribution P(D i ), after adding noise, the data characteristics are Δ f represents the sensitivity of the function, ∈ represents a parameter that measures the strength of privacy protection; C(D i ) represents the computing power limit of the client. The K-means algorithm comprehensively considers M(D i ) and C(Di ), clients with similar data distribution and computing power limitations are divided into a cluster.

[0067] S102: For each cluster, according to the computing power limit of each client in the cluster, the client with the largest computing power is positioned as the representative client (leader node) of the cluster. The representative client can select a suitable Foundation model (FM) according to its own computing power limit, such as clip-base, clip-large model, etc. The selected model will be deployed on the leader node as the cluster model.

[0068] Specifically, step S2 includes:

[0069] S201: In highly heterogeneous clients, even in the same cluster, the computing power limit of each client may be different. Therefore, not every client can deploy the same model (cluster model) as the representative client (leader node) in the cluster. Therefore, the present invention searches for the optimal sub model of the cluster model (optimal sub model of cluster model) for the client with insufficient computing power through the knowledge-aware model architecture search algorithm according to different computing power limits, and deploys it on the client.

[0070] S202: KAMAS algorithm is a deep pruning algorithm based on genetic algorithm, which will prune the transformer blocks (layers) completely. There are two measurement indicators, one is Neural Architecture Search without Training (NASWOT) score and the other is Kullback-Leibler (KL) Divergence score.

[0071] S = log|K|, which is the calculation method of NASWOT score, where N A is the units of the activation function, d H represents the Hamming distance, and the NASWOT score uses the initial activation pattern of the activation units in the untrained network to predict its final performance. The kernel matrix K is constructed by calculating the input data point (c1, c2, ..., c N ) is done by calculating the Hamming distance between the binary encodings of the activation states in the linear region of the network. The final NASWOT score S is derived from the logarithm of the absolute value of the determinant of K.

[0072] The other is the KL divergence measure, which is calculated as follows:

[0073]

[0074] Where p is the logits of the original model, q is the logits of the sub-model, and T is an adjustable hyperparameter to control the influence between logits. Finally, the fitness F can be calculated by considering the two indicators, F = sd.

[0075] The specific steps of the search algorithm are as follows:

[0076] Step 1: Generate multiple sub-model structures first (since the transformer-based model is composed of multiple transformer blocks stacked together, the sub-model structure is the expression of which transformer blocks are selected, for example [1,0,0,1,1,0,1...], 1 represents selection and 0 represents non-selection).

[0077] Step 2: Randomly select two sub-model structures A and B, calculate the fitness (Fitness), FA, FB, if FA>FB, then A is the winner and B is the loser, and vice versa.

[0078] Step 3: Generate a random number. If it is less than the crossover rate, the winner and loser structures are cross-calculated to obtain a new structure. If the random number is less than the mutation rate, the loser structure is flipped to obtain a new structure.

[0079] Step 4: Recalculate the fitness of the updated structure. If the new fitness is greater than the loser’s fitness, replace the loser mechanism in the population with the new structure.

[0080] Step5: Repeat Step1 to Step4 until the cycle is complete.

[0081] Specifically, step S3 specifically includes:

[0082] S301: The client uses its own private data for efficient fine-tuning (using Lora), and only saves the Lora parameters after fine-tuning.

[0083] S302: upload the lora parameters of each client to the representative client (leader node) in the cluster, and then complete the parameter aggregation here, such as Figure 2 and resend it to each client.

[0084] S303: Repeat steps S301 to S302 until the intra-cluster training is completed.

[0085] Specifically, step S4 specifically includes:

[0086] S401: Knowledge transfer is performed through the representative client (leader node) in each cluster. Since the capabilities provided by each representative client are different, in order to ensure that the server model obtains more accurate knowledge, the present invention uses the accurate design of the cluster-aware knowledge transfer module after training in each cluster. The weight calculation formula for each representative client is as follows:

[0087] Where M is the number of clusters, N k represents the amount of data on the client side, x i is the original data, y i For labels;

[0088] S402: Use the unlabeled public data set for knowledge distillation, use the representative client (leadernode) in each cluster as the teacher model, generate pseudo labels for the unlabeled data, and calculate the cross-entropy loss through the pseudo labels and the predicted values ​​of the server model. The calculation formula is as follows

[0089]

[0090] in, represents unlabeled data, yes After θ leader(m) Generated pseudo labels;

[0091] S403: The logits calculated by the teacher model using the public data set are transferred to the server, and the KL divergence is calculated with the logits calculated by the server model. The calculation formula is as follows:

[0092] D KL is the representation of KL divergence, σ represents the activation function;

[0093] S404: Combine cross-entropy loss and KL loss to get the total Loss. Finally, optimize the server model by minimizing the Loss, so as to achieve fine-tuning of the server model. The total Loss calculation formula is as follows:

[0094] α represents a hyperparameter that controls the ratio of cross-entropy loss and KLloss.

[0095] Specifically, step S5 includes:

[0096] S501: After updating the server model, the representative model (leader node) of each cluster is updated using the knowledge distillation method through the unlabeled public data set. At the same time, the model of the leader node also saves the lora parameters.

[0097] S502: After the leader node of each cluster is updated, the lora parameters are issued according to the sub-model structure of each client in the cluster, and the parameters of the corresponding sub-model structure are issued to update the model of each client.

[0098] S503: After all clients are updated, it means that a round of overall training is completed, and step S1 is restarted until the fine-tuning is completed.

[0099] Compared with traditional solutions, the performance of the present invention significantly exceeds existing solutions. In a large number of experiments, the present invention has achieved significant improvements. Specifically, in the cifar10, cifar100, and tiny-imagenet datasets, FedCAMS improves image classification accuracy by 3-10% compared to other baseline methods. At the same time, compared with the partial training based baseline method, the communication overhead is greatly reduced, and the communication overhead is almost negligible. By using the lora efficient fine-tuning method, compared with full fine-tuning, the trainable parameters are only about 1% of the original, and the computational overhead is also greatly reduced.

[0100] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. Clustering-based adaptive fine-tuning method for heterogeneous federated basic models, characterized by: include: S1. Through the multi-factor heterogeneous perception clustering module, the computing resource limitations and data distribution of each client are comprehensively considered for clustering, and a representative client is selected for each cluster. The representative client will select a corresponding model as the cluster model according to its own computing power limitations; S2. Through the knowledge-aware model architecture search algorithm, according to the heterogeneous computing power limit of each client, the optimal sub-model based on the cluster model is searched for all clients in each cluster, and the optimal sub-model is deployed on the client; S3, perform local training on the clients in each cluster, then upload the parameters to the representative client, aggregate the corresponding parameters on the representative client, and send them to the clients in the cluster after aggregation, repeating step S3 until the training in the cluster is completed; S4, through the cluster-aware knowledge transfer module, the knowledge of each cluster is transferred to the server model to realize the training of the server model; S5. Through reverse knowledge distillation, the knowledge of the server model is transmitted back and the representative client of each cluster is updated. The updated representative client sends the corresponding parameters to each client in the cluster.

2. The clustering-based adaptive fine-tuning method for heterogeneous federated basic models according to claim 1, characterized in that: Step S1 specifically includes: The multi-factor heterogeneous perception clustering module uses the K-means algorithm to comprehensively consider the computing resource limitations and data distribution of each client, and divides clients with similar data distribution and computing resource limitations into one cluster; When clustering, we use the differential privacy method to add Gaussian noise to the data of each client. For client i, its data distribution is P(D i ), after adding Gaussian noise, the data characteristics are: Δ f represents the sensitivity of the function, ∈ represents a parameter that measures the strength of privacy protection; For each cluster, according to the computing power limit of each client in the cluster, the client with the largest computing power is used as the representative client of this cluster. The representative client selects the corresponding basic model according to its own computing power limit, and the selected basic model is deployed on the representative client as the cluster model.

3. The clustering-based adaptive fine-tuning method for heterogeneous federated basic models according to claim 1, characterized in that: Step S2 specifically includes: According to different computing power limitations, the knowledge-aware model architecture search algorithm is used to search for the optimal sub-model of the cluster model for the client with insufficient computing power and deploy it on the client; The knowledge-aware model architecture search algorithm is a deep pruning algorithm based on a genetic algorithm. It prunes the entire transformerblocks and calculates the fitness through two metrics, one of which is the NASWOT metric, which is calculated as follows: S = log|K|, N A is the unit of the activation function, d H represents the Hamming distance, K represents the kernel matrix, and S represents the NASWOT metric; The other is the KL divergence measure, which is calculated as follows: Where p is the logits of the original model, q is the logits of the sub-model, T is an adjustable hyperparameter to control the influence between logits, and d is a measure of KL divergence; Then the fitness F, F = Sd; The specific search of the knowledge-aware model architecture search algorithm includes: Step 1: Generate the structure of multiple sub-models; Step 2: Randomly select two sub-model structures and calculate the fitness of the two sub-model structures. If the fitness of the first sub-model structure is greater than the fitness of the second sub-model structure, the first sub-model structure is the winner and the second sub-model structure is the loser. Otherwise, the second sub-model structure is the winner and the first sub-model structure is the loser. Step 3: Generate a random number. If it is less than the crossover rate, cross the sub-model structures corresponding to the winner and the loser to obtain a new structure. If the random number is less than the mutation rate, flip the sub-model structure corresponding to the loser to obtain a new structure. Step 4: Calculate the fitness of the new structure. If the fitness of the new structure is greater than the fitness of the sub-model structure corresponding to the loser, replace the sub-model structure corresponding to the loser with the new structure. Step5: Repeat Step1 to Step4 until the cycle is complete.

4. The clustering-based adaptive fine-tuning method for heterogeneous federated basic models according to claim 1, characterized in that: Step S3 specifically includes: S301, fine-tuning is performed using private data of the client, and only parameters are saved after fine-tuning; S302, uploading the parameters of each client to the representative client in the corresponding cluster, and then performing parameter aggregation; S303: Repeat steps S301 to S302 until the intra-cluster training is completed.

5. The clustering-based adaptive fine-tuning method for heterogeneous federated basic models according to claim 1, characterized in that: Step S4 specifically includes: S401. Knowledge is transferred through the representative clients in each cluster. The weight calculation formula of each representative client is as follows: where ω m represents the weight of each representative client, M is the number of clusters, N k represents the amount of data on the client side, x i is the original data, y i For labels; S402: Use unlabeled public data sets for knowledge distillation, use the representative client in each cluster as the teacher model, generate pseudo labels for the unlabeled data, and calculate the cross-entropy loss using the pseudo labels and the predicted values ​​of the server model. The calculation formula is as follows: in, represents unlabeled data, yes After θ leader(m) Generated pseudo labels; S403, the logits calculated by the teacher model using the public data set are transferred to the server, and the KL divergence is calculated with the logits calculated by the server model. The calculation formula is as follows: D KL is the representation of KL divergence, σ represents the activation function; S404. Combine cross-entropy loss and KL loss to obtain the total Loss. By minimizing the Loss, the server model is optimized to achieve fine-tuning of the server model. The total Loss calculation formula is as follows: α represents a hyperparameter that controls the ratio of cross-entropy loss and KL loss.

6. The clustering-based adaptive fine-tuning method for heterogeneous federated basic models according to claim 1, characterized in that: Step S4 specifically includes: S501, after optimizing the server model, the representative model of each cluster is updated by using the knowledge distillation method through the unlabeled public data set, and the representative model also saves the parameters; S502, after the representative model of each cluster is updated, the parameters are issued according to the sub-model structure of each client in the cluster, and the parameters corresponding to the sub-model structure are issued to update the model of each client; S503, executing step S1 until the fine-tuning is completed.

7. A computer device comprising a memory, wherein the memory stores program instructions, characterized in that: When the program instructions are executed, the clustering-based adaptive fine-tuning method for the heterogeneous federated basic model as described in any one of claims 1 to 6 is executed.

Images