Federal learning model fine tuning method and system for heterogeneous data

Abstract

The invention discloses a federal learning model fine tuning method and system for heterogeneous data, and belongs to the technical field of artificial intelligence, and the fine tuning method specifically comprises the following steps: S101, each client carries out the local training of a deep learning model of a random initialization parameter until a convergence standard is reached on a local data set; the method can reduce the number of iteration rounds of federal training, reduce the communication cost, enable the local model to be aligned to the fine-tuned global model in the early stage of training, reduce the weight divergence, reduce the constraint on the parameter update in the later stage of training, facilitate jumping out of the local optimal solution, reduce the risk of privacy leakage to a great extent, and improve the privacy protection efficiency. The decision-making layer is usually easily influenced by data isomerism in federal training, and the global decision-making layer is finely adjusted by using feature representatives of each category, so that the generalization performance of a global model is enhanced at very low calculation cost.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for fine-tuning a federated learning model for heterogeneous data. Background Art

[0002] The rapid development of artificial intelligence technology is inseparable from the support of a large amount of training data, which is usually generated by individuals or large corporate organizations. A more intuitive method is to collect data from all parties and centralize training on a trusted server. However, in the application scenarios involving some large institutions with strong privacy (banks, hospitals), it is difficult to find a trusted server, and the data transmission process may also be attacked by the network, resulting in the leakage of private data. Federated learning is a distributed machine learning method with a privacy protection paradigm. It aims to optimize the model with the idea of ​​"data does not move, model moves" without exposing the privacy of the participants. Federated training is usually carried out in the way of "local training, global aggregation". During the entire training process, only the model is shared externally, and the original data will not be leaked. The classic FedAvg algorithm collects local models to the server and then performs weighted average on them according to the number of local samples. Federated learning has great application prospects in scenarios involving important privacy, such as smart driving, smart wearable devices, health testing equipment, smart driving, and smart IoT devices.

[0003] Federated learning brings privacy protection features but also brings some other additional problems, which will greatly damage the normal training process. Compared with centralized machine learning, the data sets in federated learning are stored on some edge devices. Ideally, all participants optimize the model in the same direction, and the aggregated global model continues to be broadcast to the local for further optimization. However, in a data heterogeneous environment, these local data sets may be non-independent and identically distributed due to different collection methods, personal habits or business scenarios. In this environment, the optimization goals of local training are inconsistent, resulting in greater differences between local models, and further, the performance of the aggregated global model drops sharply, and federated training falls into a vicious circle. Existing federated learning algorithms mainly solve this challenge from two aspects. The first category is to use various algorithms to optimize the performance of the global model, aiming to improve the generalization performance of the global model. The second category no longer requires all clients to jointly maintain a global model, but allows clients with similar distributions to establish a cluster model or each client to establish a personalized model that conforms to its own local distribution. Existing technologies often combine some complex optimization algorithms or use technologies from other related fields to optimize the training process, such as regularization, knowledge distillation, meta-learning, etc. Most of these methods have problems such as high computational complexity or limited usage scenarios. Some other technologies enhance local data sets by sharing part of private data with other clients or training a generative adversarial network on the server. This method has performed well in a large number of experiments, but its practice of sharing data violates the principle of privacy protection in federated learning. The problem of data heterogeneity in federated learning still needs to be solved. Summary of the invention

[0004] The purpose of the present invention is to solve the defects in the prior art and to propose a method and system for fine-tuning a federated learning model for heterogeneous data.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for fine-tuning a federated learning model for heterogeneous data. The specific steps of the fine-tuning method are as follows:

[0007] S101: Each client performs local training on a deep learning model with randomly initialized parameters until the convergence standard is reached on the local data set. The server collects the local models trained by each client and performs weighted aggregation to form a preheated global model.

[0008] S102: At the beginning of each round of training, the server randomly selects multiple groups of client subsets participating in the training of this round from the active clients, and sends the aggregated global model to the selected client subsets for local prototype extraction and local training;

[0009] S103: The server receives local models and prototype features from each active client participating in the training, assigns corresponding aggregation weights according to the volume of each client's local data set, aggregates the local models, generates the latest global model, and fine-tunes the global model obtained by preliminary aggregation;

[0010] S104: The server sends the latest global model to the next round of random client subsets again and starts a new round of federated training until the specified number of training rounds is reached or the global model performance meets the requirements of each client.

[0011] As a further solution of the present invention, the specific steps of local training in S101 are as follows:

[0012] Q1.1: The server creates a deep learning model and randomly initializes its parameters. The deep learning model with randomly initialized parameters is then used as the starting point for all client training.

[0013] Q1.2: After each client receives the parameters of the initial deep learning model, it trains the model based on its local data set. At the same time, the model loss is calculated using the balance loss function. Minimizing the balance risk is used as the optimization goal of the local warm-up model. The deep learning model on the client is trained for multiple rounds to obtain a local warm-up model that converges on the local data set.

[0014] Q1.3: The server collects the local preheating models trained and generated on each client, assigns equal weights to the local preheating model parameters of each client, and then performs weighted aggregation on each group of local preheating models to form a preheating global model.

[0015] As a further solution of the present invention, the specific calculation formula of the balance loss function described in Q1.2 is as follows:

[0016]

[0017] Where N i,c is the local dataset D i The number of samples with label c∈C, where C is the label space, The logit output of the sample with label c; is the local model of client i in round t;

[0018] The optimization objective of the local preheating model described in Q1.2 is specifically expressed as follows:

[0019]

[0020] In the formula, θ t ∈R drepresents the global model of round t, N represents the number of clients participating in the training; here f i BSM (θ t ) represents the local data set D on the client i i The expected loss on .

[0021] As a further solution of the present invention, the specific steps of local prototype extraction and local training in S102 are as follows:

[0022] Q2.1: Each client divides the samples in the local dataset into multiple subsets according to categories, and inputs the sample subset of each category into the global model in turn to obtain the feature vector of each sample subset;

[0023] Q2.2: The feature vectors of each sample subset are averaged to obtain the local prototypes corresponding to each category of sample subsets. After generating the local prototypes, the client inputs the local data set into the local model in batches and gradually iterates and updates the model parameters.

[0024] Q2.3: The client combines the regularization term with the minimization of the loss function as the model optimization goal, repeatedly trains the model, and optimizes the model parameters. After the local training is completed, the client will optimize the local model. and a collection of locally generated prototypes Upload to the server.

[0025] As a further solution of the present invention, the specific extraction formula of the local prototype described in Q2.2 is as follows:

[0026]

[0027] Where D i,c Represents the local private dataset D i The set of samples with label c; Represents the prototype corresponding to the sample with label c; Represents the corresponding feature vector when the input is x; Represents the parameters of the underlying global model; Represents the embedding function corresponding to the global model;

[0028] The specific expression of the model optimization objective described in Q2.3 is as follows:

[0029]

[0030] In the formula, represents the dynamic regularization term, and the coefficient of the dynamic regularization term λ(t) decreases with the number of training rounds; μ represents a fixed regularization parameter; represents the cross entropy loss function, where represents the model output, i.e. and Represents the local model The feature extraction layer parameters and decision layer parameters of and Represent the corresponding embedding functions respectively; T represents the total number of global training rounds.

[0031] As a further solution of the present invention, the steps of aggregating the local models in S103 to generate the latest global model and fine-tuning the global model obtained by preliminary aggregation are as follows:

[0032] Q3.1: After receiving the local models and local prototypes uploaded by each client, the server assigns corresponding weighting coefficients according to the size of each client's local data set. Then, the server performs weighted average on each local model and local prototype based on the assigned weighting coefficients to generate the latest global model and the global prototype set corresponding to each category.

[0033]

[0034] Q3.2: Input the obtained global prototype set into the decision layer of the global model. With the goal of minimizing the empirical risk, perform multiple rounds of iterative optimization on the global decision layer until the empirical risk reaches the preset threshold, so as to enhance the global model's ability to discriminate different categories of data.

[0035] As a further solution of the present invention, the specific calculation formula of the weighted average described in Q3.1 is as follows:

[0036]

[0037] In the formula, S t represents the client subset selected in round t, S c,t represents the set of clients with category c in the local dataset of this subset;

[0038] The specific expression of minimizing empirical risk in Q3.2 is as follows:

[0039]

[0040] In the formula, Represents the fine-tuned global decision maker, which will then overwrite the aggregated global decision maker ω t , and combined with the original global model to form a new global model and sent to a new round of clients.

[0041] A federated learning model fine-tuning system for heterogeneous data, including a server and multiple groups of clients;

[0042] The client has a local private dataset D i , and present a non-independent and identically distributed state. Each client uploads the model, downloads the model, and performs local training under the coordination of the server. The entire process only communicates between the server and the client;

[0043] After downloading the latest global model sent by the server, the client overwrites its original local model and optimizes the model using the local data set, and this process is performed in parallel by multiple clients;

[0044] After receiving the local models, the server assigns aggregation weights to each local model according to the size of the client sample, and performs weighted aggregation to obtain the latest global model.

[0045] As a further solution of the present invention, the optimization goal of the entire server-client federated training is to enable the jointly maintained model to obtain the minimum expected loss on various data sets, that is:

[0046]

[0047] In the formula, θ∈R d represents the jointly maintained global model; R d ; D≡∪ i∈[N] D i represents the entire data set; f i (θ) represents the expected loss of the model on the local dataset; l(·) represents the error loss function used to evaluate the difference between the true value and the predicted value.

[0048] As a further solution of the present invention, in order to optimize the global model locally on each local model without exposing the client's private data and obtain the minimum expected loss, the global model needs to be optimized on the client's local data to learn different data features. In a neural network, the bottom layer of the model is usually called the feature extraction layer, which is used to extract basic features about the data, and the top layer is usually called the decision layer, which is used to combine the basic features to obtain abstract features about the data. The goal of the local optimization process is:

[0049]

[0050] In the formula, ω i and Represent the parameter matrix of the model respectively, that is, the model consists of two parts Represents the embedding function of the feature extraction layer, which is used to transform the high-dimensional sample space The basic features in are mapped to the latent feature space Represents the embedding function of the decision layer, which is used to map hidden features to the output space

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention creates a deep learning model through a server and randomly initializes its parameters, and then uses the deep learning model with randomly initialized parameters as the starting point for training all clients. After each client receives the parameters of the initial deep learning model, each client performs local training on a deep learning model with randomly initialized parameters until the convergence standard is reached on the local data set. The server collects the local models trained by each client and performs weighted aggregation to form a preheated global model. At the beginning of each round of training, the server randomly selects multiple groups of client subsets participating in the training in this round from the active clients, and sends the aggregated global model to the selected client subset for local prototype extraction and local training. The server receives local models and prototype features from each active client participating in the training, and extracts the local prototype features according to the local data of each client. According to the size of the data set, the corresponding aggregation weights are assigned, the local models are aggregated, the latest global model is generated, and the global model obtained by the initial aggregation is fine-tuned. The latest global model is then sent to the next round of random client subsets, and a new round of federated training is started until the specified number of training rounds is reached or the global model performance meets the requirements of each client. This can reduce the number of iterations of federated training and the communication cost, align the local model with the fine-tuned global model in the early stage of training, reduce weight divergence, and reduce the constraints on its parameter update in the later stage of training, which is conducive to jumping out of the local optimal solution and can greatly reduce the risk of privacy leakage. In addition, the decision layer is usually susceptible to data heterogeneity in federated training. The feature representative of each category is used to fine-tune the global decision layer, which enhances the generalization performance of the global model at a very low computational cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0054] Figure 1 A flowchart of a method for fine-tuning a federated learning model for heterogeneous data proposed by the present invention;

[0055] Figure 2 A system block diagram of a federated learning model fine-tuning system for heterogeneous data proposed by the present invention;

[0056] Figure 3Classification accuracy graph of each category in the heterogeneous CIFAR10 dataset for a federated learning model fine-tuning method for heterogeneous data proposed in the present invention and a baseline method. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0058] Example 1

[0059] Reference Figure 1 as well as Figure 3 This embodiment discloses a method for fine-tuning a federated learning model for heterogeneous data. The specific steps of the fine-tuning method are as follows:

[0060] Each client performs local training on a deep learning model with randomly initialized parameters until it reaches convergence on the local data set. The server collects the local models trained by each client and performs weighted aggregation to form a preheated global model.

[0061] Specifically, the server creates a deep learning model and randomly initializes its parameters, and then uses the deep learning model with randomly initialized parameters as the starting point for training all clients. After each client receives the parameters of the initial deep learning model, the model is trained based on its local data set. At the same time, the model loss is calculated through the balance loss function, and the local preheating model optimization goal is to minimize the balance risk. The deep learning model on the client is trained for multiple rounds to obtain a local preheating model that converges on the local data set. The server collects each group of local preheating models generated by training on each client, assigns equal weights to the local preheating model parameters of each client, and then weighted aggregates each group of local preheating models to form a preheating global model.

[0062] It should be further explained that the specific calculation formula of the balance loss function is as follows:

[0063]

[0064] Where N i,c is the local dataset D i The number of samples with label c∈C, where C is the label space, The logit output of the sample with label c; is the local model of client i in round t;

[0065] The optimization objective of the local warm-up model is specifically expressed as follows:

[0066]

[0067] In the formula, θ t ∈R d represents the global model of round t, N represents the number of clients participating in the training; here f i BSM (θ t ) represents the local data set D on the client i i The expected loss on .

[0068] At the beginning of each round of training, the server randomly selects multiple groups of client subsets participating in the training from the active clients, and sends the aggregated global model to the selected client subsets for local prototype extraction and local training.

[0069] Specifically, each client divides the samples in the local data set into multiple subsets according to categories, and inputs the sample subsets of each category into the global model in turn to obtain the feature vectors of each sample subset, and averages the feature vectors of each sample subset to obtain the local prototypes corresponding to the sample subsets of each category. After generating the local prototypes, the client inputs the local data set into the local model in batches, and gradually iterates and updates the model parameters. The client combines the regularization term with the minimization of the loss function as the model optimization goal, repeatedly trains the model, and optimizes the model parameters. After the local training is completed, the client will optimize the local model. and a collection of locally generated prototypes Upload to the server.

[0070] It should be further explained that the specific formula for extracting the local prototype is as follows:

[0071]

[0072] Where D i,c Represents the local private dataset D i The set of samples with label c; Represents the prototype corresponding to the sample with label c; Represents the corresponding feature vector when the input is x; Represents the parameters of the underlying global model; Represents the embedding function corresponding to the global model;

[0073] The specific expression of the model optimization objective is as follows:

[0074]

[0075] In the formula, represents the dynamic regularization term, and the coefficient of the dynamic regularization term λ(t) decreases with the number of training rounds; μ represents a fixed regularization parameter; represents the cross entropy loss function, where represents the model output, i.e. and Represents the local model The feature extraction layer parameters and decision layer parameters of and Represent the corresponding embedding functions respectively; T represents the total number of global training rounds.

[0076] The server receives local models and prototype features from each active client participating in the training, assigns corresponding aggregation weights based on the size of each client's local data set, aggregates local models, generates the latest global model, and fine-tunes the global model obtained by the preliminary aggregation.

[0077] Specifically, after receiving the local models and local prototypes uploaded by each client, the server assigns corresponding weighting coefficients according to the size of each client's local data set. Then, the server performs weighted averaging on each local model and local prototype based on the assigned weighting coefficients to generate the latest global model and the global prototype set corresponding to each category. The obtained global prototype set is input into the decision layer of the global model. With the goal of minimizing the empirical risk, the global decision layer is iteratively optimized for multiple rounds until the empirical risk reaches the preset threshold, so as to enhance the global model's ability to discriminate different categories of data.

[0078] It should be further explained that the specific calculation formula of weighted average is as follows:

[0079]

[0080]

[0081] In the formula, S t represents the client subset selected in round t, S c,t represents the set of clients with category c in the local dataset of this subset;

[0082] The specific expression of minimizing empirical risk is as follows:

[0083]

[0084] In the formula, Represents the fine-tuned global decision maker, which will then overwrite the aggregated global decision maker ω t , and combined with the original global model to form a new global model and sent to a new round of clients.

[0085] The server sends the latest global model again to the next round of random client subsets and starts a new round of federated training until the specified number of training rounds is reached or the global model performance meets the requirements of each client.

[0086] Example 2

[0087] Reference Figure 2 ,This embodiment discloses a federated learning model fine-tuning system for heterogeneous data, including a server and multiple groups of clients.

[0088] The client has a local private dataset D i , and presents a non-independent and identically distributed state. Each client uploads the model, downloads the model, and performs local training under the coordination of the server. The entire process only involves communication between the server and the client. After downloading the latest global model sent by the server, the client overwrites its original local model and uses the local data set to optimize the model. This process is executed in parallel by multiple clients. After receiving the local model, the server assigns aggregation weights to each local model according to the size of the client sample, and performs weighted aggregation to obtain the latest global model.

[0089] In this embodiment, the optimization goal of the entire server-client federated training is to enable the jointly maintained model to obtain the minimum expected loss on various data sets, that is:

[0090]

[0091] In the formula, θ∈R d represents the jointly maintained global model; R d ; D≡∪ i∈[N] D i represents the entire data set; f i (θ) represents the expected loss of the model on the local dataset; l(·) represents the error loss function used to evaluate the difference between the true value and the predicted value.

[0092] In order to optimize the global model locally on each local model and obtain the minimum expected loss without exposing the client's private data, the global model needs to be optimized on the client's local data to learn different data features. In a neural network, the bottom layer of the model is usually called the feature extraction layer, which is used to extract basic features about the data. The top layer is usually called the decision layer, which is used to combine basic features to obtain abstract features about the data. The goals of the local optimization process are:

[0093]

[0094] In the formula, ω i and Represent the parameter matrix of the model respectively, that is, the model consists of two parts Represents the embedding function of the feature extraction layer, which is used to transform the high-dimensional sample space The basic features in are mapped to the latent feature space Represents the embedding function of the decision layer, which is used to map hidden features to the output space

Claims

1. A method for fine-tuning a federated learning model for heterogeneous data, characterized in that: The specific steps of the fine-tuning method are as follows: S101: Each client performs local training on a deep learning model with randomly initialized parameters until the convergence standard is reached on the local data set. The server collects the local models trained by each client and performs weighted aggregation to form a preheated global model. S102: At the beginning of each round of training, the server randomly selects multiple groups of client subsets participating in the training of this round from the active clients, and sends the aggregated global model to the selected client subsets for local prototype extraction and local training; S103: The server receives local models and prototype features from each active client participating in the training, assigns corresponding aggregation weights according to the volume of each client's local data set, aggregates the local models, generates the latest global model, and fine-tunes the global model obtained by preliminary aggregation; S104: The server sends the latest global model to the next round of random client subsets again and starts a new round of federated training until the specified number of training rounds is reached or the global model performance meets the requirements of each client.

2. A method for fine-tuning a federated learning model for heterogeneous data according to claim 1, characterized in that: The specific steps of local training in S101 are as follows: Q1.1: The server creates a deep learning model and randomly initializes its parameters. The deep learning model with randomly initialized parameters is then used as the starting point for all client training. Q1.2: After each client receives the parameters of the initial deep learning model, it trains the model based on its local data set. At the same time, the model loss is calculated using the balance loss function. Minimizing the balance risk is used as the optimization goal of the local warm-up model. The deep learning model on the client is trained for multiple rounds to obtain a local warm-up model that converges on the local data set. Q1.3: The server collects the local preheating models trained and generated on each client, assigns equal weights to the local preheating model parameters of each client, and then performs weighted aggregation on each group of local preheating models to form a preheating global model.

3. The method for fine-tuning a federated learning model for heterogeneous data according to claim 2, characterized in that: The specific calculation formula of the balance loss function described in Q1.2 is as follows: Where N i,c is the local dataset D i The number of samples with label c∈C, where C is the label space, is the logit output of the sample with label c; is the local model of client i in round t; The optimization objective of the local preheating model described in Q1.2 is specifically expressed as follows: In the formula, θ t ∈R d represents the global model of round t, and N represents the number of clients participating in the training; Denotes the local dataset D on the client i i Expected loss on .

4. The method for fine-tuning a federated learning model for heterogeneous data according to claim 2, characterized in that: The specific steps of local prototype extraction and local training in S102 are as follows: Q2.1: Each client divides the samples in the local dataset into multiple subsets according to categories, and inputs the sample subset of each category into the global model in turn to obtain the feature vector of each sample subset; Q2.2: The feature vectors of each sample subset are averaged to obtain the local prototypes corresponding to each category of sample subsets. After generating the local prototypes, the client inputs the local data set into the local model in batches and gradually iterates and updates the model parameters. Q2.3: The client combines the regularization term with the minimization of the loss function as the model optimization goal, repeatedly trains the model, and optimizes the model parameters. After the local training is completed, the client will optimize the local model θ i t and a collection of locally generated prototypes Upload to the server.

5. The method for fine-tuning a federated learning model for heterogeneous data according to claim 4, characterized in that: The specific formula for extracting the local prototype described in Q2.2 is as follows: Where D i,c Represents the local private dataset D i The set of samples with label c; Represents the prototype corresponding to the sample with label c; Represents the corresponding feature vector when the input is x; Represents the parameters at the bottom of the global model; Represents the embedding function corresponding to the global model; The specific expression of the model optimization objective described in Q2.3 is as follows: In the formula, represents the dynamic regularization term, and the coefficient of the dynamic regularization term λ(t) decreases with the number of training rounds; μ represents a fixed regularization parameter; represents the cross entropy loss function, where represents the model output, i.e. and Represents the local model The feature extraction layer parameters and decision layer parameters of and Represent the corresponding embedding functions respectively; T represents the total number of global training rounds.

6. The method for fine-tuning a federated learning model for heterogeneous data according to claim 4, characterized in that: The specific steps of aggregating the local models in S103 to generate the latest global model and fine-tuning the global model obtained by preliminary aggregation are as follows: Q3.1: After receiving the local models and local prototypes uploaded by each client, the server assigns corresponding weighting coefficients according to the size of each client's local data set. Then, the server performs weighted average on each local model and local prototype based on the assigned weighting coefficients to generate the latest global model and the global prototype set corresponding to each category. Q3.2: Input the obtained global prototype set into the decision layer of the global model. With the goal of minimizing the empirical risk, perform multiple rounds of iterative optimization on the global decision layer until the empirical risk reaches the preset threshold, so as to enhance the global model's ability to discriminate different categories of data.

7. The method for fine-tuning a federated learning model for heterogeneous data according to claim 6, characterized in that: The specific calculation formula for the weighted average mentioned in Q3.1 is as follows: In the formula, S t represents the client subset selected in round t, S c,t represents the set of clients with category c in the local dataset of this subset; The specific expression of minimizing empirical risk in Q3.2 is as follows: In the formula, Represents the fine-tuned global decision maker, which will then overwrite the aggregated global decision maker ω t , and combined with the original global model to form a new global model and sent to a new round of clients.

8. A system for fine-tuning a federated learning model for heterogeneous data, used to implement the method for fine-tuning a federated learning model for heterogeneous data as claimed in any one of claims 1 to 7, characterized in that: Including server and multiple groups of clients; The client has a local private dataset D i , and present a non-independent and identically distributed state. Each client uploads the model, downloads the model, and performs local training under the coordination of the server. The entire process only communicates between the server and the client; After downloading the latest global model sent by the server, the client overwrites its original local model and optimizes the model using the local data set, and this process is performed in parallel by multiple clients; After receiving the local models, the server assigns aggregation weights to each local model according to the size of the client sample, and performs weighted aggregation to obtain the latest global model.

Images