A federated active learning model training method and system

Abstract

The present invention relates to a method and system for training a federated active learning model, which combines federated learning with active learning, and can fully utilize a large amount of unlabeled data of each participant for collaborative model training under the premise of protecting data privacy. In each round of federated learning training, the global model obtained by federated learning is used to guide local data labeling, thereby improving labeling efficiency and overcoming the data island problem to a certain extent. A hybrid sampling strategy is proposed based on loss prediction and feature space distance in selecting informative samples for active learning, which can select high-quality samples and reduce labeling costs; reducing sample labeling costs can improve the accuracy and generalization performance of the model.

Description

Technical Field

[0001] The present invention relates to information technology, and in particular to a method and system for training a federated active learning model. Background Art

[0002] With the rapid development of artificial intelligence, especially the widespread application of deep learning models in various fields, social production efficiency has been greatly improved. However, the premise of obtaining a good-performance deep learning model in a certain field is to have a high-quality and high-volume data set. The quality of data is the premise for ensuring the generalization performance of the deep learning model. Nowadays, many unlabeled data are generated every day in the medical, industrial, financial or other fields, such as hospitals, factories, Internet companies, securities companies, insurance companies, etc., and these data have the characteristics of repetition and low value density. How to reasonably and efficiently use these unlabeled data to train deep learning models is a major practical problem. The most direct way is to upload these data to a high-performance server center and manually annotate these data, and then train the deep learning model. However, considering issues such as privacy protection, industry competition, legal constraints, intellectual property protection, data storage and communication, it is very difficult to centralize data from all parties, and manually annotating all data is also a huge expense. Therefore, these companies or factories are like data islands, and this phenomenon is more obvious in the medical and financial industries. This greatly restricts the development of related fields.

[0003] Federated learning is a distributed machine learning paradigm proposed in recent years. Under the premise that the private data of each participant does not leave the local area, each participant can upload the local model parameters to a server to aggregate the global model through homomorphic encryption, differential privacy, secure multi-party computing and other means. In each round of training, each participant uses its own local data to train the local model, encrypts the model parameters through homomorphic encryption and other methods, and transmits them to the server. The server aggregates the model parameters sent by each participant according to the corresponding aggregation algorithm (such as the FedAvg algorithm) to generate a global model and broadcasts the global model at this time to each participant to complete this round of training.

[0004] Active learning is a subfield of machine learning, also known as query learning or optimal experimental design in statistics. Active learning methods attempt to solve the bottleneck of sample labeling by interactively selecting a subset of samples that need to be labeled from the pool of unlabeled samples to be manually labeled by domain or industry experts. The main goal of active learning is to select high-quality samples and reduce the cost of labeling as much as possible while the model achieves the target performance. Summary of the invention

[0005] To address the problems of data silos and high-quality data labeling, a federated active learning model training method and system was proposed, which organically combined federated learning and active learning. Under the premise of protecting data privacy, it fully utilized the unlabeled data information of each participant and reduced the sample labeling cost as much as possible to improve the accuracy and generalization performance of the model.

[0006] The technical solution of the present invention is: a federated active learning model training method, comprising the following steps:

[0007] 1) Each participant randomly selects a portion of samples from the local unlabeled dataset in parallel and asks experts to label them. The labeled data samples are moved from the local dataset to the labeled dataset D. l The remaining unlabeled dataset is D u ,At the same time, the central server randomly generates the global model parameter W and sends it to each participant;

[0008] 2) Each participant receives the global model parameters W from the central server in parallel and loads them into the local model. l For the local model W local Train and use the local model W local The feature embedding in is taken as input, and the local model W local The loss is used as the label pair loss prediction model W loss Conduct training and update model parameters during learning and training;

[0009] 3) Each participant uses the trained loss prediction model W in parallel loss Right D u Perform model inference once for all data samples in and obtain the prediction loss value pl of all current unlabeled data;

[0010] 4) Each participant uses the trained local model W in parallel local Right D l and D u Perform a model inference on all data samples in the dataset, extract the feature embeddings in the latent space of all current data, and average the feature embeddings of all labeled data samples to get their centroids. Then calculate the Euclidean distance dist from the feature embeddings of all unlabeled data samples to the centroids of the feature embeddings of all labeled data samples.

[0011] 5) Each participant uses the pl and dist obtained in steps 3) and 4) to calculate the Rank value of each unlabeled data sample in parallel, sorts them in descending order according to the size of the Rank, selects the first b samples to be labeled by experts, and selects them from D u Move to D l ;

[0012] 6) Each participant uploads the updated local model parameters in parallel;

[0013] 7) The central server receives the local model parameters uploaded by each participant, uses the model aggregation algorithm to update the global model parameters W, and sends it to each participant;

[0014] 8) Iterative training, repeat 2) to 7) until the number of iterations is met, and the final local annotation dataset and data recognition local model W are obtained. local .

[0015] Further, the step 2) loss prediction model W loss The structure is: input labeled data samples in the local model W local The feature embedding in multiple latent spaces in the data sample is averaged and pooled, and multiple feature vectors are obtained by performing the operations of the fully connected layer on the multiple feature embeddings of the data sample. Then, the multiple feature vectors are concatenated together and passed through the fully connected layer to obtain the loss prediction model W. loss The output of is the loss prediction for the data sample.

[0016] Further, the step 2) loss prediction model W loss and the local model W local The parameter update method in the training is: using the gradient descent method, using the mean square error loss function, and the true value is to train the local model W local The training loss at this time is updated loss and W local The calculation formula of the parameters is:

[0017]

[0018]

[0019]

[0020] Where t represents the tth round of federated learning training, η is the learning rate, and L loss W is the loss prediction model loss The loss function of the training set, n is D l The number of samples in, predicted loss i is the prediction loss of the i-th sample, train loss i is the i-th sample in the local model W local The training loss in L local is the local model W local The loss function of the training set varies depending on the task.

[0021] Further, the specific calculation method of step 4) dist is: the calculation formula of the centroid of the feature embedding of the jth unlabeled data sample to the feature embedding of all labeled data samples is:

[0022]

[0023] The function d(x,y) is used to calculate the Euclidean distance between x and y, and n represents D l The number of data samples in , f(x) represents the feature embedding of data sample x, x li represents the i-th labeled data sample, x uj Describe the jth unlabeled data sample.

[0024] Furthermore, the values ​​of the first b samples in step 4) are at most a few percent of the number of samples in the initial unlabeled data set.

[0025] Furthermore, the aggregation algorithm formula of step 7) is:

[0026]

[0027] Where m is the number of participants, Represents the local model parameters uploaded by the i-th participant.

[0028] A federated active learning model training system includes a central server and several participant devices, wherein the participant devices store a large number of unlabeled data sets, each participant communicates with relevant domain experts, and the domain experts perform high-quality annotations on data samples; the central server includes a global model, and the global model is aggregated and updated according to the uploaded local model parameters of each participant; each local participant device is equipped with a local data recognition model, and the local data recognition model is trained under the model parameters issued by the central server and the recognition annotations of the domain experts, so as to mark and identify local data.

[0029] The beneficial effects of the present invention are as follows: the federated active learning model training method and system of the present invention organically combine federated learning and active learning. In each round of federated learning training, the global model obtained by federated learning is used to guide local data labeling, thereby improving labeling efficiency and overcoming the data island problem to a certain extent. Under the premise of protecting data privacy, a large amount of unlabeled data of each participant can be fully utilized for collaborative training of the model. In active learning, a hybrid sampling strategy based on loss prediction and feature space distance is proposed, which can select high-quality samples and reduce the cost of labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1It is a flowchart of the federated active learning model training method of the present invention;

[0031] Figure 2 It is a schematic diagram of the architecture of the federated active learning model training system of the present invention;

[0032] Figure 3 Schematic diagram of the sample sampling strategy in the federated active learning model training method and system of the present invention. DETAILED DESCRIPTION

[0033] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0034] like Figure 1 , 2 The flowchart and system diagram of the federated active learning model training method shown in the figure include the following steps:

[0035] S100, each participant randomly selects a part of samples from the local unlabeled data set in parallel and asks experts to label them, and moves the labeled data samples from the local data set to the labeled data set D l The remaining unlabeled dataset is D u ,At the same time, the central server randomly generates the global model parameter W and sends it to each participant;

[0036] Specifically, each participant will have two local data sets D u and D l , D u The unlabeled data samples are stored in D l The labeled data samples are saved in l An empty dataset.

[0037] S200, each participant receives the global model parameter W from the central server in parallel and loads it into the local model. l For the local model W local Train and use the local model W local The feature embedding in is taken as input, and the local model W local The loss is used as the label pair loss prediction model W loss Conduct training;

[0038] Specifically, the local model W local Input labeled dataset D l , use the obtained global model parameters W to learn and train the label features. Loss prediction model W loss The input is the labeled data sample in the local model W localThe feature embeddings in multiple latent spaces in the data sample are averaged and pooled, and multiple feature vectors are obtained by performing full connection layer operations on multiple feature embeddings of the data sample. Then, multiple feature vectors are concatenated together and passed through the full connection layer to obtain the loss prediction model W. loss The output of the loss prediction model W is the loss prediction of the data sample. loss The training method uses gradient The descent method uses the mean square error loss function, and the true value is used to train the local model W local The training loss at this time is updated loss and W local The calculation formula of the parameters is:

[0039]

[0040]

[0041]

[0042] Where t represents the tth round of federated learning training, η is the learning rate, and L loss W is the loss prediction model loss The loss function of the training set, n is D l The number of samples in, predicted loss i is the prediction loss of the i-th sample, train loss i is the i-th sample in the local model W local The training loss in L local is the local model W local The loss function of the training set varies depending on the task.

[0043] The benefits of this step are: In each round of federated learning, the local model will load the current global model parameters before local training, and use the labeled data samples in the local model W local The feature embedding featureembedding in multiple latent spaces in is used as the loss prediction model W loss The input of the local model W local The loss value generated by training on labeled data samples is train loss i As the label pair loss prediction model W loss Training is performed, and the next step will be based on the loss prediction model W loss The output value of the inferred unlabeled data sample is used as the basis for sample sampling, so as to utilize the information learned by the federated learning model and the information of the local labeled data samples to guide the sample sampling of active learning.

[0044] S300, each participant uses the trained loss prediction model W in parallel loss Right D u Perform model inference once for all data samples in and obtain the prediction loss value pl of all current unlabeled data;

[0045] S400, such as Figure 3 As shown, each participant uses the trained local model W in parallel local Right D l and D u Perform a model inference on all data samples in the dataset, extract the feature embedding featureembedding in the latent space of all current data, and average the feature embedding of all labeled data samples to get their centroid, then calculate the Euclidean distance dist from the feature embedding of all unlabeled data samples to the centroid of the feature embedding of all labeled data samples;

[0046] Specifically, the formula for calculating the centroid of the feature embedding of the jth unlabeled data sample to the feature embedding of all labeled data samples is:

[0047]

[0048] The function d(x,y) is used to calculate the Euclidean distance between x and y, and n represents D l The number of data samples in , f(x) represents the feature embedding of data sample x, x li represents the i-th labeled data sample, x uj Describe the jth unlabeled data sample.

[0049] Benefits of this step: Like S200, the calculated spatial distance is used as the basis for sample sampling in the next step. The information learned by the federated learning model and the local labeled data sample information are also used to guide the active learning sample sampling.

[0050] S500, each participant uses the obtained pl and dist to calculate the Rank value of each unlabeled data sample in parallel, sorts them in descending order according to the size of the Rank, selects the first b samples to be labeled by experts, and selects them from D u Move to D l ;

[0051] Specifically, the Rank value calculation formula for an unlabeled data sample is:

[0052] Rank u =αpl u +βdist u

[0053] Among them, α and β are weight coefficients, α = 1 / 2, β = 1 / 2, pl u is the predicted loss value of an unlabeled data sample, dist u It is the distance from an unlabeled data sample to the centroid of the feature embedding of all labeled data samples.

[0054] Specifically, the values ​​of the first b samples are much smaller than the number of samples in the local unlabeled dataset, and at most a few percent of the number of samples in the initial unlabeled dataset, so as to select high-quality sample data.

[0055] The benefits of this step are: using loss prediction value and spatial distance as quantitative indicators of active learning sampling, when the unlabeled data samples pass through the loss prediction model W loss The larger the loss prediction value pl obtained by inference, the more information the sample contains, because the loss prediction model W loss It is trained by labeled data samples. If the loss prediction value of a sample is smaller, it means that the sample is similar to the labeled data sample, which is almost useless for model training. Similarly, the larger the Euclidean distance dist between the feature embedding of the unlabeled data sample and the centroid of the feature embedding of all labeled data samples, the more information the sample contains. The present invention comprehensively considers the two factors of loss prediction and spatial distance, and can effectively sample informative samples.

[0056] S600, each participant uploads the updated local model parameters after training in parallel;

[0057] S700: The central server receives the local model parameters uploaded by each participant, updates the global model parameters W using the model aggregation algorithm, and sends the global model parameters W to each participant;

[0058] Specifically, the aggregation algorithm formula of the global model is:

[0059]

[0060] Where m is the number of participants, Represents the local model parameters uploaded by the i-th participant.

[0061] S800, iterative training, repeat S200-S700 until the number of iterations is met, and the final local annotation data set and data recognition local model W are obtained. local .

[0062] In this embodiment, if Figure 2 As shown, a federated learning model training system implemented based on the above method includes a central server and several participant devices, and the participant devices store a large number of unlabeled data sets. In addition, each participant will have relevant domain experts who can perform high-quality annotation work on data samples. The central server includes a global model, and the global model is aggregated and updated according to the uploaded local model parameters of each participant; each local participant device is equipped with a local data recognition model, and the local data recognition model is trained under the model parameters issued by the central server and the recognition annotations of domain experts, and is used to annotate and identify local data.

[0063] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A federated active learning model training method, characterized in that: The steps include: 1) Each participant randomly selects a portion of samples from the local unlabeled dataset in parallel and asks experts to label them. The labeled data samples are moved from the local dataset to the labeled dataset D. l The remaining unlabeled dataset is D u ,At the same time, the central server randomly generates the global model parameter W and sends it to each participant; 2) Each participant receives the global model parameters W from the central server in parallel and loads them into the local model. l For the local model W local Train and use the local model W local The feature embedding in is taken as input, and the local model W local The loss is used as the label pair loss prediction model W loss Conduct training and update model parameters during learning and training; 3) Each participant uses the trained loss prediction model W in parallel loss Right D u Perform model inference once for all data samples in and obtain the prediction loss value pl of all current unlabeled data; 4) Each participant uses the trained local model W in parallel local Right D l and D u Perform a model inference on all data samples in the dataset, extract the feature embeddings in the latent space of all current data, and average the feature embeddings of all labeled data samples to get their centroids. Then calculate the Euclidean distance dist from the feature embeddings of all unlabeled data samples to the centroids of the feature embeddings of all labeled data samples. 5) Each participant uses the pl and dist obtained in steps 3) and 4) to calculate the Rank value of each unlabeled data sample in parallel, sorts them in descending order according to the size of the Rank, selects the first b samples to be labeled by experts, and selects them from D u Move to D l ; 6) Each participant uploads the updated local model parameters in parallel; 7) The central server receives the local model parameters uploaded by each participant, uses the model aggregation algorithm to update the global model parameters W, and sends it to each participant; 8) Iterative training, repeat 2) to 7) until the number of iterations is met, and the final local annotation dataset and data recognition local model W are obtained. local .

2. The method for training a federated active learning model according to claim 1, characterized in that: Step 2) loss prediction model W loss The structure is: input labeled data samples in the local model W local The feature embedding in multiple latent spaces in the data sample is averaged and pooled, and multiple feature vectors are obtained by performing the operations of the fully connected layer on the multiple feature embeddings of the data sample. Then, the multiple feature vectors are concatenated together and passed through the fully connected layer to obtain the loss prediction model W. loss The output of is the loss prediction for the data sample.

3. The method for training a federated active learning model according to claim 2, characterized in that: Step 2) loss prediction model W loss and the local model W local The parameter update method in the training is: using the gradient descent method, using the mean square error loss function, and the true value is to train the local model W local The training loss at this time is updated loss and W local The calculation formula of the parameters is: Where t represents the tth round of federated learning training, η is the learning rate, and L loss W is the loss prediction model loss The loss function of the training set, n is D l The number of samples in, predictedloss i is the prediction loss of the i-th sample, train loss i is the i-th sample in the local model W local The training loss in L local is the local model W local The loss function of the training set varies depending on the task.

4. The method for training a federated active learning model according to claim 3, characterized in that: The specific calculation method of step 4) dist is as follows: the calculation formula of the centroid of the feature embedding of the jth unlabeled data sample to the feature embedding of all labeled data samples is: The function d(x,y) is used to calculate the Euclidean distance between x and y, and n represents D l The number of data samples in , f(x) represents the feature embedding of data sample x, x li represents the i-th labeled data sample, x uj Describe the jth unlabeled data sample.

5. The method for training a federated active learning model according to claim 4, characterized in that: The values ​​of the first b samples in step 4) are at most a few percent of the number of samples in the initial unlabeled data set.

6. The method for training a federated active learning model according to claim 4, characterized in that: The aggregation algorithm formula of step 7) is: Where m is the number of participants, Represents the local model parameters uploaded by the i-th participant.

7. A federated active learning model training system, characterized in that: It includes a central server and several participant devices, wherein the participant devices store a large number of unlabeled data sets, each participant communicates with relevant domain experts, and the domain experts perform high-quality annotations on data samples; the central server includes a global model, and the global model is aggregated and updated according to the uploaded local model parameters of each participant; each local participant device is equipped with a local data recognition model, and the local data recognition model is used to label and identify local data under the model parameters issued by the central server and the identification annotations of the domain experts, and the system is trained using the federated active learning model training method described in claim 1.

Images