Multi-party collaborative data learning system and learning model training method

Abstract

The present invention relates to a multi-party collaborative data learning system and a learning model training method, which fully utilizes the advantages of active learning and federated learning, uses a small amount of labeled data of each client to perform collaborative training of the model under the premise of protecting data privacy, generates probability distribution with predicted loss value in each round of federated learning training, selects clients to participate in training, can accelerate the convergence of the global model and reduce the communication volume; the client loads the current global model parameters into the local model for active learning, uses the local model to guide sample query, and samples highly informative samples for labeling; the labeled data set of each client is expanded, and federated learning training is performed again to obtain a model with better performance. The accuracy and generalization performance of the model are improved under the premise of reducing the sample labeling cost as much as possible.

Description

Technical Field

[0001] The present invention relates to a data information technology, and in particular to a multi-party collaborative data learning system and a learning model training method. Background Art

[0002] The development of artificial intelligence relies on a large amount of information data, which has driven deep learning to make great progress with the support of large labeled data sets. The deep learning models obtained have been widely used in production practice, such as face recognition systems, smoke alarm systems, smart workshops, and autonomous driving. In addition, deep learning has great development prospects in the medical field, financial field, industrial field, education field, etc. In recent years, the Internet has matured and the Internet of Things has developed rapidly. A large amount of data is generated every day in all walks of life. How to make good use of this data to train deep learning models has become a major practical challenge.

[0003] The following challenges are faced in using these data: First, these data are characterized by large data volume and low value density, and most of them need to be labeled manually or by other costly methods. Since the cost of labeling data sets is very high, this has prompted many researchers to study and find ways to reduce the cost of labeling, resulting in research directions such as active learning; second, an excellent deep learning model cannot be separated from a large amount of data drive. First, the amount of data collected by a single enterprise in a certain field may not meet the needs of training deep learning models. Secondly, the data that some enterprises need to collect does not belong to the enterprise, but may belong to individuals or other enterprises. Therefore, they want to cooperate and adopt the traditional data centralization method, that is, multiple enterprises or individuals upload these data to the same high-performance server center for deep learning model training. This breeds many problems such as privacy protection, industry competition, legal constraints, intellectual property protection, data storage and communication. To solve these problems, researchers proposed a federated learning framework. Summary of the invention

[0004] In order to solve the problem that improving the accuracy of learning models depends on a large amount of high-quality data, a multi-party collaborative data learning system and a learning model training method are proposed. The advantages of active learning and federated learning are fully utilized. For classification tasks, the labeled data information of each client is used for federated learning collaborative training under the premise of protecting data privacy. The model obtained by federated learning training is used to guide each participant to perform active learning, sample informative samples, and then further perform federated learning training. Ultimately, the sample labeling cost is reduced as much as possible, and the accuracy and generalization performance of the model are improved.

[0005] The technical solution of the present invention is: a multi-party collaborative data learning system, including a central server for configuring a global model and multiple clients with local classification models, the central server sends model parameters, the clients use local labeled data for reasoning and training, the reasoning and training results are returned to the central server, the central server receives the results of multiple clients to perform federated learning to train the global model; the client actively learns the local unlabeled data based on the global model parameters obtained after training, expands the local labeled data set, and the global model again performs federated learning on the expanded labeled data set of multiple clients.

[0006] A multi-party collaborative data learning model training method specifically includes the following steps:

[0007] 1) The central server initially randomly generates the global model parameter W and sends it to all clients;

[0008] 2) Each client receives the global model parameters W sent by the central server and loads them into the local model, and uses the local model to analyze the local labeled dataset D L Perform model inference on all samples, record the predicted loss value of this round of learning and upload it to the central server;

[0009] 3) The central server receives the set V of predicted loss values ​​uploaded by all clients, performs linear numerical mapping on all elements in the set V so that their sum is 1, and selects the clients participating in the next round of federated learning according to the size after numerical mapping as the probability distribution. The number of selected clients accounts for half of the number of clients.

[0010] 4) The selected client uses the local labeled dataset D L Train the local model and upload the updated local model parameters to the central server;

[0011] 5) The central server receives the local model parameters uploaded by the selected client, uses the model aggregation algorithm to update the global model parameters W, and sends it to all clients;

[0012] 6) Iterative training, repeat 2)-5) until a certain number of training times are met to complete the training of the federated learning global model;

[0013] 7) Each client receives the updated parameters W after global model training sent by the central server * , and load it into the local model for active learning. The local model is used to select the unlabeled samples with the highest information gain, and experts are requested to label the samples.

[0014] 8) Return to step 6) for the supplemented labeled data set and perform global model training again until the labeled data set cannot be expanded.

[0015] Furthermore, the calculation formula for the predicted loss value in step 2) is:

[0016]

[0017] Where V i t represents the predicted loss value of the tth round of federated learning for the i-th client, t represents the tth round of federated learning training, i represents the i-th client, and n i Indicates that the i-th client has currently labeled the dataset D L The number of samples in , l(·) represents the loss function, x k ,y k Represents the kth sample and sample label respectively, W local Represents local model parameters.

[0018] Furthermore, the calculation formula of the linear numerical mapping in step 3) is:

[0019]

[0020] Among them, newV i t Indicates V i t The value after linear numerical mapping, It represents the sum of all client predicted loss values. The predicted loss value after linear numerical mapping is used as a discrete probability distribution for client sampling. The higher the predicted loss value, the greater the probability that the client will be sampled.

[0021] Furthermore, the model aggregation algorithm formula in step 5) is:

[0022]

[0023] Where h is the number of currently selected clients. Indicates the local model parameters uploaded by the i-th client among the selected clients.

[0024] Furthermore, the local model active learning method in step 7) is as follows:

[0025] 7.1) In the local model architecture during global model training, two additional auxiliary classifiers are added to the local model backbone network and run in parallel with the main classifier of the local model to form a local active learning model;

[0026] 7.2) Using the labeled dataset D L And the unlabeled dataset D U Train the local active learning model;

[0027] 7.3) The difference loss function is used as the objective function to train and maximize the difference between the auxiliary classifiers to obtain a tighter decision boundary, thereby selecting highly informative samples from the unlabeled samples and adding them to the labeled data set.

[0028] Furthermore, the two additional auxiliary classifiers are exactly the same as the network architecture of the main classifier. The network parameters are generated by adding random Gaussian noise to the network parameters of the main classifier. The added random Gaussian noise is p~N(0,0.1). The feature maps obtained after the data samples pass through the backbone network enter the main classifier and the auxiliary classifier respectively, and the classifiers do not affect each other.

[0029] Furthermore, the training method of the local active learning model in step 7.2) is as follows:

[0030] θ represents the backbone network and the main classifier, b represents the backbone network, θ1 and θ2 represent the two auxiliary classifiers, p represents the probability distribution of the sample output after θ, p1 represents the probability distribution of the sample output after (b, θ1), and p2 represents the probability distribution of the sample output after (b, θ2);

[0031] A: Use labeled datasets to train local active learning models;

[0032] A-1: Calculate the cross entropy loss L generated by the sample after reasoning with θ, (b, θ1), (b, θ2) CE :

[0033] The calculation formula of the cross entropy loss function is:

[0034]

[0035] Where C is the total number of sample categories, c represents the sample category, 1 represents the indicator function, and p c (y|x) represents the probability that sample x belongs to class c;

[0036] A-2: Update the local active learning model parameters, where η is the learning rate, is the gradient:

[0037]

[0038]

[0039]

[0040] B: Use unlabeled datasets to train auxiliary classifiers;

[0041] B-1: Calculate the difference loss L generated by the sample after (b, θ1) and (b, θ2) reasoningdist :

[0042] L dist =d(p1,p2)+d(p1,p)+d(p2,p),

[0043]

[0044] B-2: Update the auxiliary classifier parameters:

[0045]

[0046]

[0047] The beneficial effects of the present invention are as follows: the multi-party collaborative data learning system and learning model training method of the present invention fully utilize a small amount of labeled data sets of all clients through federated learning to guide sample sampling for active learning of a single client; in federated learning training, the predicted loss value is used as a probability distribution for sampling training of the client, which can accelerate model convergence and reduce communication volume; in active learning, by maximizing the difference between auxiliary classifiers, a tighter decision boundary can be obtained, thereby effectively selecting highly informative samples; by utilizing the advantages of active learning and federated learning, a large amount of unlabeled data of each participant can be fully utilized for collaborative training of the model under the premise of protecting data privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the architecture of the multi-party collaborative data learning system of the present invention;

[0049] Figure 2 It is a flowchart of the multi-party collaborative data learning model training method of the present invention;

[0050] Figure 3 Schematic diagram of sample sampling strategy in the multi-party collaborative data learning system of the present invention. DETAILED DESCRIPTION

[0051] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0052] like Figure 1The architecture diagram of the multi-party collaborative data learning system shown in the figure shows that the local client loads the global model parameters into the local model, uses the local label data to perform local model inference, obtains the predicted loss value and sends it back to the central server for the central server to select the clients participating in the next round of federated learning; the selected client uses the local labeled data to train the local model, and then uploads the local model parameters to the central server. The central server aggregates the model parameters of the selected clients to obtain the updated global model, and sends the updated global model parameters to all clients for the next round of federated learning. When the global model converges, the federated learning training ends. The client loads the trained global model parameters into the local model, uses the local model to actively learn the local unlabeled data, expands the local labeled data set, and then performs federated learning.

[0053] like Figure 2 The flowchart of the multi-party collaborative data learning model training method is shown in FIG. Figure 2 , the specific methods of implementing the multi-party active federated learning system and training the learning model are further explained, which specifically include the following steps:

[0054] S100, the central server initially randomly generates global model parameters W and sends them to all clients;

[0055] S200, each client receives the global model parameter W sent by the central server and loads it into the local model, and uses the local model to analyze the local labeled data set D L Perform a model inference on all samples and record the predicted loss value V of this round of federated learning i t And upload to the central server;

[0056] Specifically, the predicted loss value V i t The calculation formula is:

[0057]

[0058] Where t represents the tth round of federated learning training, i represents the i-th client, and n i Indicates that the i-th client has currently labeled the dataset D L The number of samples in , l(·) represents the loss function, x k ,y k Represents the kth sample and sample label respectively, W local Represents local model parameters.

[0059] Each client will store a small number of labeled data sets D locally. L And a large number of unlabeled datasets D UThe reason why the denominator of the calculation formula for the prediction loss value is the square root of the number of labeled samples is that, on the one hand, it is biased towards clients with more labeled data samples; on the other hand, it reduces the impact that may be caused by a small number of samples in a client's labeled data set and the presence of a large amount of noise data.

[0060] S300: The central server receives a set of predicted loss values ​​V uploaded by all clients. m represents the number of clients. Linear numerical mapping is performed on all elements in the predicted loss value set V so that their sum is 1. The size after numerical mapping is used as the probability distribution to select the clients to participate in the next round of federated learning. The number of selected clients accounts for half of the number of clients.

[0061] Specifically, the calculation formula for linear numerical mapping is:

[0062]

[0063] Among them, newV i t Indicates V i t The value after linear numerical mapping, Represents the sum of all client prediction loss values.

[0064] The benefit of this step is that the central server maps the predicted loss value to the linear value. Client sampling is performed as a discrete probability distribution. The higher the predicted loss value, the greater the probability that the client will be sampled. First, the local labeled dataset of the client with a higher loss prediction value is more helpful for the training of the current global model, which can accelerate the convergence of the global model. Secondly, in each round of federated learning training, since not all clients participate in the training, the communication volume can be reduced.

[0065] S400, the selected client uses the local labeled dataset D L Train the local model and update the local model parameters W local Upload to the central server;

[0066] S500: The central server receives the local model parameters uploaded by the selected client, updates the global model parameters W using the model aggregation algorithm, and sends the updated global model parameters to all clients.

[0067] Specifically, the algorithm formula for aggregating the global model is:

[0068]

[0069] Where h is the number of currently selected clients. Indicates the local model parameters uploaded by the i-th client among the selected clients.

[0070] S600, iterative training, repeating S200-S500 until a certain number of training times is met, completing the training of the federated learning global model;

[0071] The benefit of this step is that repeating S200-S500 can make full use of the information of the existing labeled data samples of each client. The global model obtained through federated learning training has better performance than the local model trained by a single client using a small number of labeled samples. Using the global model to guide each client to perform active learning can better query samples that have a benefit to the model performance.

[0072] S700, each client receives the updated parameters W after global model training sent by the central server * , and load it into the local model for active learning. The local model is used to select a portion of unlabeled samples with the highest information gain, and experts are requested to label the samples. Figure 3 As shown, each client in S700 performs active learning including the following steps:

[0073] S700-1. Modify the local model architecture. Add two auxiliary classifier modules, Classifier1 and Classifier2, to the main classifier module and connect them to the backbone network module to establish a local active learning model.

[0074] S700-2. Using the labeled dataset D L And the unlabeled dataset D U Train the local active learning model;

[0075] S700-3. Using local active learning models from unlabeled datasets D U Select a part of the unlabeled samples with the highest information gain and move them to the labeled dataset D L .

[0076] Specifically, in sub-step S700-1 of step S700, two additional auxiliary classifiers Classifier1 and Classifier2 are added with the same network architecture as the main classifier module, and the network parameters are generated by adding random Gaussian noise to the network parameters of the main classifier. The added random Gaussian noise p~N(0,0.1), the feature map obtained after the data sample passes through the backbone network module will enter the main classifier and the auxiliary classifier respectively, and the classifiers do not affect each other.

[0077] Specifically, in sub-step S700-2 of step S700, θ represents the backbone network module and the main classifier module, b represents the backbone network, θ1 and θ2 represent the two auxiliary classifier modules, p represents the probability distribution of the sample output after θ, p1 represents the probability distribution of the sample output after (b, θ1), and p2 represents the probability distribution of the sample output after (b, θ2). The training method of the local active learning model is as follows: 1. The local active learning model is trained using the labeled data set;

[0078] (1) Calculate the cross entropy loss L generated by the sample after reasoning with θ, (b, θ1), and (b, θ2) CE : The calculation formula of the cross entropy loss function is:

[0079]

[0080] Where C is the total number of sample categories, c represents the sample category, 1 represents the indicator function, and p c (y|x) represents the probability that sample x belongs to class c.

[0081] (2) Update the parameters of the local active learning model, where η is the learning rate, is the gradient:

[0082]

[0083]

[0084]

[0085] 2. Use unlabeled data sets to train auxiliary classifiers;

[0086] (1) Calculate the difference loss L generated by the sample after (b, θ1) and (b, θ2) reasoning dist :

[0087] The calculation formula of the difference loss function is:

[0088] L dist =d(p1,p2)+d(p1,p)+d(p2,p),

[0089]

[0090] (2) Update the auxiliary classifier parameters:

[0091]

[0092]

[0093] The auxiliary classifier is exactly the same as the main classifier in terms of model architecture, except that:

[0094] (1) Different parameters. The parameters of the auxiliary classifier are obtained by adding random Gaussian noise to the parameters of the main classifier, so the parameters of the three classifiers are different.

[0095] (2) The present invention can be briefly described as three stages: the initial federated learning stage, the local active learning stage, and the federated learning stage after the local labeled data set is expanded. The parameters of the main classifier are obtained through the initial federated learning training. In the local active learning stage, after adding two auxiliary classifiers to the local model, the local model needs to be trained. The parameters of the main classifier are only updated when the labeled data set is used for training.

[0096]

[0097] However, the parameters of the two auxiliary classifiers are not only updated when training with the labeled dataset:

[0098]

[0099]

[0100] It is also updated when training with an unlabeled dataset:

[0101]

[0102]

[0103] The auxiliary classifier uses the unlabeled dataset to update its parameters. Its purpose is to use the difference loss function as the objective function to train to maximize the difference between the auxiliary classifiers, obtain a tighter decision boundary, and thus select highly informative samples to be added to the labeled dataset.

[0104] Specifically, in sub-step S700-3 of step S700, the unlabeled dataset D is sorted in descending order according to the numerical value of F(x). U Select a part of the unlabeled samples with the highest information gain and move them to the labeled dataset D L , the calculation formula of F(x) is:

[0105] F(x)=d(p1(x),p2(x)).

[0106] The benefit of this step is that two auxiliary classifiers are added to the local model. Since their architecture is the same as the main classifier, the implementation is simple. The difference loss function is used as the objective function to train the maximum difference between the auxiliary classifiers, which can obtain a tighter decision boundary and effectively select highly informative samples.

[0107] S800, returning to step S600 to perform global model training again on the supplemented labeled data set until the labeled data set cannot be expanded.

[0108] The benefit of this step is that after S700, the labeled data sets of each client are further expanded, and then the federated learning training can obtain a model with better performance.

[0109] In this embodiment, if Figure 2 As shown, an active federated learning model training system implemented based on the above method includes a central server and several participant devices, and the participant devices store a large amount of unlabeled data sets. In addition, each participant will have relevant field experts who can perform high-quality labeling work on data samples.

[0110] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A multi-party collaborative data learning system, characterized in that: It includes a central server for deploying a global model and multiple clients with local classification models. The central server sends model parameters, and the clients use local labeled data for inference and training. The inference and training results are returned to the central server. The central server receives the results from multiple clients for federated learning to train the global model. The client actively learns the local unlabeled data based on the trained global model parameters and expands the local labeled data set. The global model performs federated learning again on the expanded labeled data set of multiple clients. The specific steps are as follows: 1) The central server initially randomly generates the global model parameter W and sends it to all clients; 2) Each client receives the global model parameters W sent by the central server and loads them into the local model, and uses the local model to analyze the local labeled dataset D L Perform model inference on all samples, record the predicted loss value of this round of learning and upload it to the central server; 3) The central server receives the set V of predicted loss values ​​uploaded by all clients, performs linear numerical mapping on all elements in the set V so that their sum is 1, and selects the clients participating in the next round of federated learning according to the size after numerical mapping as the probability distribution. The number of selected clients accounts for half of the number of clients. 4) The selected client uses the local labeled dataset D L Train the local model and upload the updated local model parameters to the central server; 5) The central server receives the local model parameters uploaded by the selected client, uses the model aggregation algorithm to update the global model parameters W, and sends it to all clients; 6) Iterative training, repeat 2)-5) until a certain number of training times are met to complete the training of the federated learning global model; 7) Each client receives the updated parameters W after global model training sent by the central server * , and load it into the local model for active learning. The local model is used to select the unlabeled samples with the highest information gain, and experts are requested to label the samples. 8) Return to step 6) for the supplemented labeled data set and perform global model training again until the labeled data set cannot be expanded.

2. A multi-party collaborative data learning model training method, characterized in that: The specific steps include: 1) The central server initially randomly generates the global model parameter W and sends it to all clients; 2) Each client receives the global model parameters W sent by the central server and loads them into the local model, and uses the local model to analyze the local labeled dataset D L Perform model inference on all samples, record the predicted loss value of this round of learning and upload it to the central server; 3) The central server receives the set V of predicted loss values ​​uploaded by all clients, performs linear numerical mapping on all elements in the set V so that their sum is 1, and selects the clients participating in the next round of federated learning according to the size after numerical mapping as the probability distribution. The number of selected clients accounts for half of the number of clients. 4) The selected client uses the local labeled dataset D L Train the local model and upload the updated local model parameters to the central server; 5) The central server receives the local model parameters uploaded by the selected client, uses the model aggregation algorithm to update the global model parameters W, and sends it to all clients; 6) Iterative training, repeat 2)-5) until a certain number of training times are met to complete the training of the federated learning global model; 7) Each client receives the updated parameters W after global model training sent by the central server * , and load it into the local model for active learning. The local model is used to select the unlabeled samples with the highest information gain, and experts are requested to label the samples. 8) Return to step 6) for the supplemented labeled data set and perform global model training again until the labeled data set cannot be expanded.

3. The multi-party collaborative data learning model training method according to claim 2, characterized in that: The calculation formula of the predicted loss value in step 2) is: Where V i t represents the predicted loss value of the tth round of federated learning for the i-th client, t represents the tth round of federated learning training, i represents the i-th client, and n i Indicates that the i-th client has currently labeled the dataset D L The number of samples in , l(·) represents the loss function, x k ,y k Represents the kth sample and sample label respectively, W local Represents local model parameters.

4. The multi-party collaborative data learning model training method according to claim 3 is characterized in that: The calculation formula of the linear numerical mapping in step 3) is: Among them, newV i t Indicates V i t The value after linear numerical mapping, It represents the sum of all client predicted loss values. The predicted loss value after linear numerical mapping is used as a discrete probability distribution for client sampling. The higher the predicted loss value, the greater the probability that the client will be sampled.

5. The multi-party collaborative data learning model training method according to any one of claims 2 to 4, characterized in that: The model aggregation algorithm formula in step 5) is: Where h is the number of currently selected clients. Indicates the local model parameters uploaded by the i-th client among the selected clients.

6. The multi-party collaborative data learning model training method according to claim 5, characterized in that: The local model active learning method in step 7) is as follows: 7.1) In the local model architecture during global model training, two additional auxiliary classifiers are added to the local model backbone network and run in parallel with the main classifier of the local model to form a local active learning model; 7.2) Using the labeled dataset D L And the unlabeled dataset D U Train the local active learning model; 7.3) The difference loss function is used as the objective function to train and maximize the difference between the auxiliary classifiers to obtain a tighter decision boundary, thereby selecting highly informative samples from the unlabeled samples and adding them to the labeled data set.

7. The multi-party collaborative data learning model training method according to claim 6, characterized in that: The two additional auxiliary classifiers have the same network architecture as the main classifier. The network parameters are generated by adding random Gaussian noise to the network parameters of the main classifier. The added random Gaussian noise is p~N(0,0.1). The feature maps obtained after the data samples pass through the backbone network enter the main classifier and the auxiliary classifier respectively, and the classifiers do not affect each other.

8. The multi-party collaborative data learning model training method according to claim 7, characterized in that: The training method of the local active learning model in step 7.2) is as follows: θ represents the backbone network and the main classifier, b represents the backbone network, θ1 and θ2 represent the two auxiliary classifiers, p represents the probability distribution of the sample output after θ, p1 represents the probability distribution of the sample output after (b, θ1), and p2 represents the probability distribution of the sample output after (b, θ2); A: Use the labeled dataset to train the local active learning model; A-1: Calculate the cross entropy loss L generated by the sample after reasoning with θ, (b, θ1), (b, θ2) CE : The calculation formula of the cross entropy loss function is: Where C is the total number of sample categories, c represents the sample category, 1 represents the indicator function, and p c (y|x) represents the probability that sample x belongs to class c; A-2: Update the local active learning model parameters, where η is the learning rate, is the gradient: B: Use unlabeled datasets to train auxiliary classifiers; B-1: Calculate the difference loss L generated by the sample after (b, θ1) and (b, θ2) reasoning dist : L dist =d(p1,p2)+d(p1,p)+d(p2,p), B-2: Update the auxiliary classifier parameters:

Images