A method for federated learning membership inference based on prediction confidence sequence

Abstract

The present invention provides a method for inferring federated learning members based on prediction confidence sequences, and belongs to the field of artificial intelligence security. The method aims to reveal the privacy risks of federated learning, and solves the technical problem of how attackers of different roles infer whether a given data sample is a member of the training set of the participant in a federated learning scenario. The present invention utilizes the difference in prediction confidence sequences between the training data and the test data of the participants in federated learning, establishes an attack model to learn this difference, and thus implements member inference. The present invention generates labeled prediction confidence sequence samples through a shadow model and a data set owned by the attacker, and trains an attack model; for a given target sample, a corresponding prediction confidence sequence is generated using the target model, and the trained attack model is used to infer its member information. The member inference method provided by the present invention supports malicious participants or malicious servers to steal the training data privacy of normal participants.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security, and in particular to a federated learning member inference method based on prediction confidence sequence. Background Art

[0002] With the emergence of data islands and the emphasis on personal privacy protection, the application model of centralized learning has been restricted. Federated learning, as a distributed machine learning framework, can complete model training without leaking user data. It has attracted much attention since its birth. The architecture of federated learning includes two types of roles: multiple participants and an aggregation server. Each participant has a data set with complete data features, and there is no intersection or very small intersection between the data samples owned by different participants. They can work together to train a global model. The specific training process is as follows: Federated learning includes multiple rounds of iterations. In each round of iteration, the aggregation server sends the current global model to the participants. The participants use local data to train the global model, calculate new local model parameters, and send them to the server. The server aggregates the received local model parameters of the participants to form a new global model for the next round of iteration. During the iterative training process, the training data is always stored locally on the participants, and the participants and the server only communicate model parameters, thereby ensuring the data privacy of the participants. However, the model parameters communicated during the communication process still have the problem of privacy leakage. Various privacy attacks, including member inference, seriously undermine the confidentiality of federated learning.

[0003] In federated learning, membership inference attacks are used to infer whether a given target data sample is a member of the training data set of a normal participant. It is essentially a binary classification problem. A data sample is input and an inference label of 1 or 0 is output. 1 indicates that the data sample is a member of the participant's training set, and 0 indicates that the sample is not a member. Existing membership inference attacks for federated learning can be divided into two categories according to the attacker's capabilities: black-box attacks and white-box attacks. Among them, black-box attacks refer to the attacker can only submit data records to the target model and obtain the output results of the model, but cannot know other information about the target model. Therefore, the model is usually used to infer the difference between the output of training and test data. In white-box attacks, the attacker can obtain the structure, parameters and intermediate operation results of the model, and can use the gradient and intermediate result differences of the training and test data at each layer of the target model for inference. However, existing membership inference attacks cannot be applied to different application scenarios of federated learning. When the attacker is a participant in federated learning, the existing black-box attack is less effective because the aggregation algorithm of federated learning will reduce the contribution of each participant's training data; and the existing white-box attack can only be effective in federated learning applications with a small number of participants, and cannot be applied to scenarios with dozens of participants. Summary of the invention

[0004] The present invention provides a method for inferring federated learning members based on prediction confidence sequences, and designs different attack methods according to the role of attackers in the federated learning system, which can be divided into local member inference based on prediction confidence sequences and global member inference based on prediction confidence sequences. The local member inference based on prediction confidence sequences is implemented by malicious participants, and the attacker is one of the participants in the federated learning; while the global member inference based on prediction confidence sequences is initiated by malicious servers, and the attacker is an aggregation server of the federated learning. The member inference method provided by the present invention supports different types of attackers to infer the privacy data of normal participants in a federated learning environment, thereby revealing the privacy risks of federated learning.

[0005] The method for inferring federated learning members based on prediction confidence sequence provided by the present invention comprises the following steps:

[0006] (1) Before federated learning begins, the attacker divides his dataset into two parts: a member subset and a non-member subset;

[0007] (2) In each iteration of federated learning, the attacker integrates the member subset into the training of federated learning, saves the shadow model and target model of this iteration, and participates in the training of the global model at the same time;

[0008] (3) After the federated learning is completed, the attacker selects a member to infer the structure of the attack model, uses the saved multiple shadow models to generate a prediction confidence sequence for each sample in the member subset and non-member subset, marks the corresponding label 1 or 0, and uses the generated data to train the attack model;

[0009] (4) The attacker uses multiple saved target models to generate a prediction confidence sequence for a given target sample, and then asks the members to infer the attack model to infer whether the target sample is a member of the training set of the normal participant.

[0010] Furthermore, the specific content of step (2) varies depending on the role of the attacker, including: for local member inference, the attacker first saves a copy of the received global model as the shadow model and target model of this iteration, then uses the member subset to train the global model, calculates the new local model parameters, and uploads them to the server. For global member inference, the attacker first trains the global model with the member subset, uses the calculated local model as the shadow model, and uses the model uploaded by the target participant as the target model, then the attacker aggregates the local model and all the participant models received to form a new global model and sends it to the participants.

[0011] Furthermore, step (3) trains members to infer the attack model, which specifically includes the following contents:

[0012] (3.1) The attacker selects a binary classification neural network model and determines the specific structure and hyperparameters of the model as the member inference attack model;

[0013] (3.2) For each sample (x, y) in the member subset, the attacker calculates the prediction confidence of each shadow model in the shadow model sequence for (x, y) to form a prediction confidence sequence of (x, y) and marks it with a label of 1 as a training sample for the attack model; for each sample in the non-member subset, the attacker also calculates the prediction confidence sequence of the shadow model for the sample and marks it with a label of 0 as a training sample for the attack model;

[0014] (3.3) The attacker uses the attack model training samples generated in step (3.2) to train the member inference model determined in step (3.1).

[0015] Furthermore, in step (4), the attacker calculates the prediction confidence sequence of the target sample in the target model and uses it as the input of the membership inference model trained in step (3.3) to obtain the membership inference result of the target sample: 1 or 0.

[0016] The advantages of the present invention are:

[0017] (1) The present invention can use different attack methods for different roles of attackers in the federated learning system.

[0018] (2) The present invention can achieve good attack effects in federated learning applications involving dozens of participants. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a federated learning member inference method based on prediction confidence sequence;

[0020] Figure 2 A flowchart of a global membership inference method based on a prediction confidence sequence;

[0021] Figure 3 Local membership inference on CIFAR-10, CIFAR-100, Purchase100, and Texas100 datasets;

[0022] Figure 4 Global membership inference on CIFAR-10, CIFAR-100, Purchase100, and Texas100 datasets;

[0023] Figure 5 Comparison of local membership inference performance on the Purchase100 dataset with different total number of participants. DETAILED DESCRIPTION

[0024] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0025] The present invention mainly utilizes two characteristics of federated learning: (1) the deep learning model has different prediction confidence for training data and test data; (2) federated learning involves multiple rounds of iterative training, and attackers have access to multiple versions of the model. With the iteration of federated learning, the prediction confidence of training and test data will show different trends. Although the prediction confidence of both will continue to rise, the increase in training data is greater. The present invention establishes an attack model to learn the difference in the prediction confidence sequence of training and test data, thereby inferring whether a given target sample is a member of the training set of the normal participant.

[0026] The present invention provides a method for federated learning member inference based on prediction confidence sequence, and designs different attack methods according to the role of the attacker in the federated learning system, which can be divided into local member inference based on prediction confidence sequence and global member inference based on prediction confidence sequence. The local member inference based on prediction confidence sequence is implemented by a malicious participant, and the attacker is one of the participants in the federated learning. He can infer whether a data sample is a member of the training set of other participants, but cannot clearly point out which specific participant it is. The global member inference based on prediction confidence sequence is initiated by a malicious server, and the attacker is an aggregation server of federated learning, and can implement member inference for specific target participants. Figure 1 As shown, the member inference method provided by the present invention specifically includes the following steps:

[0027] Step 1: Before federated learning begins, the attacker divides his own dataset into two parts: member subset and non-member subset. If the attacker is a participant in the federated learning system, he can directly divide the local private data; if the attacker is an aggregation server, he needs to collect an auxiliary dataset in advance, which is required to have the same distribution as the training set of the participant. Each sample (x, y) in the dataset contains sample feature x and label y.

[0028] Step 2: In each iteration of federated learning, the attacker needs to integrate the member subset into the training of federated learning and save the shadow model M of this iteration. s and the target model M t , and participate in the training of the global model at the same time. Different operations need to be performed according to the attacker's role in the federated learning system, as follows:

[0029] If the attacker is a participant, then in each round of iteration, the attacker will receive the global model sent by the server. The attacker first saves a copy of the global model as the shadow model and target model for this round of iteration; then uses the member subset to train the global model, calculates the new local model parameters, and uploads them to the server. After aggregating the model parameters uploaded by the participants, the server will form a new global model and enter the next round of iteration.

[0030] If the attacker is a server, then in each round of iteration the attacker will receive the model parameters sent by the participants. Figure 2 As shown in Figure 1, the attacker first trains the current global model with a subset of members, calculates the local model parameters, and uses the local model as the shadow model for this round of iteration, while the model uploaded by the target participant is used as the target model. Afterwards, the attacker aggregates the local model and all the participant models received to form a new global model and sends it to the participants to enter the next round of iteration.

[0031] The above step 2 is a cyclic iterative process. Assuming that the training process of federated learning includes r rounds of iterations, the shadow model sequence and target model sequence finally saved by the attacker contain r models respectively.

[0032] Step 3: After the federated learning is completed, the attacker selects a binary classification neural network model, determines the specific structure and hyperparameters of the model, and uses it as the member inference attack model to start the training phase. For each sample (x, y) in the member subset, the attacker calculates the prediction confidence of each shadow model pair (x, y) in the shadow model sequence, where the i-th shadow model Confidence of prediction for (x,y) i The calculation method is as follows:

[0033]

[0034] in For model M i Infer the posterior probability that the sample feature x belongs to the true label y, For model M i Infer the posterior probability that the sample feature x belongs to category k. After the calculation is completed, the attacker obtains the prediction confidence sequence {Conf1,Conf2,...,Conf r}, and marked with label 1, as a training sample of the attack model ({Conf1,Conf2,...,Conf r},1). For each sample in the non-member subset, the attacker also calculates the prediction confidence sequence of the shadow model for the sample and labels it with 0. The attacker uses the set of all labeled prediction confidence sequences generated by the member subset and the non-member subset as the training data set of the attack model to train the attack model.

[0035] Step 4: After the attack model training is completed, the attacker can enter the inference phase. Given a target sample (x t ,y t ), the attacker uses the method in step 3 to calculate each target model M in the target model sequence t i Yes (x t ,y t ) prediction confidence, forming (x t ,y t ) is used as the prediction confidence sequence of the attack model. The attack model finally outputs (x t ,y t ) member inference result: 1 or 0, 1 means (x t ,y t ) is a member of the training set of normal participants, and 0 means (x t ,y t ) is not a member.

[0036] In order to fully illustrate the member inference method proposed in the present invention, the attack performance is evaluated by the following two indicators: the accuracy of member inference and the F1 score. An equal amount of data samples are selected from the training set and test set of the normal participants as the test data set of the member inference model, and the accuracy and F1 score of the member inference are calculated according to the inference results of the attack model on the test set. This embodiment uses Python 3.7 and PyTorch 1.3.1 to implement the federated learning member inference method based on the prediction confidence sequence. The configuration of the server running all experiments is as follows: Intel Golden 6240CPU, 384GB memory and an NVIDIA V100 GPU with 32GB GDDR5X.

[0037] The experimental scenario of this embodiment uses CIFAR-10, CIFAR-100, Purchase100 and Texas100 datasets. For the CIFAR-10 and CIFAR-100 datasets, this embodiment uses the Alexnet model as the model for federated learning training; and for the Purchase100 and Texas100 datasets, this embodiment uses a fully connected network with four hidden layers for training, and the four hidden layers have 1024, 512, 256 and 128 units respectively. In addition, this embodiment uses a hidden layer with 64 units and a Sigmoid layer to construct a member inference attack model.

[0038] This embodiment sets 5 participants to evaluate the local member inference effect and global member inference effect of the above data set. In the local member inference experiment, one participant is an attacker and the other four participants are normal users. This embodiment sets 50 rounds of federated learning iterations. Figure 3 As shown in Figure 2, the accuracy of local membership inference on the CIFAR-10, CIFAR-100, Purchase100, and Texas100 datasets reached 99.954%, 99.958%, 98.488%, and 99.748%, respectively, and the F1 scores were 0.99954, 0.99958, 0.98495, and 0.99748, respectively. The experiment proves that the local membership inference method based on the prediction confidence sequence can enable malicious parties to effectively infer member records and non-member records in the target dataset. The results of the global membership inference experiment are shown in Figure 2. Figure 4 As shown in the figure, the accuracy of global membership inference on the four data sets reached 99.934%, 99.996%, 94.15% and 100% respectively, and the F1 scores were 0.99934, 0.9999, 0.93787 and 1 respectively, proving that the malicious server can also successfully infer whether the sample is a member of the participant's training set and complete the membership inference.

[0039] To further evaluate the member inference method provided by the present invention, this example compares the local member inference effect of the Purchase100 dataset with different total number of participants. The experiment tests the accuracy and F1 score of local member inference with {10, 20, ..., 100} participants in turn. Figure 5 As shown in the figure, the effect of local member inference gradually decreases with the increase of the total number of participants. When the federated learning contains 70 participants, the accuracy of local member inference can reach 59.907%. The attacker can still try to infer the member samples of the training set of normal participants.

Claims

1. A method for inferring federated learning members based on prediction confidence sequence, comprising the following steps: (1) Before federated learning begins, the attacker divides his dataset into two parts: a member subset and a non-member subset; (2) In each iteration of federated learning, the attacker integrates the member subset into the training of federated learning, saves the shadow model and target model of this iteration, and participates in the training of the global model. Different operations are performed according to the attacker's role in the federated learning system, as follows: If the attacker is one of the participants in federated learning, the attacker first saves a copy of the received global model as the shadow model and target model for this round of iteration, then uses the member subset to train the global model, calculates the new local model parameters, and uploads them to the server. After aggregating the model parameters uploaded by the participants, the server forms a new global model and enters the next round of iteration; If the attacker is a federated learning server, the attacker first trains the current global model with a member subset, calculates the local model parameters, uses the local model as the shadow model for this round of iteration, and uses the model uploaded by the target participant as the target model. The attacker then aggregates the local model with all the participant models received to form a new global model and sends it to the participants to enter the next round of iteration. The above step (2) is a cyclic iterative process. Assuming that the training process of federated learning includes r rounds of iterations, the shadow model sequence and target model sequence finally saved by the attacker contain r models respectively. (3) After the federated learning is completed, the attacker selects a member to infer the structure of the attack model, uses the saved multiple shadow models to generate a prediction confidence sequence for each sample in the member subset and non-member subset, marks the corresponding label 1 or 0, and uses the generated data to train the attack model; (4) The attacker uses multiple saved target models to generate a prediction confidence sequence for a given target sample, and then asks the members to infer the attack model to infer whether the target sample is a member of the training set of the normal participant.

2. The method according to claim 1, characterized in that The step (3) specifically includes the following contents: (3.1) The attacker selects a binary classification neural network model and determines the specific structure and hyperparameters of the model as the member inference attack model; (3.2) For each sample (x, y) in the member subset, the attacker calculates the prediction confidence of each shadow model in the shadow model sequence for (x, y) to form a prediction confidence sequence of (x, y) and marks it with a label of 1 as a training sample for the attack model; for each sample in the non-member subset, the attacker also calculates the prediction confidence sequence of the shadow model for the sample and marks it with a label of 0 as a training sample for the attack model; (3.3) The attacker uses the attack model training samples generated in step (3.2) to train the member inference model determined in step (3.1).

Images