Member inference attack method and device in federated learning

Abstract

The application relates to a member inference attack method and device in federated learning. The method comprises the following steps: determining a key round K of an attacker in a training process of a target model; inputting a target sample into the target model to obtain sample features; the sample features comprise loss values of the target sample at the key round K of the target model and at the last round, a real label, a parameter gradient and output; assigning weights to the loss values of the target sample at the key round K of the target model and at the last round; inputting the sample features with the assigned weights into an attack model to output a probability value of the target sample being a member sample, so as to determine whether the target sample is a member sample. The application adopts overfitting feature perception, finds a key round K capable of distinguishing similar members and non-members in the training process of the target model, assigns weights of the key round and the last round to different samples by using a Transformer model, and improves attack accuracy.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and more specifically, to a method and apparatus for member reasoning attacks in federated learning. Background Technology

[0002] Federated learning, used to protect user privacy, is essentially a distributed machine learning framework. It enables several participants to share data and collaboratively model a model, achieving a "data usable but not visible" data application model while ensuring data privacy, security, and legal compliance. In federated learning, several participants, each with their own local training set, collaboratively train a model under the control of a central server. Specifically, each participant first trains a local model using their local dataset and then uploads their local model parameters to the server. The server aggregates the local model parameters received from the participants, updates them using an average aggregation algorithm, and obtains the final local model parameters. Each participant downloads their local model parameters and updates them on their private dataset; this process continues until the local model converges. However, federated learning models remain vulnerable to various privacy attacks, such as member inference attacks.

[0003] Existing member inference attack algorithms can be broadly categorized into two types: metric-based and neural network-based. Metric-based methods calculate a sample's metric based on its output on the target model and certain rules, then set a threshold to determine whether a sample is a member. For example, in prediction entropy-based attacks, the prediction entropy of a sample is first calculated, and then a threshold is set; samples below the threshold are considered members. Neural network-based methods train an attack model to distinguish between members and non-members. The attacker inputs the features of the target sample into the attack model, which predicts the member probability of the target sample. Both methods ignore certain implicit features of members and non-members. For member and non-member data that behave similarly on the target model, the attack accuracy is only close to random guessing, failing to accurately distinguish data with these features. Summary of the Invention

[0004] To overcome at least one deficiency in the prior art, this application provides a method and apparatus for member reasoning attacks in federated learning.

[0005] Firstly, a method for member reasoning attacks in federated learning is provided, including:

[0006] Determine the critical rounds K in the training process of the target model by the attacker; the critical rounds K are the rounds in which the performance difference between member samples and non-member samples on the target model is the greatest.

[0007] Input the target sample into the target model to obtain sample features; the sample features include the loss value of the target sample in the key K rounds of the target model and the loss value in the last round, the true label, the parameter gradient and the output;

[0008] Assign weights to the loss values ​​of the target samples in the key rounds K of the target model and in the last round;

[0009] The weighted sample features are input into the attack model, and the probability value of the target sample being a member sample is output.

[0010] Determine whether the target sample is a member sample based on the probability value.

[0011] In one embodiment, the key round K of the attacker's training process on the target model is determined using the following formula:

[0012]

[0013] Wherein, the key round K is the value of K when ΔTr-ΔTe takes the maximum value, ΔTr is the difference between the key round K in the training process and the expected loss value of the last round, and ΔTe is the difference between the key round K in the testing process and the expected loss value of the last round.

[0014]

[0015]

[0016] Where l represents the last round, z represents the sample data, and L is the loss function of the target model. This represents the expected loss value of sample data z in training set D on target model A at round l. This represents the expected loss value of sample data z in training set D at key round K of target model A. This represents the expected loss value of sample data z in test set S on target model A at round l. It is the expected loss value of the sample data z in the test set S on the key round K of the target model A.

[0017] In one embodiment, the method further includes:

[0018] The loss values ​​of the target sample at key rounds K and at the last round of the target model are input into the Transformer model to obtain the weights of the loss values ​​of the target sample at key rounds K and at the last round of the target model.

[0019] In one embodiment, the sample features are input into the attack model, and the output is a probability value that the target sample is a member sample, including:

[0020] The attack model includes fully connected neural network components, convolutional neural network components, and an encoder;

[0021] The loss value of the target sample in the key K rounds of the target model, as well as the loss value in the last round, the true label, and the output are input into the fully connected neural network component;

[0022] The parameter gradient of the target sample in the last round is input into the convolutional neural network component;

[0023] The outputs of both the fully connected neural network component and the convolutional neural network component are input into the encoder, which outputs the probability value that the target sample is a member sample.

[0024] In one embodiment, the loss function of the attack model is:

[0025]

[0026] Where B represents the attack model, z represents the sample data, D represents the training set, and S represents the test set.

[0027] Secondly, a member reasoning attack device in federated learning is provided, comprising:

[0028] The key round determination module is used to determine the key round K in the training process of the target model by the attacker; the key round K is the round in which the difference in performance between member samples and non-member samples on the target model is the greatest.

[0029] The sample feature acquisition module is used to input the target sample into the target model and acquire the sample features. The sample features include the loss value of the target sample in the key K rounds of the target model and the loss value in the last round, the true label, the parameter gradient and the output.

[0030] The weight allocation module is used to assign weights to the loss values ​​of the target samples in the key rounds K of the target model and in the last round.

[0031] The probability value determination module is used to input the weighted sample features into the attack model and output the probability value that the target sample is a member sample.

[0032] The member sample determination module is used to determine whether a target sample is a member sample based on its probability value.

[0033] In one embodiment, the key round determination module is further configured to:

[0034] The critical round K of the attacker during the training process of the target model is determined using the following formula:

[0035]

[0036] Wherein, the key round K is the value of K when ΔTr-ΔTe takes the maximum value, ΔTr is the difference between the key round K in the training process and the expected loss value of the last round, and ΔTe is the difference between the key round K in the testing process and the expected loss value of the last round.

[0037]

[0038]

[0039] Where l represents the last round, z represents the sample data, and L is the loss function of the target model. This represents the expected loss value of sample data z in training set D on target model A at round l. This represents the expected loss value of sample data z in training set D at key round K of target model A. This represents the expected loss value of sample data z in test set S on target model A at round l. It is the expected loss value of the sample data z in the test set S on the key round K of the target model A.

[0040] In one embodiment, a weight determination module is further included, for:

[0041] The loss values ​​of the target sample at key rounds K and at the last round of the target model are input into the Transformer model to obtain the weights of the loss values ​​of the target sample at key rounds K and at the last round of the target model.

[0042] In one embodiment, the sample feature acquisition module is used for:

[0043] The attack model includes fully connected neural network components, convolutional neural network components, and an encoder;

[0044] The loss value of the target sample in the key K rounds of the target model, as well as the loss value in the last round, the true label, and the output are input into the fully connected neural network component;

[0045] The parameter gradient of the target sample in the last round is input into the convolutional neural network component;

[0046] The outputs of both the fully connected neural network component and the convolutional neural network component are input into the encoder, which outputs the probability value that the target sample is a member sample.

[0047] In one embodiment, the loss function of the attack model is:

[0048]

[0049] Where B represents the attack model, z represents the sample data, D represents the training set, and S represents the test set.

[0050] Compared with the prior art, this application has the following beneficial effects: This application uses overfitting feature perception to find a key round K that can distinguish members and non-members with similar performance during the training process of the target model, and uses the Transformer model to assign weights to the key round and the last round to different samples, thereby improving the attack accuracy. Attached Figure Description

[0051] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings:

[0052] Figure 1 A flowchart illustrating a member inference attack method in federated learning according to an embodiment of this application is shown;

[0053] Figure 2 A structural block diagram of a member inference attack apparatus in federated learning according to an embodiment of this application is shown;

[0054] Figure 3 The chart shows a comparison of the loss values ​​of the four datasets at various training epochs of the target model; where (a) represents the loss value of the CIFAR10-Alexnet dataset at various training epochs of the target model, (b) represents the loss value of the CIFAR100-Alexnet dataset at various training epochs of the target model, (c) represents the loss value of the Purchase100-FCN dataset at various training epochs of the target model, and (d) represents the loss value of the Texas100-FCN dataset at various training epochs of the target model.

[0055] Figure 4The graph shows a comparison of the average loss values ​​of members and non-members with similar performance in four datasets at various training epochs of the target model. (a) represents the average loss value of the CIFAR10-Alexnet dataset at various training epochs of the target model, (b) represents the average loss value of the CIFAR100-Alexnet dataset at various training epochs of the target model, (c) represents the average loss value of the Purchase100-FCN dataset at various training epochs of the target model, and (d) represents the average loss value of the Texas100-FCN dataset at various training epochs of the target model.

[0056] Figure 5 The following diagram shows the attack accuracy comparison of four datasets using the method of this application, where (a) represents the attack accuracy of the CIFAR10-Alexnet dataset, (b) represents the attack accuracy of the CIFAR100-Alexnet dataset, (c) represents the attack accuracy of the Purchase100-FCN dataset, and (d) represents the attack accuracy of the Texas100-FCN dataset.

[0057] Figure 6 The graph shows a comparison of attack accuracy for members and non-members with similar performance in four datasets, where (a) represents the attack accuracy of the CIFAR10-Alexnet dataset, (b) represents the attack accuracy of the CIFAR100-Alexnet dataset, (c) represents the attack accuracy of the Purchase100-FCN dataset, and (d) represents the attack accuracy of the Texas100-FCN dataset.

[0058] Figure 7 The graph shows the attack accuracy distribution for each class in the CIFAR10 dataset with and without round K added.

[0059] Figure 8 The gradient norm distribution of members and non-members in each class in the CIFAR10 dataset is shown after adding round K;

[0060] Figure 9 The diagram shows the distribution of prediction uncertainty for members and non-members in each class of the CIFAR10 dataset after adding round K. Detailed Implementation

[0061] Exemplary embodiments of the present application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment to achieve the developer’s specific objectives, and these decisions may vary as the embodiments differ.

[0062] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution according to this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0063] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.

[0064] Membership inference attacks are attacks targeting the training set of a model, aiming to determine whether a sample data belongs to the target model's training set. These attacks have proven effective against various machine learning models, such as classification and generative models. Membership inference attacks can serve as an effective tool for assessing the privacy risks of machine learning models. During the model training phase, they can help model trainers analyze potential privacy threats, leading to better model design and protection of user privacy. The goal of a membership inference attack is to determine whether a given sample is present in the target model's training dataset. The attacker, acting as a trainer in a federated learning system, aims to determine whether a sample data belongs to the datasets of the other participants.

[0065] The feasibility of membership inference attacks lies in the difference in characteristics exhibited by members and non-members on the target model. Existing work cannot accurately determine the membership information of a target sample for members and non-members with similar performance. The feasibility of membership inference attacks is closely related to the overfitting of the machine learning model; the larger the generalization error of the model, the higher the accuracy of the membership inference attack. Due to overfitting, the target model remembers its training data. When the target model predicts data, it has more confidence in the training samples, thus containing more information about the training set, with the last round of the target model containing the most membership information. Therefore, this application, targeting members and non-members with similar performance, uses causal analysis and, based on the characteristics generated by model overfitting, searches for features that can distinguish members and non-members during the training process of the target model. These features, combined with information from the last round of the target model, are weighted by a Transformer model and input into the attack model, outputting the probability value that the target sample is a member sample.

[0066] For member inference attacks, given sample data x, target model, and attacker information; the attack model is essentially a binary classifier. Given a sample data x, the output of the attack model is a value between 0 and 1. The threshold is set to 0.5. If the output is in [0, 0.5], the sample data will be judged as a non-member; otherwise, the sample will be judged as a member.

[0067] Figure 1 A flowchart illustrating a member inference attack method in federated learning according to an embodiment of this application is shown. The method includes:

[0068] Step S110: Determine the key round K in the training process of the target model by the attacker; the key round K is the round in which the difference in performance between member samples and non-member samples on the target model is the greatest.

[0069] In this step, for the target model, given a sample data (x, y), where x is the feature of the sample data and y is the true label, the goal of the target model is to find an f(·) that assigns the correct label to the sample data. The target model is trained using a loss function, and for each participant, the goal is to find an optimal f(·) that minimizes the loss function. Key rounds correspond to the rounds where the accuracy on the test set of the target model begins to stabilize or decrease. This means that key rounds only appear after several training rounds, not just the first few. However, during the target model's learning process, the accuracy on the test set fluctuates before stabilizing. Therefore, to determine key rounds, the fluctuations at the beginning of training need to be considered. In this step, the attacker is a participant who can compute the accuracy of the global model on its test set at each stage of the training process; key rounds are when the performance of the global model stabilizes on the test set. Therefore, the prediction accuracy gap between two consecutive rounds is calculated first. If the accuracy gap decreases, the search continues from the current round.

[0070] Overfitting is closely related to member inference attacks. Models with higher overfitting levels also have larger generalization errors and are more prone to leaking member information. Overfitted models have higher prediction accuracy on the training set than on the test set, leading to the target model containing rich member information. Therefore, this embodiment determines key epochs based on overfitting characteristics. When overfitting occurs, as the number of iterations increases, the performance of the training set on the target model gradually surpasses that on the test set. Specifically, after overfitting, the prediction accuracy on the training set continuously improves, while the prediction accuracy on the test set increases slowly or decreases. If a sample data point is not a member of the training set, and this sample data can be accurately predicted by the target model in the last epoch, then this sample data is very likely to be correctly predicted by the target model with a high probability in the key epoch. The prediction confidence of the sample data in these two epochs will be very close, and the difference in loss values ​​between these two epochs will also be relatively small. However, for sample data in the training set, the difference in loss values ​​between these two epochs may be large because the target model is continuously trained on the training set. The prediction confidence of the target model in the key epochs is greater than the prediction confidence of non-member data. The purpose of this step is to find the round K that has this distinguishing ability.

[0071] In summary, the key round K of the attacker's training process on the target model can be determined using the following formula:

[0072]

[0073] Wherein, the key round K is the value of K when ΔTr-ΔTe takes the maximum value, ΔTr is the difference between the key round K in the training process and the expected loss value of the last round, and ΔTe is the difference between the key round K in the testing process and the expected loss value of the last round.

[0074]

[0075]

[0076] Where l represents the last round, z represents the sample data, and L is the loss function of the target model. This represents the expected loss value of sample data z in training set D on target model A at round l. This represents the expected loss value of sample data z in training set D at key round K of target model A. This represents the expected loss value of sample data z in test set S on target model A at round l. It is the expected loss value of the sample data z in the test set S on the key round K of the target model A.

[0077] Step S120: Input the target sample into the target model and obtain the sample features; the sample features include the loss value of the target sample in the key round K of the target model and the loss value in the last round, the true label, the parameter gradient and the output;

[0078] In this step, the target sample is the sample whose status needs to be determined as either a member sample or a non-member sample. The attacker, acting as a participant in the training of the target model, inputs the target sample into the target model, obtaining the outputs h(x) of all hidden layers and the output f(x) of the last layer. Based on the output f(x) of the last layer and the true label y, the loss value of the target sample can be calculated using the loss function L(f(x),y). The attacker can also calculate the parameter gradients of each layer of the target model during backpropagation based on the loss value. Therefore, the attacker can obtain the sample features at each training epoch, including the loss value at the key epoch K and the loss value, true label, parameter gradients, and output at the last epoch. Specifically, the parameter gradients and output at the last epoch can be obtained using the parameter gradients and outputs of the last layer of the target model, ensuring both attack effectiveness and speed.

[0079] Step S130: Assign weights to the loss values ​​of the target samples in the key rounds K of the target model and in the last round.

[0080] In this step, among the loss value, true label, parameter gradient, and output in each round, the loss value is the most important and easiest sample feature to infer sample membership information. Therefore, a Transformer model is used to assign weights to the loss values ​​in the key round K and the last round, and the original loss value is multiplied by the weights and input into the attack model. Specifically, the Transformer model is a single-layer Transformer model, including an encoder and a decoder. Each encoder contains a self-attention layer and a feedforward neural layer. The Transformer model can help locate the focal region of the current task. This embodiment adds a Transformer-based attention mechanism to the attack model, allowing the attack model to focus on more important information in different situations.

[0081] Step S140: Input the weighted sample features into the attack model and output the probability value of the target sample being a member sample.

[0082] Step S150: Determine whether the target sample is a member sample based on the probability value.

[0083] In this embodiment, by sensing overfitting features, a key round K that can distinguish between members and non-members with similar performance is found during the training process of the target model. The Transformer model is then used to assign weights to the key round and the last round to different samples, thereby improving the attack accuracy.

[0084] In one embodiment, step S140, inputting sample features into the attack model and outputting the probability value that the target sample is a member sample, includes:

[0085] The attack model includes a fully connected neural network component (FCN), a convolutional neural network component (CNN), and an encoder;

[0086] In this embodiment, the loss value of the target sample in the key round K of the target model, as well as the loss value, true label, and output in the last round, are input into the fully connected neural network component; the parameter gradient of the target sample in the last round is input into the convolutional neural network component.

[0087] The outputs of both the fully connected neural network component and the convolutional neural network component are input into the encoder, which outputs the probability value that the target sample is a member sample. Here, the encoder output is a number between 0 and 1, representing the attack result and indicating the probability value that the target sample is a member sample. The threshold is set to 0.5. If the calculated probability value is greater than 0.5, the target sample is determined to be a member sample.

[0088] In one embodiment, the loss function of the attack model is:

[0089]

[0090] Where B represents the attack model, z represents the sample data, D represents the training set, and S represents the test set.

[0091] In this embodiment, for non-member samples, the output of the attack model is made close to 0, and for member samples, the output of the attack model is made close to 1; specifically, the loss function can be ∑ z～D (B(z)-1) 2 Approaching 0, that is, for member samples, making the output of the attack model approach 1, ∑ z～S (B(z)) 2 Approaching 0 means that for non-member samples, the output of the attack model approaches 0.

[0092] Based on the same inventive concept as the member reasoning attack method in federated learning, this embodiment also provides a corresponding member reasoning attack device in federated learning. Figure 2 A structural block diagram of a member inference attack apparatus in federated learning according to an embodiment of this application is shown, including:

[0093] The key round determination module 210 is used to determine the key round K of the attacker in the training process of the target model; the key round K is the round in which the difference between member samples and non-member samples is greatest on the target model;

[0094] The sample feature acquisition module 220 is used to input the target sample into the target model and acquire the sample features; the sample features include the loss value of the target sample in the key rounds K of the target model and the loss value in the last round, the true label, the parameter gradient and the output;

[0095] The weight allocation module 230 is used to assign weights to the loss values ​​of the target samples in the key rounds K of the target model and the loss values ​​in the last round.

[0096] The probability value determination module 240 is used to input the weighted sample features into the attack model and output the probability value that the target sample is a member sample.

[0097] Member sample determination module 250 is used to determine whether a target sample is a member sample based on its probability value.

[0098] In this embodiment, by sensing overfitting features, a key round K that can distinguish between members and non-members with similar performance is found during the training process of the target model. The Transformer model is then used to assign weights to the key round and the last round to different samples, thereby improving the attack accuracy.

[0099] To verify the effectiveness of the member reasoning attack method in federated learning proposed in this application, this application uses four datasets for experimental evaluation: CIFAR10, CIFAR100, Purchase100, and Texas100. The CIFAR10 and CIFAR100 datasets are commonly used to evaluate image classification tasks, the Purchase100 dataset contains shopping records of 197,324 online customers, and the Texas100 dataset contains patient information released by the Texas Department of Health Services.

[0100] Table 1 shows the attack accuracy of existing methods on four datasets for members and non-members with similar performance. The experimental results show that existing methods cannot accurately distinguish between members and non-members with similar performance, and the attack accuracy is close to the probability of random guessing (0.5).

[0101] Table 1 shows the attack accuracy of data with similar performance.

[0102] Dataset Number of members Number of non-members Attack accuracy CIFAR10 2000 2000 50.81％ CIFAR100 2000 2000 51.69％ Purchase 100 1000 1000 50.25％ Texas100 600 600 53.66％

[0103] Figure 3The chart shows a comparison of the loss values ​​of the target model at various training epochs across four datasets. (a) represents the loss value of the CIFAR10-Alexnet dataset at various training epochs, (b) represents the loss value of the CIFAR100-Alexnet dataset, (c) represents the loss value of the Purchase100-FCN dataset, and (d) represents the loss value of the Texas100-FCN dataset. In the early stages of training, the loss values ​​of both members and non-members gradually decrease. However, after a certain period, the rate of decrease in the loss value of members is significantly greater than that of non-members, which is consistent with the characteristics produced by model overfitting. The target model is continuously trained on the training set, and after a certain period, the accuracy of the target model on the test set tends to stabilize.

[0104] Figure 4 The diagram shows a comparison of the average loss values ​​of members and non-members with similar performance across four datasets at various training epochs of the target model. (a) represents the average loss value of the target model at various training epochs on the CIFAR10-Alexnet dataset, (b) represents the average loss value of the target model at various training epochs on the CIFAR100-Alexnet dataset, (c) represents the average loss value of the target model at various training epochs on the Purchase100-FCN dataset, and (d) represents the average loss value of the target model at various training epochs on the Texas100-FCN dataset. On the CIFAR10 dataset, the loss value of non-members decreases significantly from the initial stage to a certain point in the middle of the target model training process, and then decreases slowly. However, for members, the loss value decreases slowly from the start to the end of the target model training. This means that, except for the last epoch, features that can distinguish between members and non-members can be found during the training process. This intermediate point that can distinguish between members and non-members with similar performance is the key epoch K sought in this application.

[0105] Figure 5 The diagram shows a comparison of attack accuracy on four datasets using the method described in this application, where (a) represents the attack accuracy on the CIFAR10-Alexnet dataset, (b) represents the attack accuracy on the CIFAR100-Alexnet dataset, (c) represents the attack accuracy on the Purchase100-FCN dataset, and (d) represents the attack accuracy on the Texas100-FCN dataset. Figure 6The graph shows a comparison of attack accuracy for members and non-members with similar performance across four datasets, where (a) represents the attack accuracy on the CIFAR10-Alexnet dataset, (b) represents the attack accuracy on the CIFAR100-Alexnet dataset, (c) represents the attack accuracy on the Purchase100-FCN dataset, and (d) represents the attack accuracy on the Texas100-FCN dataset. To demonstrate the importance of epoch K compared to other training periods, several epochs were uniformly selected from the beginning to the end of the target model training to establish a baseline, and the attack accuracy of each baseline epoch was compared with that of epoch K. On the CIFAR10 dataset, epoch K is close to 25, and the target model was trained for a total of 120 epochs; therefore, epochs [15, 45, 75, 100] were selected for comparison. On the CIFAR100 dataset, the number of epochs K is close to 100, and the target model was trained for 700 epochs; therefore, epochs [100, 200, 300, 400, 500, 600] were selected for comparison. On Purchase100, the number of epochs K is close to 90, and the target model was trained for 220 epochs; therefore, epochs [10, 50, 130, 170, 210] were selected for comparison. On Texas100, the number of epochs K is close to 63, and the target model was trained for 700 epochs; therefore, epochs [100, 200, 300, 400, 500, 600] were selected for comparison. Figure 5 and Figure 6 It can be seen that the round K selected in this scheme has the highest attack accuracy compared with other rounds, especially with members and non-members with similar performance, which proves the effectiveness of round K.

[0106] Table 2 compares the method of this application with existing methods based on loss value and prediction entropy. The results show that the neural network-based method used in this application has the best performance.

[0107] Table 2 Comparison of Attack Accuracy

[0108] Dataset Based on neural network Based on loss value Based on prediction entropy CIFAR10 71.54％ 70.30％ 70.20％ CIFAR100 81.55％ 79.00％ 79.60％ Purchase 100 63.80％ 62.20％ 61.50％ Texas100 73.00％ 71.90％ 71.70％

[0109] Table 3 compares the attack accuracy with and without the attention mechanism on four datasets. The results show that adding the attention mechanism can improve the accuracy of member reasoning attacks to a certain extent.

[0110] Table 3 Attack accuracy with and without attention mechanism

[0111] Dataset non-attention mechanism Attention mechanism CIFAR10 71.54％ 71.74％ CIFAR100 81.55％ 82.70％ Purchase 100 63.80％ 63.95％ Texas100 73.00％ 73.56％

[0112] Figure 7The graph shows the attack accuracy distribution for each class in the CIFAR10 dataset with and without round K. It can be seen from the graph that adding round K improves the attack accuracy for each class, but the magnitude of the improvement varies. To understand the reasons for this difference, this application investigates the gradient norm and prediction uncertainty for members and non-members in each class.

[0113] Figure 8 The diagram shows the gradient norm distribution of members and non-members in each class of the CIFAR10 dataset after adding round K. Figure 9 The diagram shows the distribution of prediction uncertainty for members and non-members in each class of the CIFAR10 dataset after adding round K. Figure 7 It can be seen that on CIFAR10, the accuracy rate is highest for attacks of classes 2, 3, 4, and 5. Figure 8 Among them, the gradient norm differences between members and non-members in classes 3 and 5 are the largest. Figure 9 Among them, the difference in prediction uncertainty between members and non-members is the largest in categories 2, 3, 4, and 5. It can be seen that both gradient norm and prediction uncertainty can be used as features to distinguish between members and non-members.

[0114] To evaluate which layer's output has the greatest impact on the target model, the attack was performed individually using the output of each layer, without using gradients. Table 4 shows the attack accuracy on four datasets. It can be seen that the attack accuracy using only the last layer is close to that using the outputs of all layers, meaning that the output of the last layer of the target model contains the most membership information.

[0115] To evaluate which layer's gradient has the greatest impact on the attack, the attack was performed using the gradient of each layer separately, without using the output. Table 5 shows the attack accuracy on four datasets. The results are similar to those above, with the last layer revealing more membership information.

[0116] Table 6 shows the attack accuracy using different layer gradients and outputs on four datasets. The results indicate that the attack accuracy using all layers is close to that using only the last layer. Considering both attack effectiveness and speed, attackers can train attack models using only information from the last layer of the target model.

[0117] Table 4 Attack accuracy of different layers

[0118] Output CIFAR10 CIFAR100 Purchase 100 Texas100 The last three floors 68.99％ 79.50％ 62.17％ 71.86％ The last two floors 69.30％ 81.01％ 62.10％ 71.96％ Last layer 69.40％ 81.61％ 62.14％ 72.39％

[0119] Table 5 Attack accuracy of different gradient layers

[0120] gradient CIFAR10 CIFAR100 Purchase 100 Texas100 The last three floors 70.29％ 78.51％ 61.32％ 70.66％ The last two floors 70.39％ 79.45％ 61.62％ 70.85％ Last layer 71.52％ 81.50％ 61.75％ 71.06％

[0121] Table 6 Attack accuracy of different layer gradient and output combinations

[0122] gradient Output CIFAR10 CIFAR100 Purchase 100 Texas100 Last layer — 71.52％ 81.45％ 61.75％ 71.06％ — Last layer 69.40％ 81.54％ 62.14％ 72.13％ Last layer Last layer 71.54％ 81.55％ 63.80％ 73.00％

[0123] Table 7 shows the attack accuracy of the attack model on four datasets when using different training scales. The results indicate that increasing the number of training iterations of the attack model can improve the attack accuracy to some extent.

[0124] Table 7. Impact of Training Data Size on Attack Accuracy

[0125] Number of members Number of non-members CIFAR10 CIFAR100 Purchase 100 Texas100 2000 2000 70.90％ 79.15％ 61.20％ 73.00％ 4000 4000 70.97％ 81.10％ 63.80％ — 6000 6000 71.40％ 81.55％ — — 8000 8000 71.54％ 81.55％ — —

[0126] In summary, this application improves attack accuracy, especially for members and non-members that perform similarly on the target model; by utilizing attention mechanisms and combining features from key rounds and the final round, it helps to infer member information for different types of samples; through experimental analysis, it is verified that the method of this application can improve attack accuracy compared with existing methods.

[0127] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for a member inference attack in federated learning, the method comprising: include: Identify the key rounds K of the attacker's actions during the training process of the target model; The key round K is the round in which the difference in performance between member samples and non-member samples on the target model is greatest. The target sample is input into the target model to obtain sample features; the sample features include the loss value of the target sample in the key round K of the target model and the loss value in the last round, the true label, the parameter gradient and the output; Weights are assigned to the loss values ​​of the target sample in the key rounds K of the target model and in the last round; The weighted sample features are input into the attack model, and the probability value of the target sample being a member sample is output. Determine whether the target sample is a member sample based on the probability value; The key round K of the attacker's training process on the target model is determined using the following formula: Wherein, the key round K is The value of K when taking the maximum value, The difference between the key round K and the expected loss value of the last round in the training process, The difference between the key round K and the expected loss value of the last round in the test process; wherein, denotes the last epoch, z denotes the sample data, L is the loss function of the target model, denotes the expected loss value of the sample data z in the training set D on the target model A epoch , denotes the expected loss value of the sample data z in the training set D on the target model A key epoch K, denotes the expected loss value of the sample data z in the test set S on the target model A epoch , is the expected loss value of the sample data z in the test set S on the target model A key epoch K.

2. The method as described in claim 1, characterized in that, The method further includes: The loss value of the target sample in the key round K of the target model and the loss value in the last round are input into the Transformer model to obtain the weights of the loss value of the target sample in the key round K of the target model and the weights of the loss value in the last round.

3. The method as described in claim 1, characterized in that, in, The sample features are input into the attack model, and the probability value that the target sample is a member sample is output, including: The attack model includes a fully connected neural network component, a convolutional neural network component, and an encoder; The loss value of the target sample at the key round K of the target model, the loss value at the last round, the true label, and the output are input into the fully connected neural network component. The parameter gradient of the target sample in the last round is input into the convolutional neural network component; The outputs of the fully connected neural network component and the convolutional neural network component are both input into the encoder, which outputs the probability value that the target sample is a member sample.

4. The method as described in claim 1, characterized in that, The loss function of the attack model is: Where B represents the attack model, z represents the sample data, D represents the training set, and S represents the test set.

5. A member reasoning attack device in federated learning, characterized in that, include: The key round determination module is used to determine the key round K in the training process of the target model by the attacker; the key round K is the round in which the difference in performance between member samples and non-member samples on the target model is the greatest. The sample feature acquisition module is used to input the target sample into the target model and acquire the sample features; the sample features include the loss value of the target sample in the key rounds K of the target model and the loss value in the last round, the true label, the parameter gradient and the output; The weight allocation module is used to assign weights to the loss value of the target sample in the key round K of the target model and the loss value in the last round. The probability value determination module is used to input the weighted sample features into the attack model and output the probability value that the target sample is a member sample. A member sample determination module is used to determine whether the target sample is a member sample based on the probability value; The key round determination module is also used for: The critical round K of the attacker during the training process of the target model is determined using the following formula: Among them, the key round K is The value of K when the maximum value is reached. This represents the difference between the expected loss value of the critical epoch K and the last epoch in the training process. This is the difference between the expected loss value of the critical round K in the test process and the last round; in, This indicates the last round, and z represents the sample data. L Let the loss function be the target model. This indicates that the sample data z in the training set D is in round A of the target model. The expected loss value on, This represents the expected loss value of sample data z in training set D at key round K of target model A. This indicates that the sample data z in the test set S is in round A of the target model. The expected loss value on, It is the expected loss value of the sample data z in the test set S on the key round K of the target model A.

6. The apparatus as claimed in claim 5, characterized in that, It also includes a weight determination module, used for: The loss value of the target sample in the key round K of the target model and the loss value in the last round are input into the Transformer model to obtain the weights of the loss value of the target sample in the key round K of the target model and the weights of the loss value in the last round.

7. The apparatus as claimed in claim 5, characterized in that, The sample feature acquisition module is used for: The attack model includes a fully connected neural network component, a convolutional neural network component, and an encoder; The loss value of the target sample at the key round K of the target model, the loss value at the last round, the true label, and the output are input into the fully connected neural network component. The parameter gradient of the target sample in the last round is input into the convolutional neural network component; The outputs of the fully connected neural network component and the convolutional neural network component are both input into the encoder, which outputs the probability value that the target sample is a member sample.

8. The apparatus as claimed in claim 5, characterized in that, The loss function of the attack model is: Where B represents the attack model, z represents the sample data, D represents the training set, and S represents the test set.

Images