Adversarial attack based longitudinal federated learning label poisoning method

Abstract

The application provides a longitudinal federated learning label poisoning method based on an adversarial attack, realizes label replacement of longitudinal federated learning based on a label poisoning backdoor attack, obtains general backdoor disturbance based on a semi-honest client training without interfering with a main task, and can realize label flipping of a rejected sample by adding the general backdoor disturbance to the semi-honest client in a longitudinal federated inference stage. The application can train general backdoor disturbance in a longitudinal federated learning training stage, obtain general disturbance for changing labels without being perceived by other participants and without affecting the training of the main task, the general disturbance can change the prediction result of a specific sample in a longitudinal federated inference stage, and the research on the label poisoning backdoor method has great significance for promoting longitudinal federated security cooperation.

Description

Technical Field

[0001] This invention belongs to the field of federated learning technology, specifically relating to a vertical federated learning label poisoning method based on adversarial attacks. Background Technology

[0002] With the implementation of data privacy laws, centralized data-driven deep learning applications are facing limitations, and the data silo effect is becoming increasingly pronounced in financial scenarios. Federated learning, as a distributed privacy-preserving computing framework, can complete joint training without data leakage, and has great application prospects in fields such as finance. In the financial field, federated learning is mainly structured as vertical federated learning, comprising two roles: multiple clients and one server. Different clients and the server possess the same data samples, but their data features do not overlap. Each client possesses some data features, while the server possesses label information. Multiple clients and the server jointly train a global model. The specific training process is as follows: Vertical federated learning involves multiple rounds of iterative training. In each iteration, the client sends the feature embedding of the current model to the server. The server aggregates the received feature embedding with its own data feature embedding and then uses the labels to train the server's model parameters. Simultaneously, the server outputs the gradient of the label with respect to the client's feature embedding and returns it to the client. The client updates its own model parameters based on the returned gradient. During iterative training, the training data does not leave the local machine; communication is achieved through feature embeddings and gradients, ensuring data security for all participants. However, algorithm backdoor vulnerabilities still exist in the vertical federated learning communication process, and various privacy attacks, including adversarial attacks, seriously undermine the security of federated learning.

[0003] Current research on adversarial attacks largely focuses on image and language data, using methods such as FGSM and Deepfool to create adversarial examples. However, financial scenario data is often heterogeneous chart data with non-Gaussian distributions, requiring consideration of inherent characteristics like immutability and correlation when using adversarial attacks. Existing research primarily uses rule constraints combined with adversarial attack methods to create feasible adversarial examples. Vertical federated learning, as a privacy-preserving computation framework, enables joint computation of features from multiple parties without data leaving the database, effectively addressing the data security requirements of financial scenarios. Currently, research on adversarial poisoning of vertical federated learning clients is lacking. Semi-honest and malicious clients pose significant threats to the security of vertical federated data, urgently requiring resolution. Therefore, to better identify security vulnerabilities, research corresponding countermeasures, and validate them, a method for effectively adversarially attacking vertical federated learning in financial scenarios is needed. Summary of the Invention

[0004] To address the aforementioned issues, a label poisoning method based on adversarial attacks is proposed for vertical federated learning. This method employs projective gradient descent to perform adversarial attacks on client input features, obtaining general attack perturbations that can poison labels during the training phase. This enables label replacement of rejected samples during the inference phase of vertical federated learning. This method can serve as an effective means for verifying and researching the security of vertical federated data.

[0005] The present invention adopts the following technical solution:

[0006] This invention provides a label poisoning method for vertical federated learning based on adversarial attacks, used by an attacker client participating in vertical federated learning to perform label poisoning attacks on a server. The client and server are applied in a financial scenario. The method comprises the following steps: Step S1, the attacker client converts local financial table data to a homogeneous continuous space, thereby converting the attacker client's features into homogeneous features; Step S2, the attacker client adds a randomly initialized general backdoor perturbation to the homogeneous features to generate poisoned features; Step S3, during the training phase of the vertical federated learning, the attacker client sends the poisoned features to the server. The server receives global features containing the poisoned features, calculates the gradient information of the poisoned features based on the global features and the labels, and sends it back; Step S4, the attacker client uses the received gradient information to optimize the general backdoor perturbation using a stochastic gradient ascent method; Step S5, steps S3 and S4 are repeated until the training phase ends, resulting in the iteratively optimized general backdoor perturbation as a global perturbation. This global perturbation is used to flip the predicted values ​​of rejected samples during the model inference phase of the vertical federated learning.

[0007] The vertical federated learning label poisoning method based on adversarial attacks provided by this invention may also have the following technical features: In step S3, after receiving the global features, the server performs minimum cross entropy loss optimization in combination with its local labels, and returns the gradient information of the output loss function with respect to the poisoned features to the attacker client; In step S4, the attacker client iteratively optimizes the general backdoor perturbation in a stochastic gradient ascent manner without changing the labels and the minimum cross entropy loss on the server.

[0008] The vertical federated learning label poisoning method based on adversarial attacks provided by this invention may also have the following technical features, wherein, in step S2, the poisoning feature generated by the attacker client is represented as:

[0009]

[0010] In step S3, the global feature received by the server is represented as follows:

[0011]

[0012] In step S4, the gradient information returned by the server to the attacker's client is represented as follows:

[0013]

[0014]

[0015] In the formula, K is the number of participants in the vertical federated learning, σ(·) is the server-side nonlinear function, and f k This is a sub-model for the k-th participant; The feature embedding of the i-th data point representing the k-th participant, y i Here, δ represents the server-side label, δ represents the learnable general backdoor perturbation, and N represents the number of samples in the dataset. B The number of samples in each batch.

[0016] The longitudinal federated learning label poisoning method based on adversarial attacks provided by this invention may also have the following technical features, wherein, in step S1, the financial table data is transformed into a homogeneous continuous space through the table transformation module of the attacker's client, and the table transformation module performs self-supervised learning based on the features of the attacker's client and the homogeneous features.

[0017] The longitudinal federated learning label poisoning method based on adversarial attacks provided by this invention may also have the following technical features, wherein the training objectives of the self-supervised learning of the table transformation module include mask feature prediction and damage feature prediction.

[0018] The longitudinal federated learning label poisoning method based on adversarial attacks provided by this invention may also have the following technical feature, wherein the training objective of the self-supervised learning is represented as:

[0019]

[0020] In the formula, M m and C r These represent mask feature prediction and damage feature prediction, respectively. m and l r These refer to binary cross-entropy and mean squared error loss, respectively. β is the balancing hyperparameter, and E() represents the table transformation encoder. Calculate using the following formula:

[0021]

[0022] In the formula, x A This is the attacker's client's feature data, where m is a randomly sampled binary mask. Based on x A Random sampling of the empirical distribution function.

[0023] The vertical federated learning label poisoning method based on adversarial attacks provided by the present invention may also have the following technical feature, wherein, in step S3, the attacker client sends the poisoning feature to the server in a predetermined number of training rounds.

[0024] Invention Function and Effect

[0025] According to the label poisoning method for vertical federated learning based on adversarial attacks of the present invention, the attacker client first transforms the metal table data into a homogeneous continuous space, thereby obtaining homogeneous features. Perturbations are then injected into these homogeneous features to generate poisoned features. Further, during the training phase of vertical federated learning, the attacker client sends the poisoned features to the server, which returns corresponding gradient information. The attacker client uses this gradient information to iteratively optimize the general backdoor perturbation in a gradient ascent manner, thereby training a global perturbation used to flip the predicted values ​​of rejected samples. In other words, the method of the present invention achieves label replacement in vertical federated learning based on label poisoning backdoor attacks. A general backdoor perturbation is obtained through training on the attacker client without interfering with the main task. During the vertical federated inference phase, the general backdoor perturbation added to the attacker client enables label flipping of rejected samples. This method obtains a label-changing general backdoor perturbation without being perceived by other participants or affecting the main task training, exhibiting stealth. This general backdoor perturbation can change the prediction results of specific samples during the vertical federated inference phase. This method can serve as an effective means for verifying and researching the security of vertical federated data, and is of great significance for promoting secure collaboration in vertical federation. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of a credit assessment scenario in an embodiment of the present invention;

[0027] Figure 2 This is a flowchart of the vertical federated learning label poisoning method based on adversarial attacks in an embodiment of the present invention;

[0028] Figure 3 This is a diagram of the federated learning label poisoning attack framework in an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram illustrating the performance of classification indicators in an embodiment of the present invention;

[0030] Figure 5This is a visualization of the confusion matrix on the LendingClub dataset in an embodiment of the present invention;

[0031] Figure 6 This is a visualization of the confusion matrix on the Zhongyuan dataset in an embodiment of the present invention. Detailed Implementation

[0032] To make the technical means, creative features, objectives and effects of this invention easy to understand, the following describes the vertical federated learning label poisoning method based on adversarial attacks in detail with reference to the embodiments and accompanying drawings.

[0033] <Example>

[0034] Figure 1 This is a schematic diagram of the credit assessment scenario in this embodiment.

[0035] like Figure 1 As shown, the vertical federated learning label poisoning method in this embodiment is applied to a credit assessment scenario. Vertical federated learning includes a server and two clients (federated learning participants). The two clients and the server have the same data samples, each client has some data features, and there is no overlap in the data features. The server has label information, and multiple clients and the server train a global model simultaneously.

[0036] In credit assessment scenarios, the server-side is located within banks, while the client-side is located in various third-party institutions. The server-side, combined with third-party data, can more accurately assess credit risk. However, if third parties possess backdoors that allow default samples to smoothly pass bank loan approvals, it would pose a serious threat to credit assessment and other financial sectors.

[0037] Vertical federated learning involves multiple rounds of iterative training. In each iteration, the client sends the feature embeddings of the current model to the server. The server aggregates the received feature embeddings with its own data feature embeddings and then uses the labels to train the server-side model parameters. Simultaneously, the server outputs the gradient of the labels with respect to the client's feature embeddings and returns it to the client. The client updates its own model parameters based on the returned gradient. During iterative training, the training data does not leave the local machine; communication occurs only through feature embeddings and gradients.

[0038] The method in this embodiment mainly utilizes two characteristics of vertical federated learning:

[0039] (1) Vertical federated learning involves multiple rounds of iterative training, so the attacker client can obtain the gradient information returned by the server in each round of iterative training;

[0040] (2) The data on the attacker's client is not visible to the server.

[0041] The method in this embodiment obtains a general attack perturbation that can viralize labels during the training phase of longitudinal federated learning, and uses this perturbation to replace the labels of rejected samples during the model inference phase.

[0042] During the training phase, the data from multiple clients (participants) involved in the vertical federated learning are first aligned using the PSI method to select data from the same sample space. Then, within the same sample space, data cleaning methods are used to clean the features and labels. The server has the label y, and each client saves its own feature data.

[0043] The system comprises multi-layered nonlinear segmentation networks distributed across the various clients and servers. After setting the hyperparameters, joint training for longitudinal federated learning can begin, as shown in the following formula.

[0044]

[0045]

[0046]

[0047] In the formula, σ(·) is a nonlinear function on the server side, and f k For the sub-model at the k-th end; The feature embedding represents the i-th data at the k-th end; γ(·) represents the regularization term, and λ is a hyperparameter; Refers to dataset θ k These are the parameters of the k-th end model.

[0048] After a predetermined number of iterations of training, the trained joint model is obtained.

[0049] The method in this embodiment is implemented as a computer program, deployed and run on one of the clients, which is the semi-honest attacker client, launching a label poisoning attack during the joint training process. The other client does not have this program deployed and is a benign client.

[0050] Figure 2 This is a flowchart of the vertical federated learning label poisoning method based on adversarial attacks in this embodiment.

[0051] like Figure 2 As shown, the method in this embodiment specifically includes the following steps:

[0052] Step S1: The attacker client transforms its local financial table data into a homogeneous continuous space, thereby converting the attacker client's features into homogeneous features.

[0053] Financial tabular data is heterogeneous, containing dense continuous features and sparse discrete features, and possesses certain semantic information. However, general adversarial attack methods are mostly applicable to homogeneous continuous data and cannot adapt to financial tabular data with diverse feature dimensions. General backdoor perturbations generated based on financial tabular data need to consider both effectiveness and stealth. Based on these two characteristics, this embodiment adds a table transformation module to the attacker's client model to transform the heterogeneous financial tabular data into a homogeneous continuous space, injecting a general backdoor perturbation into the homogeneous features with uniform dimensions.

[0054] The table transformation module performs self-supervised learning based on the attacker's client's feature data (i.e., features before and after transformation), with training objectives including mask feature prediction and damage feature prediction.

[0055]

[0056] In the formula, M m and C r These represent mask feature prediction and damage feature prediction, respectively. m and l r These refer to binary cross-entropy and mean squared error loss, respectively.

[0057] β is the balancing hyperparameter, and E() is the table transform encoder, where Calculate using the following formula:

[0058]

[0059] In the formula, x A This is the attacker's client feature data, where m is a randomly sampled binary mask. Based on x A Random sampling of the empirical distribution function.

[0060] Step S2: The attacker's client adds a randomly initialized general backdoor perturbation (adversarial perturbation) to its original homogeneous features to generate the perturbed features, i.e., the poisoned features.

[0061] Step S3: During the training phase of longitudinal federated learning, the attacker client sends the poisoned features to the server. The server receives the global features containing the poisoned features, calculates the gradient information of the poisoned features based on the global features and the labels, and sends it to the attacker client.

[0062] That is, there are K participants in the joint training. The attacker's client outputs the perturbed feature representation and performs joint training with the other K-1 participants.

[0063]

[0064]

[0065] In the formula, It is the poisoning characteristic output by the attacker. δ represents the global features received by the server, and δ is a learnable general backdoor perturbation.

[0066] After receiving the global features, the server combines them with the local labels to optimize the cross-entropy loss, and outputs the gradient of the loss function with respect to the perturbed feature representation, returning it to the attacker's client.

[0067]

[0068]

[0069] In the formula, g represents the gradient information required to train the global backdoor perturbation in the attacker, and N is the number of samples in the dataset. B The number of samples in each batch.

[0070] In step S4, the attacker's client uses the received gradient information to optimize the general backdoor perturbation using stochastic gradient ascent.

[0071] In this process, the attacker's client iteratively optimizes the general backdoor perturbation using stochastic gradient ascent without changing the server-side labels and minimizing the cross-entropy loss, thereby achieving label replacement based on the attacker's client perturbation input.

[0072] Step S5: Repeat steps S3 and S4 until the longitudinal federated training ends. The resulting general backdoor perturbation, optimized through multiple rounds of iterations, is used as a global perturbation. This global perturbation can be used to flip the predicted values ​​of rejected samples during the model inference stage of longitudinal federated learning.

[0073] In other words, the general backdoor perturbation is iteratively optimized using the received gradient information. The resulting general backdoor perturbation can be used in the model inference stage to achieve a backdoor attack on the vertical federated prediction value. That is, by adding a global perturbation, the prediction value of the rejected sample can be flipped.

[0074] In this embodiment, Python 3.7 and PyTorch 1.8.0 are used to implement the above method. The server and client configurations for running the experiment are as follows: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz, 64GB of memory, and equipped with two NVIDIA Tesla T4 GPUs.

[0075] In this embodiment, the LendingClub and Zhongyuan datasets were used. For both datasets, a multilayer perceptron model was employed as the training model for the vertical federated participants. This multilayer perceptron model consists of three linear layers, with the first two layers followed by linear rectified functions. The server-side model is a classifier containing one linear layer. For the LendingClub dataset, after cleaning, it contains 86,291 training samples and 21,573 test samples; 83 features were split between the two clients, with 15 features on the benign client and 68 features on the attacker client. For the Zhongyuan dataset, after SMOTE enhancement, it contains 8,316 training samples and 2,080 test samples; the benign client contains 15 features, and the attacker client contains 22 features.

[0076] As described above, this embodiment uses two clients: an attacker's client and a benign client, to evaluate the effect of label poisoning. In this embodiment, 100 rounds of federated learning iterations are used, with adversarial attacks injected every 10 rounds.

[0077] Figure 4 This is a schematic diagram illustrating the performance of the classification index in this embodiment.

[0078] like Figure 4 As shown, the F1 scores of the labeled poisoned BVFL-R on the LendingClub and Zhongyuan datasets are 72.34% and 40.47%, respectively; compared with the unattacked model BVFL-S, the F1 scores decrease by 14% and 30%, respectively, which demonstrates the effectiveness of the method in this embodiment.

[0079] Figure 5 This is a visualization of the confusion matrix on the LendingClub dataset in this embodiment. Figure 6 This is a visualization of the confusion matrix on the Zhongyuan dataset in this embodiment.

[0080] like Figure 5 , Figure 6 As shown, on the LendingClub and Zhongyuan datasets, 97.7% and 100% of the defaulting positive samples were predicted as credible negative samples, respectively; meanwhile, 100% and 100% of the credible negative samples were still predicted as negative samples, respectively. The experimental results demonstrate that the attack method described in this embodiment is effective and stealthy; that is, the defaulting target samples are effectively poisoned, and the correct identification of credible negative samples ensures that the attack is not easily detected by the server, thus possessing stealth.

[0081] Functions and effects of the embodiments

[0082] According to the label poisoning method for vertical federated learning based on adversarial attacks provided in this embodiment, the attacker's client first transforms the metal table data into a homogeneous continuous space, thereby obtaining homogeneous features. Perturbations are then injected into these homogeneous features to generate poisoned features. Further, during the training phase of vertical federated learning, the attacker's client sends the poisoned features to the server, which returns corresponding gradient information. The attacker's client uses this gradient information to iteratively optimize the general backdoor perturbation in a gradient ascent manner, thereby training a global perturbation used to flip the predicted values ​​of rejected samples. In other words, the method in this embodiment achieves label replacement in vertical federated learning based on a label poisoning backdoor attack. A general backdoor perturbation is obtained through training on the attacker's client without interfering with the main task. During the vertical federated inference phase, the general backdoor perturbation added to the attacker's client enables label flipping of rejected samples. This method obtains a general backdoor perturbation that changes labels without being perceived by other participants or affecting the main task training, exhibiting stealth. This general backdoor perturbation can change the prediction results of specific samples during the vertical federated inference phase. This method can serve as an effective means for verifying and researching the security of vertical federated data, and is of great significance for promoting secure collaboration in vertical federation.

[0083] In the embodiments, the method was verified on the LendingClub and Zhongyuan datasets, proving that the method can effectively poison default target samples, making them predict as credible negative samples. At the same time, credible negative samples are still predicted as negative samples, proving that the attack of the method is stealthy.

[0084] The above embodiments are only used to illustrate specific implementations of the present invention, and the present invention is not limited to the scope of the description of the above embodiments.

[0085] In the above embodiment, vertical federated learning has two participants, one of whom is an attacker client. In an alternative, there may be more participants, one or more of whom are attacker clients.

Claims

1. A label poisoning method for vertical federated learning based on adversarial attacks, used by an attacker client participating in vertical federated learning to perform label poisoning attacks on a server, wherein the client and the server are applied in a financial scenario, characterized in that, Includes the following steps: Step S1: The attacker's client transforms the local financial table data into a homogeneous continuous space, thereby converting the features of the attacker's client into homogeneous features; Step S2, the attacker's client adds a randomly initialized general backdoor perturbation to the homogeneous feature to generate a poisoned feature; Step S3, during the training phase of the longitudinal federated learning, the attacker client sends the poisoned features to the server. The server receives global features containing the poisoned features, calculates the gradient information of the poisoned features based on the global features and the label, and sends it back. Step S4: The attacker client uses the received gradient information to optimize the general backdoor perturbation using a stochastic gradient ascent method. Step S5: Repeat steps S3 and S4 until the training phase ends, obtaining the iteratively optimized general backdoor perturbation as a global perturbation. This global perturbation is used to flip the predicted values ​​of rejected samples during the model inference phase of the longitudinal federated learning. In step S2, the poisoning feature generated by the attacker's client is represented as follows: In step S3, the global feature received by the server is represented as follows: In step S4, the gradient information returned by the server to the attacker's client is represented as follows: In the formula, The number of participants in vertical federated learning. This is a server-side nonlinear function. This is a sub-model for the Ath participant, namely the attacker's client. For this sub-model Model parameters, For table transformation encoder, The first of the attacker's client Feature data of each data point The poisoning signature output by the attacker's client. The global feature received by the server; Representing the The first participating party Feature embedding of data, For server-side tags, For learnable general backdoor perturbations, To train the global backdoor perturbation in the attacker's client, the gradient information is required. The number of samples in the dataset. The number of samples in each batch.

2. The vertical federated learning label poisoning method based on adversarial attacks according to claim 1, characterized in that: in, In step S3, after receiving the global features, the server performs cross-entropy loss optimization by combining them with its local labels, and returns the gradient information of the loss function with respect to the poisoned features to the attacker's client. In step S4, the attacker client iteratively optimizes the general backdoor perturbation using a stochastic gradient ascent method without changing the label on the server side or the minimum cross entropy loss.

3. The vertical federated learning label poisoning method based on adversarial attacks according to claim 1, characterized in that: in, In step S1, the financial table data is converted into a homogeneous continuous space using the table transformation module of the attacker's client. The table transformation module performs self-supervised learning based on the features of the attacker's client and the homogeneous features.

4. The vertical federated learning label poisoning method based on adversarial attacks according to claim 3, characterized in that: in, The training objectives of the self-supervised learning of the table transformation module include mask feature prediction and damage feature prediction.

5. The vertical federated learning label poisoning method based on adversarial attacks according to claim 4, characterized in that: in, The training objective of the self-supervised learning is expressed as: In the formula, and These represent mask feature prediction and damage feature prediction, respectively. and These refer to binary cross-entropy and mean squared error loss, respectively. , , To balance hyperparameters, Calculate using the following formula: In the formula, It is the attacker's client's characteristic data. It is a randomly sampled binary mask. Based on Random sampling of the empirical distribution function.

6. The vertical federated learning label poisoning method based on adversarial attacks according to claim 1, characterized in that: in, In step S3, the attacker client sends the poisoning feature to the server in multiple predetermined training rounds.

Images