Backdoor defense method, equipment and product based on semantic consistency vertical federated learning

Abstract

The application discloses a backdoor defense method, device and product based on semantic consistency longitudinal federal learning, first adopts a latent mask autoencoder to evaluate the semantic consistency of the embedding provided by different longitudinal federal learning participants; then, with the help of semantic consistency evaluation, the malicious embedding is identified, and the attacker is identified; finally, the embedding reconstructed by the latent mask autoencoder is used to replace the embedding of the attacker, so that the backdoor semantics is removed, and the top model realizes the robust prediction against the backdoor. Based on the discovery that backdoor attacks may cause inconsistent embedding semantics, the application proposes a new defense technology, which does not depend on any auxiliary data and has no premise assumption on backdoor attacks or model structure. The application develops a latent mask autoencoder to evaluate the semantic consistency between the embeddings of each participant. In addition, the autoencoder is used for attacker identification and embedding reconstruction, so that the backdoor samples realize robust prediction.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence security, and relates to a longitudinal federated learning backdoor defense method, equipment and product, in particular to a longitudinal federated learning backdoor defense method, equipment and product based on semantic consistency. BACKGROUND

[0002] As a distributed machine learning framework, federated learning allows multiple data owners (such as different enterprises, organizations) to collaboratively train a shared model without sharing raw data. Federated learning aims to solve the data island problem, promote the efficient use of data and optimization of models, while complying with strict privacy protection regulations. According to how data is distributed among multiple parties, federated learning can be divided into two main types: horizontal federated learning and vertical federated learning. Participants in vertical federated learning (VFL) usually have different features in the same set of data samples, so VFL is more suitable for collaboration between different types of organizations, such as banks and e-commerce platforms.

[0003] Recent research has shown that VFL is vulnerable to backdoor attacks. Attackers can manipulate their local information such as data, models, embeddings, etc. during training to inject backdoors into the top-level model, with the aim of misleading the top-level model to make incorrect predictions during inference using malicious embeddings. The key challenge faced by backdoor defense techniques in VFL is the lack of information about other parties' features and models, as well as the lack of clean auxiliary data sets. SUMMARY

[0004] To solve the technical problem of backdoor attack risk in VFL, the application proposes a longitudinal federated learning backdoor defense method, equipment and product based on semantic consistency based on the discovery that backdoor attacks may cause semantic inconsistency in embeddings.

[0005] The technical solution adopted by the method of the application is: a longitudinal federated learning backdoor defense method based on semantic consistency, comprising the following steps:

[0006] Step 1: Use a latent mask autoencoder to evaluate the semantic consistency of the embeddings provided by different longitudinal federated learning participants;

[0007] The latent mask autoencoder comprises multiple Transformer-based encoder modules, employing a multi-stage encoder skip connection design to gradually reconstruct masked input data through multiple encoders. First, the input data is masked to block some information, creating partially lost or damaged data as input. Each encoder module receives the output of the previous layer, gradually encodes and extracts features from the input, and simultaneously introduces skip connections between encoder layers at preset distance intervals, passing the output of each encoder not only to the next encoder but also to the encoder after the preset distance interval.

[0008] Step 2: Use semantic consistency assessment to identify malicious embeddings and identify the attacker.

[0009] Preferably, in step 2, each time the active party receives a set of embeddings from the passive party, it checks these embeddings one by one and evaluates the semantic consistency of each embedding with other embeddings using the reconstruction error; if the largest reconstruction error in the group is greater than a pre-set threshold ε, then the participant corresponding to the embedding is considered a potential attacker; if no reconstruction loss in the group exceeds ε, then the set of embeddings is considered clean.

[0010] Preferably, the specific implementation of step 2 includes the following steps:

[0011] Step 2.1: Receive each participant’s embedding of sample x, denoted as Where K represents the number of participants;

[0012] Step 2.2: Aggregate and predict in, It is the top-level model, Aggregate() is the aggregation function;

[0013] Step 2.3: Initialize the reconstruction loss set loss rec and prediction set is empty;

[0014] Step 2.4: Traverse each participant and perform the following operations:

[0015] Step 2.4.1: Mask the embedding of the current participant k;

[0016] Step 2.4.2: Preprocess the embedding and add position information;

[0017] Step 2.4.3: Using Latent Mask Autoencoder Model Reconstruct the embedding and calculate the reconstruction loss loss k ;

[0018] Step 2.4.4: Use the top model Make predictions on the reconstructed embeddings,

[0019] Step 2.4.5: Reconstruction loss loss k and the reconstructed prediction Record the loss set loss rec and prediction set middle;

[0020] Step 2.5: Filter out loss rec The participant with the greatest reconstruction loss adv;

[0021] Step 2.6: If the reconstruction loss of adv is greater than the threshold ε and only the prediction after adv reconstruction is not equal to The attacker is adv, and the prediction for anti-backdoor is Otherwise, there is no attacker and the prediction for anti-backdoor is

[0022] As an option, the embedding is preprocessed as described in step 2.4.2, and M is denoted by e i A mask with the same shape, where the dimension corresponding to the selected embedding is set to 0 and the other dimensions are set to 1; the masked embedding is padded with 0 to hide the embedding, and then combined with the indicator vector s∈{0,1} K Splicing is used to tell the latent mask autoencoder the embedding position that is masked from the input and needs to be reconstructed, i.e. Where s is derived from M, the blocked party is 0, and the other parties are 1; and Concatenate() represent matrix Hadamard product and concatenation operations respectively.

[0023] Preferably, the latent mask autoencoder is a trained latent mask autoencoder; the training process comprises the following steps:

[0024] (1) Pretreatment;

[0025] If there are K participants, record is the embedding of sample x by each participant, M is the embedding of sample x by each participant, and M is the embedding of sample x by each participant. i A mask with the same shape, where the dimension corresponding to the selected embedding is set to 0 and the other dimensions are set to 1; the masked embedding is padded with 0 to hide the embedding, and then combined with the indicator vector s∈{0,1} K Splicing is used to tell the latent mask autoencoder the embedding position that is masked from the input and needs to be reconstructed, i.e. Where s is derived from M, the blocked party is 0, and the other parties are 1; and Concatenate() represent matrix Hadamard product and concatenation operation respectively;

[0026] (2) Add position encoding to the pre-processed embedding, i.e. i ′=e i ′+v pos ; where v pos is a trainable position encoding vector.

[0027] (3) Update the latent mask autoencoder;

[0028] The latent mask autoencoder takes a batch of embeddings E' = {e i ′,..., e i+b-1 ′} as input and reconstructs the masked embeddings The purpose of training the latent mask autoencoder is to minimize the mean square error (MSE) between the masked embeddings and their reconstructed embeddings, so the loss function used in training is:

[0029]

[0030] where E = {e i ,..., e i+b-1} represents the original embedding data before being masked, and J is a mask similar to M, but all dimensions of data are set to 1.

[0031] The technical scheme adopted by the device of the application is: an electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the semantic consistency-based longitudinal federated learning backdoor defense method when executing the program.

[0032] The technical scheme adopted by the product of the application is: a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the semantic consistency-based longitudinal federated learning backdoor defense method.

[0033] The application further provides a computer program product comprising a computer program, wherein the computer program is executed by a processor to implement the semantic consistency-based longitudinal federated learning backdoor defense method.

[0034] Compared with the prior art, the application has the following advantages:

[0035] (1) This paper studies the risk of backdoor attacks in VFL and, based on the insight that such attacks may lead to potential embedding inconsistencies, proposes a backdoor defense technique for vertical federated learning. This defense does not rely on any auxiliary data, makes no assumptions about the backdoor attack method or model architecture, and has minimal impact on model accuracy. It can successfully defend against backdoor attacks, enabling attacker identification and robust prediction.

[0036] (2) The present invention designs a latent mask autoencoder to evaluate the semantic consistency between latent embeddings. In addition, the latent mask autoencoder is used to identify attackers and reconstruct clean embeddings, thereby achieving robust prediction against backdoor attacks.

[0037] (3) In combination with the characteristics of vertical federated learning, the present invention redefines the characteristics and goals required for backdoor defense in vertical federated learning: 1) follow the principles of VFL, such as not directly sharing data between participants, 2) be able to distinguish malicious inputs in the top-level model input and identify attackers, and 3) be able to make correct predictions when there is an attack. In addition, the defense should not significantly affect the model accuracy under clean input. Based on this, the present invention proposes two new backdoor defense evaluation indicators: attacker identification accuracy IA and robust accuracy RA, which are used to comprehensively measure the effectiveness of vertical federated learning backdoor defense technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The technical solution of the present invention is further illustrated below using embodiments and specific implementation methods. In addition, some drawings are also used in the process of illustrating the technical solution. For those skilled in the art, other drawings and the intention of the present invention can be obtained based on these drawings without making any creative efforts.

[0039] Figure 1 This is a schematic diagram of a defenseless deployment scenario for vertical federated learning (VFL) according to an embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram of the defense method according to an embodiment of the present invention;

[0041] Figure 3 2 is a structural diagram of a latent mask autoencoder according to an embodiment of the present invention;

[0042] Figure 4 Comparative data of defense experiment results on multiple data sets of the embodiments of the present invention;

[0043] Figure 5 This is the attacker identification accuracy data of the method of the embodiment of the present invention in different scenarios;

[0044] Figure 6Robustness accuracy comparison data of the defense method according to the embodiment of the present invention;

[0045] Figure 7 Schematic diagram of model output visualization used in the experiment of the embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to facilitate ordinary technicians in this field to understand and implement the present invention, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0047] Vertical federated learning (VFL) Figure 1 Figure 1 shows a simple scenario involving three parties. A typical VFL system consists of one or more passive parties and an active party, each with a bottom-level model. The active party has both a bottom-level model and a top-level model. During training and inference, the passive parties map their local data to embeddings, which are then shared with the active party for prediction. In the presence of a backdoor attack, one of the passive parties acts as the attacker, while the active party acts as the defender. During training, the attacker can manipulate local information such as data, models, and embeddings to inject a backdoor into the top-level model. The attacker's goal is to use malicious embeddings to mislead the top-level model into making incorrect predictions during inference.

[0048] The unique structure of VFL presents several challenges for backdoor defense techniques. First, the features of each data sample in VFL are owned by different parties, and the defender cannot access the features owned by other parties. Therefore, data preprocessing or cleansing is impossible. Second, the active party relies on the passive party's local models but cannot directly access these models, making it impossible to defend against the entire VFL model. Third, the defender relies on the passive party to provide embeddings and has no way of knowing whether the received embeddings are clean or abnormal, making it impossible to obtain clean auxiliary data.

[0049] This paper investigates the risk of backdoor attacks in VFL. Based on the discovery that backdoor attacks can lead to semantic inconsistencies in embeddings, we propose a novel defense technique that does not rely on any auxiliary data and makes no assumptions about backdoor attacks or model structure. We develop a latent mask autoencoder to assess the semantic consistency between the embeddings of each participant. Furthermore, this autoencoder is used for attacker identification and embedding reconstruction, achieving robust prediction of backdoored samples.

[0050] Please see Figure 2 This embodiment provides a backdoor defense method for longitudinal federated learning based on semantic consistency, including the following steps:

[0051] Step 1: Use a latent mask autoencoder to evaluate the semantic consistency of the embeddings provided by different vertical federated learning participants;

[0052] Please see Figure 3 In one embodiment, the latent mask autoencoder includes multiple Transformer-based encoder modules, which adopt a skip connection design of a multi-stage encoder to gradually reconstruct the masked input data through multiple encoders. First, the input data is masked to block part of the information and produce partially lost or damaged data as input. Each encoder module receives the output of the previous layer, gradually encodes and extracts features from the input, and introduces skip connections between encoder layers at preset distance intervals, so that the output of each encoder is not only passed to the next encoder, but also the output of the encoder is additionally passed to the encoder after the preset distance interval.

[0053] This example introduces a Transformer-based latent mask autoencoder (LMAE), which leverages the semantic information carried by embeddings and is trained to reconstruct each embedding given other embeddings as input. A traditional MAE slices the input content and maps it to a feature space, then uses a decoder to reconstruct and remap it back to the input space. Unlike traditional MAEs, the latent mask autoencoder operates in the latent space (i.e., feature space), does not involve a mapping between the input space and the latent space, and is trained on the embeddings produced by the underlying model.

[0054] The latent mask autoencoder of this embodiment first masks the input data, shielding some information and creating partially lost or damaged data as input. The latent mask autoencoder comprises multiple encoder modules, each of which receives the output of the previous layer and progressively encodes and extracts features from the input. These encoders are based on the Transformer encoder and are used to capture different levels of information from the input. The latent mask autoencoder uses a multi-level encoder skip connection design, introducing connections between distant encoder layers. Taking a six-layer encoder as an example, the latent mask autoencoder not only passes the output of each encoder to the next encoder, but also passes the output of the encoder three layers further away. The multi-level information fusion enabled by skip connections enables the model to more accurately reconstruct missing information when processing masked data. Global features captured by early layers can be reused in later layers. Fusion of feature information at different levels across multiple encoders addresses information loss issues in deep networks and enhances the latent mask autoencoder's reconstruction capabilities when processing incomplete data.

[0055] Step 2: Use semantic consistency assessment to identify malicious embeddings and identify the attacker;

[0056] In one implementation, each time the active party receives a set of embeddings from the passive party, it checks these embeddings one by one, using the reconstruction error to evaluate the semantic consistency of each embedding with the other embeddings; if the largest reconstruction error in the group is greater than a pre-set threshold ε, then the participant corresponding to the embedding is considered a potential attacker; if no reconstruction loss in the group exceeds ε, then the set of embeddings is considered clean.

[0057] While the loss distributions of clean and backdoor embeddings are highly discriminative for attack detection, they often overlap slightly, leading to false positives and false negatives in practice. To alleviate this problem, this embodiment further verifies each identified potential attacker by comparing the prediction results before and after the embedding reconstruction. Specifically, if the potential attacker's embedding is masked and reconstructed using the LMAE model, it will change the prediction results of the top-level model, while the same operation on other embeddings will not affect the prediction results, then the potential attacker is confirmed as a real attacker.

[0058] After identifying the attacker, the embedding reconstructed by the LMAE model can be used to replace the attacker's embedding, thereby removing the backdoor semantics and enabling the top-level model to achieve robust prediction against backdoors.

[0059] In one embodiment, the specific implementation of step 2 includes the following steps:

[0060] Step 2.1: Receive each participant’s embedding of sample x, denoted as Where K represents the number of participants;

[0061] Step 2.2: Aggregate and predict in, is the top-level model (the classification model controlled by the active party), and Aggregate() is the aggregation function;

[0062] Step 2.3: Initialize the reconstruction loss set loss rec and prediction set is empty;

[0063] Step 2.4: Traverse each participant and perform the following operations:

[0064] Step 2.4.1: Mask the embedding of the current participant k;

[0065] Step 2.4.2: Preprocess the embedding and add position information;

[0066] The embedding is preprocessed, and M is recorded as iA mask with the same shape, where the dimension corresponding to the selected embedding is set to 0 and the other dimensions are set to 1; the masked embedding is padded with 0 to hide the embedding, and then combined with the indicator vector s∈{0,1} K Splicing is used to tell the latent mask autoencoder the embedding position that is masked from the input and needs to be reconstructed, i.e. Where s is derived from M, the blocked party is 0, and the other parties are 1; and Concatenate() represent matrix Hadamard product and concatenation operations respectively;

[0067] Step 2.4.3: Using Latent Mask Autoencoder Model Reconstruct the embedding and calculate the reconstruction loss loss k ;

[0068] Step 2.4.4: Use the top model Make predictions on the reconstructed embeddings,

[0069] Step 2.4.5: Reconstruction loss loss k and the reconstructed prediction Record the loss set loss rec and prediction set middle;

[0070] Step 2.5: Filter out loss rec The participant with the greatest reconstruction loss adv;

[0071] Step 2.6: If the reconstruction loss of adv is greater than the threshold ε and only the prediction after adv reconstruction is not equal to The attacker is adv, and the prediction for anti-backdoor is Otherwise, there is no attacker and the prediction for anti-backdoor is

[0072] In one embodiment, the embedding reconstructed by the latent mask autoencoder is used to replace the attacker's embedding, thereby removing the backdoor semantics, so that the top-level model (the classification model controlled by the active party) can achieve robust prediction against backdoors.

[0073] This embodiment introduces a Transformer-based latent mask autoencoder (LMAE), which utilizes the semantic information contained in the embeddings provided by each participant and reconstructs an embedding given other embeddings. Unlike traditional mask autoencoders used in the input space, LMAE is used in the latent space and is trained based on the embeddings generated by the underlying model. Specifically, the training of LMAE is controlled by the active party and is synchronized with the top-level model. The training process includes the following steps:

[0074] (1) Pretreatment;

[0075] First, randomly mask the embeddings received from the passive party during a training process. If there are K participants, record is the embedding of sample x by each participant, M is the embedding of sample x by each participant, and M is the embedding of sample x by each participant. i A mask with the same shape, where the dimensions corresponding to the selected embedding are set to 0 and the other dimensions are set to 1; the active party's embedding is considered clean and does not need to be masked. The masked embedding is filled with 0 to hide the embedding, and then combined with the indicator vector s∈{0,1} K Splicing is used to tell the latent mask autoencoder the embedding position that is masked from the input and needs to be reconstructed, i.e. Where s is derived from M, the blocked party is 0, and the other parties are 1; and Concatenate() represent matrix Hadamard product and concatenation operations respectively;

[0076] (2) Add positional encoding to the preprocessed embedding;

[0077] Position encoding is widely used in Transformer-based models to integrate position information into embeddings. In LMAE, each embedding corresponds to a unique feature subset, so this example introduces a trainable position encoding vector v pos To encode the identities of different participants, that is, e i ′=e i ′+v pos .

[0078] (3) Update the latent mask autoencoder;

[0079] The latent mask autoencoder embeds a batch of E′={e i ′,...,e i+b-1 ′} as input and reconstructed the masked embedding The goal of training the latent mask autoencoder is to minimize the mean squared error (MSE) between the masked embedding and its reconstructed embedding, so the loss function used in training is:

[0080]

[0081] Where E={e i ,...,e i+b-1} represents the original embedding before masking, and J is a mask similar to M, but with all dimensions set to 1. The gradient of this loss does not backpropagate through the embeddings and therefore has no impact on model training for other participants. Intuitively, if the reconstruction error is small, then the correlation and semantic consistency between the masked embedding and the other embeddings are high, and vice versa.

[0082] The present invention is further described below through specific experiments.

[0083] This experiment selected multiple vertical federated learning backdoor attack methods for experiments, including GA, SDDP, VILLAIN, RPA, LR-BA, TECB, and DPBA. For comparative defense methods, this experiment selected Teco, BaDExpert, and DAE. The datasets include four datasets: CIFAR-10, CINIC-10, Yahoo-Answer, and BHI. This experiment selected two commonly used indicators for backdoor defense, CA and ASR. CA represents the accuracy of clean samples, and ASR represents the success rate of backdoor attacks. For the BHI dataset, this experiment used the F1 score to evaluate the model accuracy, and used CF1 to represent the F1 score of clean data. Considering that backdoor defense technology involves a balance between CA and ASR, this experiment defines the DER indicator to comprehensively evaluate the defense technology. The definition is as follows:

[0084]

[0085] In vertical federated learning, backdoor defense technology also needs to be able to identify attackers and achieve robust predictions. Therefore, this experiment proposes two new metrics: attacker identification accuracy (IA) and robust accuracy (RA). IA represents the percentage of backdoor samples that correctly identify the attacker. RA represents the percentage of backdoor samples that are correctly classified after clean embedding reconstruction.

[0086] The experimental results are as follows Figure 4 As shown in the figure, where ND represents the undefended scenario. Successful defense methods are highlighted in green, and the highest CA, CF1, and DER values ​​in the successful defense are shown in bold. The defense is considered successful if the ASR is lower than 20% in the multi-class classification task and lower than 30% in the binary classification task. The method of the present invention can significantly reduce ASR without affecting CA or CF1. The average CA reduction rates of CIFAR-10, CINIC-10, and Yahoo Answers are 0.42%, 0.61%, and 2.03%, respectively, and the average CF1 reduction rate of BHI is 0.12%, successfully defending against all attacks.

[0087] Furthermore, it significantly outperforms baseline methods against input-space attacks. DAE performs poorly against DPBA and RPA and struggles to defend against SDDP and VILLAIN on some datasets, demonstrating its lack of robustness against various attack strategies. BaDExpert shows some effectiveness against VILLAIN, but due to its inherent flaws, its performance against other attacks remains insufficient. Teco performs the worst among the compared methods.

[0088] The defense method of the present invention can identify attackers, but the three comparison methods are not able to correctly identify attackers from backdoor samples. This experiment evaluates the attacker identification accuracy of our defense method when the number of participants K is 4 and 8 respectively. The results are as follows: Figure 5 As shown in the figure, the attacker identification accuracy of this defense method is as high as 97.13%. Correctly identifying attackers helps clarify responsibilities and further adjust cooperation strategies.

[0089] This experiment conducted robust accuracy experiments on three multi-classification datasets. The results are as follows Figure 6 The classification accuracy of the method of the present invention for backdoor samples exceeds 54.85%, and is as high as 75.92%, an improvement of 75.84%.

[0090] This experiment visualizes the model output of backdoor samples of 7 attack methods in the CIFAR-10 dataset. The results are as follows: Figure 7 As shown in the figure, in an unprotected scenario, the backdoor attack can effectively shift the model output of backdoor samples from their original category to the target category. After adopting the defense method of the present invention, the model output of these samples is restored from the target category to their original category, showing a clear inter-class gap. This shows that the defense method of the present invention effectively resists backdoor attacks and ensures robust prediction of backdoor samples.

[0091] This paper proposes a semantically consistent backdoor defense technique for vertical federated learning. It designs a latent masked autoencoder (LMAE) to evaluate the semantic consistency of embeddings provided by different vertical federated learning participants. Based on the LMAE, an algorithm is proposed to identify attackers and perform robust predictions against backdoor attacks while simultaneously defending against them.

[0092] This paper proposes a semantic consistency-based longitudinal federated learning backdoor defense technology that does not require access to the original data or the passive party's local model and does not rely on clean auxiliary data. The defense technology of the present invention mainly consists of two parts: semantic consistency evaluation and anti-backdoor prediction. The semantic consistency between embeddings is evaluated by using a latent mask autoencoder (LMAE) trained in parallel with the top model. The encoder is designed to reconstruct the embedding of a specified party from the embeddings of other parties. During inference, malicious embeddings are identified with the help of semantic consistency evaluation, and clean embeddings are reconstructed to achieve robust prediction against backdoor attacks.

[0093] It should be understood that the above-described embodiments are part of the embodiments of the present application, but not all the embodiments. In addition, the technical features of each embodiment or individual embodiments provided by the present application can be combined with each other to form a feasible technical solution, and such combination is not restricted by the order of steps and / or structure mode, but must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or unfeasible, it should be considered that the combination of technical solutions does not exist, nor within the scope of protection required by the present application.

[0094] It should be understood that the above description of the preferred embodiments is more detailed, and therefore should not be considered as limiting the scope of patent protection of the present application. Those skilled in the art can make substitutions or modifications without departing from the scope of protection claimed by the present application, which falls within the scope of protection of the present application. The scope of protection of the present application shall be subject to the appended claims.

Claims

1. A backdoor defense method based on semantic consistency vertical federated learning, characterized by: The following steps are involved: Step 1: Design a latent mask autoencoder to evaluate the semantic consistency of the embeddings provided by different vertical federated learning participants; The latent mask autoencoder comprises multiple Transformer-based encoder modules, employing a multi-stage encoder skip connection design to gradually reconstruct masked input data through multiple encoders. First, the input data is masked to block some information, creating partially lost or damaged data as input. Each encoder module receives the output of the previous layer, gradually encodes and extracts features from the input, and simultaneously introduces skip connections between encoder layers at preset distance intervals, passing the output of each encoder not only to the next encoder but also to the encoders beyond the preset distance interval. Step 2: Use semantic consistency assessment to identify malicious embeddings and identify the attacker; Each time the active party receives a set of embeddings from the passive party, it checks these embeddings one by one and uses the reconstruction error to evaluate the semantic consistency of each embedding with other embeddings; if the largest reconstruction error in the group is greater than a pre-set threshold , then the participant corresponding to the embedding will be regarded as a potential attacker; If no reconstruction loss in the group exceeds , then this set of embeddings is considered clean.

2. The backdoor defense method based on semantic consistency vertical federated learning according to claim 1 is characterized in that: The specific implementation of step 2 includes the following steps: Step 2.1: Receive samples from each participant The embedding of ,in K Indicates the number of participants; Step 2.2: Aggregate and predict ;in, It is the top-level model, Aggregate() is the aggregation function; Step 2.3: Initialize the reconstruction loss set and prediction set is empty; Step 2.4: Traverse each participant and perform the following operations: Step 2.4.1: Block the current party Embedding; Step 2.4.2: Embed Perform preprocessing and add location information; The embedding For preprocessing, first record For A mask with the same shape, where the dimension corresponding to the selected embedding is set to 0 and the other dimensions are set to 1; the masked embedding is filled with 0 to hide the embedding, and then compared with the indicator vector Splicing is used to tell the latent mask autoencoder the embedding position that is masked from the input and needs to be reconstructed, i.e. ;in, s Depend on It turns out that the blocked participant is 0 and the other participants are 1; ⊗ and Concatenate() represent matrix Hadamard product and concatenation operations respectively; Step 2.4.3: Using Latent Mask Autoencoder Model Embed Perform reconstruction and calculate reconstruction loss ; Step 2.4.4: Use the top model The reconstructed embedding Make predictions, ; Step 2.4.5: Reconstruction loss and the reconstructed prediction Record to loss collection and prediction set middle; Step 2.5: Filter out The party with the greatest reconstruction losses ; Step 2.6: If The reconstruction loss is greater than the threshold And only The reconstructed prediction is not equal to , then the attacker is , the prediction for anti-backdoor is Otherwise, there is no attacker and the prediction for anti-backdoor is .

3. The method for backdoor defense based on semantic consistency vertical federated learning according to any one of claims 1-2, characterized in that: The latent mask autoencoder is a trained latent mask autoencoder; The training process consists of the following steps: (1) Pretreatment; If there is Participants, record For each participant to sample Embedded, For A mask with the same shape, where the dimension corresponding to the selected embedding is set to 0 and the other dimensions are set to 1; the masked embedding is filled with 0 to hide the embedding, and then compared with the indicator vector Splicing is used to tell the latent mask autoencoder the embedding position that is masked from the input and needs to be reconstructed, i.e. ;in, s Depend on It turns out that the blocked participant is 0 and the other participants are 1; and Concatenate() represent matrix Hadamard product and concatenation operations respectively; (2) Add positional encoding to the preprocessed embedding, i.e. ;in, is a trainable position encoding vector; (3) Update the latent mask autoencoder; The latent mask autoencoder embeds a batch of As input, and reconstructed the masked embedding The goal of training the latent mask autoencoder is to minimize the mean square error (MSE) between the masked embedding and its reconstructed embedding, so the loss function used in training is: ； in, represents the original embedded data before being masked, Is a Similar mask, but with all dimensions set to 1.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the longitudinal federated learning backdoor defense method based on semantic consistency is implemented as described in any one of claims 1 to 3.

5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for backdoor defense based on longitudinal federated learning based on semantic consistency as described in any one of claims 1 to 3 is implemented.

6. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for backdoor defense based on longitudinal federated learning based on semantic consistency as described in any one of claims 1 to 3 is implemented.

Images