Longitudinal federated learning back door defense method, device and product based on semantic consistency

Abstract

The invention discloses a longitudinal federated learning backdoor defense method, device and product based on semantic consistency. The method comprises the following steps: firstly, evaluating embedded semantic consistency provided by different longitudinal federated learning participants by adopting a potential mask auto-encoder; then malicious embedding is identified by means of semantic consistency evaluation, and an attacker is identified; and finally, the embedding reconstructed by the potential mask auto-encoder is utilized to replace the embedding of an attacker, so that backdoor semantics are removed, and a top-layer model realizes anti-backdoor robust prediction. Based on the discovery that backdoor attacks may cause inconsistent embedded semantics, a new defense technology is provided, and the technology does not depend on any auxiliary data and has no prerequisite hypothesis for backdoor attacks or model structures. According to the method, a potential mask auto-encoder is developed to evaluate the semantic consistency between the participants in embedding. In addition, the auto-encoder is used for attacker identification and embedded reconstruction, thereby implementing robust prediction for backdoor samples.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence security technology, and relates to a vertical federated learning backdoor defense method, device and product, and specifically to a vertical federated learning backdoor defense method, device and product based on semantic consistency. Background Art

[0002] As a distributed machine learning framework, federated learning allows multiple data owners (such as different companies and organizations) to collaborate on training a shared model without sharing the original data. Federated learning aims to solve the problem of data silos, promote efficient use of data and optimization of models, while complying with strict privacy protection regulations. According to how data is distributed among multiple parties, federated learning can be divided into two main types: horizontal federated learning and vertical federated learning. Participants in vertical federated learning (VFL) usually have different characteristics in the same set of data samples, so VFL is more suitable for collaboration between different types of organizations, such as banks and e-commerce platforms.

[0003] Recent studies have shown that VFL is vulnerable to backdoor attacks. Attackers can manipulate its local information such as data, models, embeddings, etc. during training to inject backdoors into the top-level model, with the goal of misleading the top-level model into making incorrect predictions at inference time using malicious embeddings. The key challenges facing backdoor defense techniques in VFL are the lack of information about the features and models of other parties, as well as the lack of clean auxiliary datasets. Summary of the invention

[0004] In order to solve the technical problem of backdoor attack risk in VFL, the present invention proposes a backdoor defense method, device and product based on semantic consistency vertical federated learning based on the discovery that backdoor attack may lead to inconsistent embedded semantics.

[0005] The technical solution adopted by the method of the present invention is: a method for backdoor defense based on longitudinal federated learning of semantic consistency, comprising the following steps:

[0006] Step 1: Use latent mask autoencoders to evaluate the semantic consistency of embeddings provided by different vertical federated learning participants;

[0007] The latent mask autoencoder comprises a plurality of Transformer-based encoder modules, adopts a skip connection design of a multi-level encoder, and gradually reconstructs the masked input data through a plurality of encoders; firstly, the input data is masked to shield part of the information, and produce part of the lost or damaged data as input; each encoder module receives the output of the previous layer, and gradually encodes and extracts features from the input, and at the same time introduces a skip connection between the encoder layers at a preset distance interval, and transmits the output of each encoder not only to the next encoder, but also to the encoder after the preset distance interval;

[0008] Step 2: Use semantic consistency assessment to identify malicious embeddings and identify the attacker.

[0009] Preferably, in step 2, each time the active party receives a set of embeddings from the passive party, it checks these embeddings one by one and evaluates the semantic consistency of each embedding with other embeddings based on the reconstruction error; if the largest reconstruction error in the group is greater than a preset threshold ε, then the participant corresponding to the embedding will be regarded as a potential attacker; if no reconstruction loss in the group exceeds ε, then the set of embeddings is considered clean.

[0010] Preferably, the specific implementation of step 2 includes the following steps:

[0011] Step 2.1: Receive each participant’s embedding of sample x, denoted as Where K represents the number of participants;

[0012] Step 2.2: Aggregate and predict in, It is the top-level model, and Aggregate() is the aggregation function;

[0013] Step 2.3: Initialize the reconstruction loss set loss rec and the prediction set is empty;

[0014] Step 2.4: Traverse each participant and perform the following operations:

[0015] Step 2.4.1: Mask the embedding of the current participant k;

[0016] Step 2.4.2: Preprocess the embedding and add position information;

[0017] Step 2.4.3: Using Latent Mask Autoencoder Model Reconstruct the embedding and calculate the reconstruction loss loss k ;

[0018] Step 2.4.4: Use the top model Make predictions on the reconstructed embeddings,

[0019] Step 2.4.5: Reconstruct loss loss k and the reconstructed prediction Record the loss set loss rec and the prediction set middle;

[0020] Step 2.5: Filter out loss rec The party with the greatest reconstruction loss adv;

[0021] Step 2.6: If the reconstruction loss of adv is greater than the threshold ε and only if the prediction after adv reconstruction is not equal to The attacker is adv, and the prediction for anti-backdoor is Otherwise, there is no attacker and the prediction for anti-backdoor is

[0022] Preferably, the embedding is preprocessed as described in step 2.4.2, and M is denoted by e i A mask with the same shape, where the dimension corresponding to the selected embedding is set to 0 and the other dimensions are set to 1; the masked embedding is padded with 0 to hide the embedding, and then combined with the indicator vector s∈{0,1} K splicing, which is used to tell the latent mask autoencoder the embedding positions that are masked from the input and need to be reconstructed, i.e. Where s is derived from M, the blocked party is 0, and the other parties are 1; and Concatenate() represent matrix Hadamard product and concatenation operations respectively.

[0023] Preferably, the latent mask autoencoder is a trained latent mask autoencoder; the training process comprises the following steps:

[0024] (1) Pretreatment;

[0025] If there are K participants, record is the embedding of sample x by each participant, M is the same as e i A mask with the same shape, where the dimension corresponding to the selected embedding is set to 0 and the other dimensions are set to 1; the masked embedding is padded with 0 to hide the embedding, and then combined with the indicator vector s∈{0,1} K splicing, which is used to tell the latent mask autoencoder the embedding positions that are masked from the input and need to be reconstructed, i.e. Where s is derived from M, the blocked party is 0, and the other parties are 1; and Concatenate() represent matrix Hadamard product and concatenation operations respectively;

[0026] (2) Add the positional encoding to the preprocessed embedding, i.e., e i ′=e i ′+v pos ; where v pos is a trainable position encoding vector;

[0027] (3) Update the latent mask autoencoder;

[0028] The latent mask autoencoder embeds a batch of E′={e i ′, ..., e i+b-1 ′} as input and reconstructed the masked embedding The goal of training the latent mask autoencoder is to minimize the mean square error (MSE) between the masked embedding and its reconstructed embedding, so the loss function used in training is:

[0029]

[0030] Where E = {e i , ..., e i+b-1} represents the original embedded data before masking, and J is a mask similar to M, but the data in all dimensions are set to 1.

[0031] The technical solution adopted by the device of the present invention is: an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the longitudinal federated learning backdoor defense method based on semantic consistency is implemented.

[0032] The technical solution adopted by the product of the present invention is: a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the semantic consistency-based vertical federated learning backdoor defense method is implemented.

[0033] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the semantic consistency-based vertical federated learning backdoor defense method.

[0034] Compared with the prior art, the beneficial effects of the present invention are:

[0035] (1) This paper studies the risk of backdoor attacks in VFL and proposes a backdoor defense technology for vertical federated learning for the first time based on the insight that such attacks may lead to potential embedding inconsistencies. This defense does not rely on any auxiliary data, does not have any assumptions about the backdoor attack method or model architecture, has little impact on the model accuracy, can successfully defend against backdoor attacks, and achieve attacker identification and robust prediction.

[0036] (2) The present invention designs a latent mask autoencoder to evaluate the semantic consistency between latent embeddings. In addition, the latent mask autoencoder is used to identify attackers and reconstruct clean embeddings, thereby achieving robust prediction against backdoor attacks.

[0037] (3) In combination with the characteristics of vertical federated learning, the present invention redefines the characteristics and goals required for backdoor defense in vertical federated learning: 1) following the principles of VFL, such as not directly sharing data between participants, 2) being able to distinguish malicious inputs in the top-level model input and identify attackers, and 3) being able to make correct predictions when attacks exist. In addition, the defense should not significantly affect the model accuracy under clean inputs. Based on this, the present invention proposes two new backdoor defense evaluation indicators: attacker identification accuracy IA and robust accuracy RA, which are used to comprehensively measure the effectiveness of vertical federated learning backdoor defense technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The technical solution of the present invention is further described below using embodiments and specific implementation methods. In addition, in the process of describing the technical solution, some drawings are also used. For those skilled in the art, other drawings and the intention of the present invention can also be obtained based on these drawings without paying creative work.

[0039] Figure 1 A schematic diagram of a defenseless deployment scenario of vertical federated learning (VFL) according to an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram of a defense method according to an embodiment of the present invention;

[0041] Figure 3 Schematic diagram of the latent mask autoencoder according to an embodiment of the present invention;

[0042] Figure 4 Comparative data of defense experiment results of the embodiments of the present invention on multiple data sets;

[0043] Figure 5 The attacker identification accuracy data of the method of the embodiment of the present invention in different scenarios;

[0044] Figure 6Robustness accuracy comparison data of the defense method of the embodiment of the present invention;

[0045] Figure 7 A schematic diagram of model output visualization used in the experiment of an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0047] Vertical federated learning (VFL) Figure 1 As shown in Figure 1, a simple scenario involving three parties is shown. A typical VFL system consists of one or more passive parties and an active party, each of which has a bottom-level model, and the active party has both a bottom-level model and a top-level model. During the training and inference phases, the passive party maps its local data to embeddings, which are then shared with the active party for prediction. When there is a backdoor attack, one of the passive parties is the attacker and the active party acts as the defender. The attacker can manipulate its local information, such as data, model, embedding, etc. during the training process to inject a backdoor into the top-level model, with the goal of misleading the top-level model to make incorrect predictions using malicious embeddings during inference.

[0048] The special structure of VFL brings some challenges to backdoor defense technology. First, the features of each data sample in VFL are owned by different participants, and the defender cannot obtain the features owned by other participants. Therefore, the data cannot be preprocessed or purified. Second, the active party relies on the local models of the passive party, but cannot directly access these models, making it impossible to defend the entire VFL model. Third, the defender relies on the passive party to provide embeddings, and cannot know whether the received embeddings are clean or abnormal, so the defender cannot obtain clean auxiliary data.

[0049] This paper studies the risk of backdoor attacks in VFL and proposes a new defense technique based on the discovery that backdoor attacks may lead to inconsistent embedding semantics. The technique does not rely on any auxiliary data and has no assumptions about backdoor attacks or model structures. A latent mask autoencoder is developed to evaluate the semantic consistency between the embeddings of each participant. In addition, the autoencoder is used for attacker identification and embedding reconstruction, thereby achieving robust prediction of backdoor samples.

[0050] Please see Figure 2 , this embodiment provides a semantic consistency-based longitudinal federated learning backdoor defense method, including the following steps:

[0051] Step 1: Use latent mask autoencoders to evaluate the semantic consistency of embeddings provided by different vertical federated learning participants;

[0052] Please see Figure 3 In one embodiment, the latent mask autoencoder includes a plurality of Transformer-based encoder modules, adopts a skip connection design of a multi-stage encoder, and gradually reconstructs the masked input data through a plurality of encoders; first, the input data is masked to shield part of the information, and partially lost or damaged data is produced as input; each encoder module receives the output of the previous layer, and gradually encodes and extracts features from the input, while introducing a skip connection between the encoder layers at a preset distance interval, and not only passes the output of each encoder to the next encoder, but also passes the output of the encoder to the encoder after the preset distance interval;

[0053] This embodiment introduces a Transformer-based latent mask autoencoder (LMAE), which uses the semantic information carried by the embedding and is trained to reconstruct each embedding given other embeddings as input. The traditional MAE cuts the input content and maps it to the feature space, then reconstructs it using a decoder and remaps it back to the input space. Unlike the traditional MAE, the latent mask autoencoder operates in the latent space (i.e., the feature space), does not involve the mapping between the input space and the latent space, and is trained on the embedding produced by the bottom model.

[0054] The latent mask self-encoder of this embodiment first masks the input data, shields part of the information, and produces some lost or damaged data as input. The latent mask self-encoder includes multiple encoder modules, each of which receives the output of the previous layer and gradually encodes and extracts features from the input. These encoders are based on Transformer encoders and are used to capture different levels of input information. The latent mask self-encoder adopts a jump connection design of a multi-level encoder, introducing connections between distant encoder levels. Taking a 6-layer encoder as an example, the latent mask self-encoder not only passes the output of each encoder to the next encoder, but also passes the output of the encoder to the encoder 3 layers away from it. The multi-level information fusion brought by the jump connection enables the model to reconstruct the lost information more accurately when processing masked data. The global features captured by the early layers can be reused in the later layers, and the feature information of different levels is fused between multiple encoders, which solves the problem of information loss in deep networks and enhances the reconstruction ability of the latent mask self-encoder when processing incomplete data.

[0055] Step 2: Use semantic consistency assessment to identify malicious embeddings and identify attackers;

[0056] In one implementation, each time the active party receives a set of embeddings from the passive party, it checks these embeddings one by one, and uses the reconstruction error to evaluate the semantic consistency of each embedding with other embeddings; if the largest reconstruction error in the group is greater than a pre-set threshold ε, then the participant corresponding to the embedding is considered a potential attacker; if no reconstruction loss in the group exceeds ε, then the set of embeddings is considered clean.

[0057] Although the loss distributions of clean embeddings and backdoor embeddings are highly discriminative for attack detection, they often overlap slightly, leading to false positives and false negatives in practice. To alleviate this problem, this embodiment further verifies each identified potential attacker by comparing the prediction results before and after the embedding reconstruction. Specifically, if the embedding of a potential attacker is masked and reconstructed using the LMAE model, the prediction results of the top-level model will change, while the same operation on other embeddings will not affect the prediction results, then the potential attacker is confirmed as a real attacker.

[0058] After identifying the attacker, the embedding reconstructed by the LMAE model can be used to replace the attacker’s embedding, thereby removing the backdoor semantics and enabling the top-level model to achieve robust prediction against backdoors.

[0059] In one embodiment, the specific implementation of step 2 includes the following steps:

[0060] Step 2.1: Receive each participant’s embedding of sample x, denoted as Where K represents the number of participants;

[0061] Step 2.2: Aggregate and predict in, is the top-level model (the classification model controlled by the active party), and Aggregate() is the aggregation function;

[0062] Step 2.3: Initialize the reconstruction loss set loss rec and the prediction set is empty;

[0063] Step 2.4: Traverse each participant and perform the following operations:

[0064] Step 2.4.1: Mask the embedding of the current participant k;

[0065] Step 2.4.2: Preprocess the embedding and add position information;

[0066] The embedding is preprocessed, and M is denoted by e iA mask with the same shape, where the dimension corresponding to the selected embedding is set to 0 and the other dimensions are set to 1; the masked embedding is padded with 0 to hide the embedding, and then combined with the indicator vector s∈{0,1} K splicing, which is used to tell the latent mask autoencoder the embedding positions that are masked from the input and need to be reconstructed, i.e. Where s is derived from M, the blocked party is 0, and the other parties are 1; and Concatenate() represent matrix Hadamard product and concatenation operations respectively;

[0067] Step 2.4.3: Using Latent Mask Autoencoder Model Reconstruct the embedding and calculate the reconstruction loss loss k ;

[0068] Step 2.4.4: Use the top model Make predictions on the reconstructed embeddings,

[0069] Step 2.4.5: Reconstruct loss loss k and the reconstructed prediction Record the loss set loss rec and the prediction set middle;

[0070] Step 2.5: Filter out loss rec The party with the greatest reconstruction loss adv;

[0071] Step 2.6: If the reconstruction loss of adv is greater than the threshold ε and only if the prediction after adv reconstruction is not equal to The attacker is adv, and the prediction for anti-backdoor is Otherwise, there is no attacker and the prediction for anti-backdoor is

[0072] In one embodiment, the embedding reconstructed by the latent mask autoencoder is used to replace the attacker's embedding, thereby removing the backdoor semantics, so that the top-level model (the classification model controlled by the active party) can achieve robust prediction against backdoors.

[0073] This embodiment introduces a Transformer-based latent mask autoencoder (LMAE), which utilizes the semantic information carried in the embeddings provided by each participant and reconstructs an embedding when other embeddings are input. Unlike traditional mask autoencoders used in the input space, LMAE is used in the latent space and is trained based on the embeddings generated by the underlying model. Specifically, the training of LMAE is controlled by the active party and is synchronized with the top-level model. The training process includes the following steps:

[0074] (1) Pretreatment;

[0075] First, randomly mask the embedding received from the passive party during training. If there are K participants, record is the embedding of sample x by each participant, M is the same as e i A mask with the same shape, where the dimension corresponding to the selected embedding is set to 0 and the other dimensions are set to 1; the active embedding is considered clean and does not need to be masked. The masked embedding is filled with 0 to hide the embedding, and then combined with the indicator vector s∈{0,1} K splicing, which is used to tell the latent mask autoencoder the embedding positions that are masked from the input and need to be reconstructed, i.e. Where s is derived from M, the blocked party is 0, and the other parties are 1; and Concatenate() represent matrix Hadamard product and concatenation operations respectively;

[0076] (2) Add positional encoding to the preprocessed embedding;

[0077] Position encoding is widely used in Transformer-based models to integrate position information into embeddings. In LMAE, each embedding corresponds to a unique feature subset, so this example introduces a trainable position encoding vector v pos To encode the identities of different participants, that is, e i ′=e i ′+v pos .

[0078] (3) Update the latent mask autoencoder;

[0079] The latent mask autoencoder embeds a batch of E′={e i ′, ..., e i+b-1 ′} as input and reconstructed the masked embedding The goal of training the latent mask autoencoder is to minimize the mean square error (MSE) between the masked embedding and its reconstructed embedding, so the loss function used in training is:

[0080]

[0081] Where E = {e i , ..., e i+b-1} represents the original embedding data before masking, and J is a mask similar to M, but with all dimensions set to 1. The gradient of this loss does not backpropagate through the embeddings, so it has no impact on the model training of other participants. Intuitively, if the reconstruction error is small, then the correlation and semantic consistency between the masked embedding and other embeddings are high, and vice versa.

[0082] The present invention is further described below through specific experiments.

[0083] This experiment selected multiple vertical federated learning backdoor attack methods for experiments, including GA, SDDP, VILLAIN, RPA, LR-BA, TECB, and DPBA; for comparative defense methods, this experiment selected Teco, BaDExpert, and DAE. The data set includes four data sets, CIFAR-10, CINIC-10, Yahoo-Answer, and BHI. This experiment selected two commonly used indicators for backdoor defense, CA and ASR. CA represents the accuracy of clean samples, and ASR represents the success rate of backdoor attacks. For the BHI data set, this experiment uses the F1 score to evaluate the model accuracy, and uses CF1 to represent the F1 score of clean data. Considering that backdoor defense technology involves a balance between CA and ASR, this experiment defines the DER indicator to comprehensively evaluate the defense technology. The definition is as follows:

[0084]

[0085] In vertical federated learning, backdoor defense technology also needs to be able to identify attackers and achieve robust predictions, so this experiment proposes two new indicators, attacker identification accuracy IA and robust accuracy RA. IA represents the percentage of backdoor samples that correctly identify attackers. RA represents the percentage of backdoor samples that are correctly classified after clean embedding reconstruction.

[0086] The experimental results are as follows Figure 4 As shown, where ND represents the undefended scenario. Successful defense methods are highlighted in green, and the highest CA, CF1, and DER values ​​in successful defenses are shown in bold. The defense is considered successful if the ASR is less than 20% in multi-class classification tasks and less than 30% in binary classification tasks. The method of the present invention can significantly reduce ASR without affecting CA or CF1. The average CA reduction rates of CIFAR-10, CINIC-10, and Yahoo Answers are 0.42%, 0.61%, and 2.03%, respectively, and the average CF1 reduction rate of BHI is 0.12%, successfully defending against all attacks.

[0087] In addition, it significantly outperforms the baseline methods in input space attacks. DAE performs poorly against DPBA and RPA, and has difficulty defending against SDDP and VILLAIN on some datasets, which shows that it lacks robustness in dealing with various attack strategies. BaDExpert shows some effectiveness against VILLAIN, but its performance against other attacks is still insufficient due to its own defects. Teco performs the worst among the compared methods.

[0088] The defense method of the present invention can identify attackers, but the three comparison methods do not have the ability to correctly identify attackers from backdoor samples. This experiment evaluates the attacker identification accuracy of our defense method when the number of participants K is 4 and 8 respectively. The results are as follows: Figure 5 As shown in the figure, the attacker identification accuracy of the defense method in this experiment is as high as 97.13%. Correctly identifying attackers is conducive to clarifying responsibilities and further adjusting cooperation strategies.

[0089] This experiment conducted robust accuracy experiments on three multi-classification datasets. The results are as follows Figure 6 The classification accuracy of the method of the present invention for backdoor samples exceeds 54.85%, and is as high as 75.92%, an improvement of 75.84%.

[0090] This experiment visualizes the model output of backdoor samples of 7 attack methods in the CIFAR-10 dataset. The results are as follows: Figure 7 As shown. It can be seen that in the unprotected scenario, the backdoor attack can effectively transfer the model output of the backdoor sample from the category to which it belongs to the target category. After adopting the defense method of the present invention, the model output of these samples is restored from the target category to the category to which they originally belonged, showing a clear inter-class interval. This shows that the defense method of the present invention effectively resists the backdoor attack and ensures the robust prediction of the backdoor sample.

[0091] This paper proposes a semantic consistency-based backdoor defense technology for vertical federated learning, and designs a latent masked autoencoder (LMAE) to evaluate the semantic consistency of embeddings provided by different vertical federated learning participants. Based on LMAE, an algorithm is proposed to identify attackers and perform robust prediction against backdoor attacks while defending against backdoor attacks.

[0092] The present invention proposes a longitudinal federated learning backdoor defense technology based on semantic consistency, which does not require access to the original data and the local model of the passive party, and does not rely on clean auxiliary data. The defense technology of the present invention mainly includes two parts: semantic consistency evaluation and anti-backdoor prediction. The semantic consistency between embeddings is evaluated by using a latent mask autoencoder LMAE trained in parallel with the top model, which aims to reconstruct the embedding of a specified party from the embeddings of other parties. During reasoning, malicious embeddings are identified with the help of semantic consistency evaluation, and clean embeddings are reconstructed to achieve robust prediction against backdoor attacks.

[0093] It should be understood that the embodiments described above are part of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. Such combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0094] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A backdoor defense method based on semantic consistency vertical federated learning, characterized in that: The following steps are involved: Step 1: Use latent mask autoencoders to evaluate the semantic consistency of embeddings provided by different vertical federated learning participants; The latent mask autoencoder comprises a plurality of Transformer-based encoder modules, adopts a skip connection design of a multi-level encoder, and gradually reconstructs the masked input data through a plurality of encoders; firstly, the input data is masked to shield part of the information, and produce part of the lost or damaged data as input; each encoder module receives the output of the previous layer, and gradually encodes and extracts features from the input, and at the same time introduces a skip connection between the encoder layers at a preset distance interval, and transmits the output of each encoder not only to the next encoder, but also to the encoder after the preset distance interval; Step 2: Use semantic consistency assessment to identify malicious embeddings and identify the attacker.

2. The backdoor defense method based on longitudinal federated learning of semantic consistency according to claim 1 is characterized by: In step 2, each time the active party receives a set of embeddings from the passive party, it checks these embeddings one by one and evaluates the semantic consistency of each embedding with other embeddings using the reconstruction error; if the largest reconstruction error in the group is greater than a pre-set threshold ε, then the participant corresponding to the embedding is considered a potential attacker; If no reconstruction loss in a group exceeds ε, then the set of embeddings is considered clean.

3. The backdoor defense method based on longitudinal federated learning based on semantic consistency according to claim 1 is characterized in that: The specific implementation of step 2 includes the following steps: Step 2.1: Receive each participant’s embedding of sample x, denoted as Where K represents the number of participants; Step 2.2: Aggregate and predict in, It is the top-level model, and Aggregate() is the aggregation function; Step 2.3: Initialize the reconstruction loss set loss rec and the prediction set is empty; Step 2.4: Traverse each participant and perform the following operations: Step 2.4.1: Mask the embedding of the current participant k; Step 2.4.2: Preprocess the embedding and add position information; Step 2.4.3: Using Latent Mask Autoencoder Model Reconstruct the embedding and calculate the reconstruction loss loss k ; Step 2.4.4: Use the top model Make predictions on the reconstructed embeddings, Step 2.4.5: Reconstruct loss loss k and the reconstructed prediction Record the loss set loss rec and the prediction set middle; Step 2.5: Filter out loss rec The party with the greatest reconstruction loss adv; Step 2.6: If the reconstruction loss of adv is greater than the threshold ε and only if the prediction after adv reconstruction is not equal to The attacker is adv, and the prediction for anti-backdoor is Otherwise, there is no attacker and the prediction for anti-backdoor is 4. The method for backdoor defense based on longitudinal federated learning based on semantic consistency according to claim 1 is characterized in that: Preprocess the embedding as described in step 2.4.2, and denote M as i A mask with the same shape, where the dimension corresponding to the selected embedding is set to 0 and the other dimensions are set to 1; the masked embedding is padded with 0 to hide the embedding, and then combined with the indicator vector s∈{0,1} K splicing, which is used to tell the latent mask autoencoder the embedding positions that are masked from the input and need to be reconstructed, i.e. Where s is derived from M, the blocked party is 0, and the other parties are 1; and Concatenate() represent matrix Hadamard product and concatenation operations respectively.

5. The method for backdoor defense based on longitudinal federated learning based on semantic consistency according to any one of claims 1 to 4, characterized in that: The latent mask autoencoder is a trained latent mask autoencoder; The training process includes the following steps: (1) Pretreatment; If there are K participants, record is the embedding of sample x by each participant, M is the same as e i A mask with the same shape, where the dimension corresponding to the selected embedding is set to 0 and the other dimensions are set to 1; the masked embedding is padded with 0 to hide the embedding, and then combined with the indicator vector s∈{0,1} K splicing, which is used to tell the latent mask autoencoder the embedding positions that are masked from the input and need to be reconstructed, i.e. Where s is derived from M, the blocked party is 0, and the other parties are 1; and Concatenate() represent matrix Hadamard product and concatenation operations respectively; (2) Add the positional encoding to the preprocessed embedding, i.e., e i ′=e i ′+v pos ; where v pos is a trainable position encoding vector; (3) Update the latent mask autoencoder; The latent mask autoencoder embeds a batch of E′={e i ′, ..., e i+b-1 ′} as input and reconstructed the masked embedding The goal of training the latent mask autoencoder is to minimize the mean square error (MSE) between the masked embedding and its reconstructed embedding, so the loss function used in training is: Where E = {e i , ..., e i+b-1 } represents the original embedded data before masking, and J is a mask similar to M, but the data in all dimensions are set to 1.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the semantic consistency-based vertical federated learning backdoor defense method as described in any one of claims 1 to 5.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the semantic consistency-based vertical federated learning backdoor defense method as described in any one of claims 1 to 5.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the semantic consistency-based vertical federated learning backdoor defense method as described in any one of claims 1 to 5.

Images