Online trigger filter text backdoor defense method based on hidden features

Abstract

The invention discloses an online trigger filter text backdoor defense method based on hidden features, and relates to the technical field of artificial intelligence security, and the method comprises the steps: obtaining a text classification data set D1, a sub-data set D2 and a classification neural network; using the D1 to train a classification neural network to obtain a text classification model; poisoning the D2 to obtain a poisoning data set D3; constructing a backdoor data set D4 to carry out backdoor attack on the text classification model to obtain a backdoor model; constructing a trigger data set D5 based on hidden layer data of the hidden layer of the backdoor model and the position of the trigger word; training a trigger detector model by using D5; and detecting and filtering the trigger words in the to-be-filtered text. The method does not need to depend on specific information of an attacked algorithm, the identification capability of various triggers is enhanced through hidden feature identification and word replacement technologies, meanwhile, the normal operation efficiency of a protected model during benign sample processing is ensured, and the safety and reliability of the whole system are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security technology, and particularly to an online trigger filter text backdoor defense method based on hidden features. Background Art

[0002] Natural Language Processing (NLP) is an artificial intelligence technology that enables computers to understand and process human natural language. In recent years, with the rapid development of deep learning and big data technologies, NLP technology has made remarkable progress and has been widely applied in many fields. For example, in text classification, NLP is used to automatically classify news articles, emails, or social media posts; chatbots use NLP to achieve human-computer conversations; sentiment analysis monitors public sentiment and social dynamics through NLP; and speech recognition converts speech into text to support functions such as intelligent assistants.

[0003] Among them, the text classification task, which usually involves assigning labels, categories, or tags to text bodies (sentences, paragraphs, or documents), as an important part of the natural language processing field, has attracted the attention of many researchers and has derived extensive application scenarios. For example, spam filtering, sentiment analysis, topic modeling, etc. all rely on efficient text classification algorithms. With the progress of NLP technology, these applications not only improve efficiency but also expand new possibilities, such as personalized recommendations, intelligent customer service, etc.

[0004] With the continuous expansion of the model scale and the increase in computing resources in the NLP field, the training of models is usually based on pre-trained models for fine-tuning of downstream tasks. This trend has significantly increased the threat of backdoor injection in the form of white-box attacks. At the same time, the text classification task, as an important task in the NLP field, has become one of the main targets of text backdoor attacks. In the backdoor attack of the text classification task, the attacker injects a backdoor into the model during the training phase and constructs toxic samples containing specific triggers during the attack, so that the model outputs the labels expected by the attacker when these samples are input.

[0005] Text backdoor defense methods are mainly divided into two categories: those acting on the model training stage and the model deployment stage. In the training stage, the defense measures usually involve detecting and discarding potential poisoned samples, and then retraining the model with the cleaned data; while in the deployment stage, it is to perform real-time detection on the input samples, and take discard or filtering measures after identifying the toxic samples. Based on relevant experiments, there are the following problems with the current defense methods in the deployment stage: (1) The defense method based on sample discrimination in the deployment stage has a high rejection rate. This method works by distinguishing between benign samples and toxic samples. Once a sample is identified as toxic, access is rejected. However, this method has a high rejection rate and may cause some clean samples to be mislabeled as toxic samples, affecting the normal usage experience of non-malicious users. (2) The defense based on data reconstruction in the deployment stage is prone to significant semantic transformation of samples. This method filters out possible triggers by reconstructing the input data, provides real-time processing capabilities when the actual target model is running online, reduces the rejection rate of input samples, and ensures the security of the target model. However, in the absence of precise trigger identification, this method may cause a large number of conversions of words in toxic samples, resulting in a significant change in the semantics of the original samples and affecting the performance of the target model. Summary of the Invention

[0006] The object of the present invention is to provide an online trigger filter text backdoor defense method based on hidden features that solves the above problems, has no rejection rate, has better detection rate and general defense effect.

[0007] To achieve the above object, the technical solution adopted by the present invention is as follows: An online trigger filter text backdoor defense method based on hidden features, comprising the following steps; S1. Obtain a text classification dataset D1, a sub-dataset D2, and a classification neural network. The text classification dataset D1 contains a total of C categories, and each category contains several clean samples; D2 is composed of several clean samples randomly selected from D1. The hidden layer of the classification neural network is L layers; S2. Train the classification neural network with D1 to obtain a text classification model for inputting clean samples and outputting their predicted categories; S3. Select a target label and a trigger, and poison D2 using a text backdoor attack method to obtain a poisoned dataset D3. The trigger is composed of multiple trigger words; S4. Combine the clean samples in D1 except D2 with D3 to form a backdoor dataset D4, and use D4 to perform a backdoor attack on the text classification model based on the text backdoor attack method to obtain a backdoor model; S5. For each sample in D4, extract the hidden layer data through the hidden layer of the backdoor model, and construct a trigger vector based on the positions of the trigger words in the trigger. Combine the hidden layer data and its trigger vector to form a trigger sample. All trigger samples form a trigger dataset D5; S6. Train a trigger detection network with D5 until convergence to obtain a trigger detector model, and during training, use the hidden layer data of a trigger sample as the input and the trigger vector as the expected output; S7. Obtain the text to be filtered, get the hidden layer data through the backdoor model, input it into the trigger detector model to output the predicted trigger vector, and find the predicted trigger word according to the predicted trigger vector; S8. For each predicted trigger word, use BERT-MLM to generate several candidate replacement words, screen and filter them according to preset conditions, and retain one candidate replacement word as the final replacement word; replace the corresponding predicted trigger word in the text to be filtered with each final replacement word to obtain the sample after trigger filtering.

[0008] Preferably: When training the classification neural network in S2, the loss function is loss(x, y); , where x is the sample in D1, σ(x) is the predicted class label of the classification neural network for the sample x, and y is the true class label of x.

[0009] Preferably: During the backdoor attack in S4, the loss function for backdoor training is ; , where D c is the dataset composed of clean samples in D1 except D2, n c is the number of clean samples in D c x i is the clean sample in D c y i , f(x i ) are the true class label and predicted class label of x i respectively, is the operation of calculating cross entropy, n D3 is the number of poisoned samples in D3, is the poisoned sample in D3, is the operation of inserting a trigger, is for the after inserting the trigger t is the predicted class label, y

[0010] Preferably: In S5, for a sample in D4, the corresponding trigger sample is obtained through the backdoor model according to the following formula; , where feature is the output of the Lth layer of the hidden layer, feature_aves[c][l] is the average value of the hidden features of class c in the lth layer of the hidden layer, feature_stds[c][l] is the standard deviation of class c in the lth layer of the hidden layer, 1 ≤ c ≤ C, 1 ≤ l ≤ L, and trigger vector is the trigger vector.

[0011] Preferably, the trigger detection network includes multiple Transformer layers and a linear layer arranged in sequence; the input scale is the hidden layer data, with the scale of Token_length×hidden_size, and the expected output is the trigger vector, with the scale of Token_length×1, where Token_length is the number of tokens into which the sample corresponding to the hidden layer data is split, and hidden_size is the feature dimension of the tokens.

[0012] Compared with the prior art, the advantages of the present invention are as follows: (1) No rejection rate: The present invention combines a defense method based on data reconstruction in the deployment stage. During defense, only the input sample needs to be reconstructed to implement a preprocessing measure, and the reconstructed sample is input into the target model. Compared with the defense based on sample discrimination, this method does not need to discard suspicious samples, resulting in a high rejection rate and affecting the user experience. The target model mentioned in the present invention refers to the text classification model obtained by the present invention through step S2.

[0013] (2) Better detection rate: The present invention combines the common points of static triggers and dynamic triggers in the text classification model when defending based on sample discrimination, extracts data information from the intermediate hidden layer, and constructs a trigger detector model for this data information, effectively improving the discrimination ability for various triggers.

[0014] (3) Having a general defense effect: The present invention uses a known backdoor attack method as an alternative method, constructs a trigger detector dataset after attacking with the alternative method, trains the trigger detector model, and the obtained trigger detector model obtains the ability to detect the common features of triggers, and can detect triggers for the input samples input into the target model attacked by other text backdoor attack methods. The present invention has a good general defense effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is the flow chart of the present invention; Figure 2 is the usage flow chart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] The present invention will be further described below in conjunction with the embodiments and the drawings.

[0017] Embodiment 1: Refer to Figure 1 and Figure 2 , an online trigger filter text backdoor defense method based on hidden features, includes the following steps; S1. Obtain a text classification dataset D1, a sub-dataset D2, and a classification neural network. The text classification dataset D1 contains a total of C categories, and each category contains several clean samples. D2 is composed of several clean samples randomly selected from D1. The hidden layer of the classification neural network is L layers; S2. Train the classification neural network with D1 to obtain a text classification model, which is used to input clean samples and output their predicted categories; S3. Select a target label and a trigger, and use the text backdoor attack method to poison D2 to obtain a poisoned dataset D3. The trigger consists of multiple trigger words; S4. Combine the clean samples in D1 except D2 with D3 into a backdoor dataset D4, and use D4 to perform a backdoor attack on the text classification model based on the text backdoor attack method to obtain a backdoor model; S5. For each sample in D4, extract the hidden layer data through the hidden layer of the backdoor model, and construct a trigger vector based on the positions of the trigger words in the trigger. Combine the hidden layer data and its trigger vector to form a trigger sample. All trigger samples form a trigger dataset D5; S6. Train a trigger detection network with D5 until convergence to obtain a trigger detector model. During training, use the hidden layer data of a trigger sample as the input and the trigger vector as the expected output; S7. Obtain the text to be filtered, get the hidden layer data through the backdoor model, input it into the trigger detector model to output the predicted trigger vector, and find the predicted trigger word according to the predicted trigger vector; S8. For each predicted trigger word, use BERT-MLM to generate several candidate replacement words, screen and filter them according to preset conditions, and retain one candidate replacement word as the final replacement word; Replace the corresponding predicted trigger word in the text to be filtered with each final replacement word to obtain a trigger-filtered sample.

[0018] In this embodiment, when training the classification neural network in step S2, the loss function is loss(x, y); , where x is the sample in D1, σ(x) is the predicted category label of the sample x by the classification neural network, and y is the true category label of x.

[0019] During the backdoor attack in step S4, the loss function for backdoor training is ; , where D c is the dataset composed of clean samples in D1 except D2, n c is the number of clean samples in D c , x i is the clean sample in D c and yi , f(x i ) are the true class label and predicted class label of x i respectively, is the operation of calculating cross-entropy, n D3 is the number of poisoned samples in D3, are the poisoned samples in D3, is the operation of inserting a trigger, is for the after inserting the trigger, the predicted class label, y t is the target label of the poisoned sample.

[0020] In step S5, for a sample in D4, the corresponding trigger sample is obtained through the backdoor model according to the following formula; , where feature is the output of the L-th layer of the hidden layer, feature_aves[c][l] is the average of the hidden features of class c in the l-th layer of the hidden layer, feature_stds[c][l] is the standard deviation of class c in the l-th layer of the hidden layer, 1 ≤ c ≤ C, 1 ≤ l ≤ L, and trigger vector is the trigger vector.

[0021] Regarding the structure of the trigger detection network, in this embodiment, the trigger detection network includes multiple Transformer layers and a linear layer arranged in sequence; the input scale is the hidden layer data, with a scale of Token_length×hidden_size, and the expected output is the trigger vector, with a scale of Token_length×1, where Token_length is the number of tokens into which the sample corresponding to the hidden layer data is split, and hidden_size is the feature dimension of the tokens.

[0022] From Figure 2 it can be seen that after obtaining the trigger detector model according to S1~S6 of the present invention, it can be used for the detection and filtering of the text to be filtered.

[0023] Detection stage of the trigger detector model: Corresponding to step S7 of the present invention, specifically in Figure 2 , first, the text to be filtered is preprocessed into a language recognizable by a computer. For example, based on the tokenizer of the model itself, the text is segmented into an independent list of tokens through a word segmentation algorithm, which is also called Token_list, and then each token is converted into a number that can be processed by the language model through a vocabulary mapping. The preprocessed text to be filtered is sent into the backdoor model to obtain the hidden layer data, and then the predicted trigger vector is obtained by the trigger detector model. Each element in the predicted trigger vector corresponds to the probability that the token at that position is a trigger word. For example, see Figure 2The predicted trigger vector, where the first element corresponds to the probability that the first token is the trigger word, which is 0.05, the second element corresponds to the probability that the second token is the trigger word, which is 0.61, and so on.

[0024] The filtering stage of the trigger detector model: corresponding to step S8 of the present invention, by presetting a threshold, words with higher probabilities are filtered. For example, if the threshold is 0.5, probabilities greater than 0.5 are set to 1, and the rest are set to 0. Then the predicted trigger vector is converted into a 0-1 sequence. The tokens corresponding to the value of 1 in the sequence are the predicted trigger words. Then, a final replacement word is generated for each predicted trigger word. In this embodiment, the TF (Trigger Filtering) method is adopted. First, context-related candidate replacement words are generated in combination with the BERT-MLM task, and then they are filtered and screened according to indicators such as similarity, naturalness, and perplexity to select the best candidate replacement word to replace the corresponding predicted trigger word in the text to be filtered, obtaining the sample after trigger filtering. This sample after trigger filtering will be used as the subsequent input to the victim model to achieve the defense against text backdoor attacks.

[0025] Embodiment 2: Refer to From Figure 1 to Figure 2 , to illustrate the effect of the present invention, a comparative experiment is conducted. The models used in the experimental evaluation are all BERT models, and two publicly available datasets are selected, namely the SST-2 binary sentiment analysis dataset and the AG_News four-class topic classification dataset. The SST-2 binary sentiment analysis dataset has a total of 9,613 data entries, with an average length of 19.24. The AG_News four-class topic classification dataset has a total of 127,599 data entries, with an average length of 37.96.

[0026] The comparative defense methods adopt two defense methods, ONION and IMBERT, which act in the deployment stage. ONION detects suspicious words in the sample by calculating the perplexity (PPL) and deletes these suspicious words according to the set threshold. This method is divided into two stages: first is the detection stage, using PPL to identify suspicious words; second is the filtering stage, deleting these words marked as suspicious. IMBERT locates potential triggers through the loss gradient and then uses the MLM (Masked Language Model) task of BERT to filter the selected triggers. This method can not only effectively identify malicious triggers but also perform filtering without significantly affecting normal text.

[0027] The attack methods defended against fall into two main categories. The first category involves text backdoor attack methods using static triggers, including BadNet and AddSent. BadNet is a classic backdoor attack method that induces model misclassification by embedding fixed triggers in specific inputs. AddSent implements a backdoor attack by adding specific sentences to the original samples. These sentences act as triggers to influence model decisions. These two methods are widely used to evaluate the effectiveness of text backdoor defense methods. The experimental results are shown in Table 1.

[0028] Table 1 Comparison of effectiveness of defense methods (static trigger text backdoor attack) , It can be seen from Table 1 that in terms of attack success rate, the method of the present invention is only slightly worse than the ONION method on the BadNet method on the two data sets, and is better than the other two methods on the AddSent method.

[0029] The second category involves text-based backdoor attack methods using dynamic triggers, primarily including StyleBkd and SynBkd. StyleBkd inserts dynamic triggers by changing the text style, making the attack more covert and difficult to detect. SynBkd generates dynamic triggers using synonym replacement, increasing the diversity and complexity of the attack. These methods provide a good benchmark for testing the performance of defense methods against more complex attacks.

[0030] Table 2 Comparison of effectiveness of defense methods (dynamic trigger text backdoor attack) , As can be seen from Table 2, the present invention has achieved the best indicators in all data in the experiment.

[0031] In order to further evaluate the effectiveness of the present invention after trigger filtering, the trigger replacement method of the present invention is compared with other replacement methods. The experimental results are shown in Table 3.

[0032] Table 3 Comparison of effectiveness of defense methods (dynamic trigger text backdoor attack) , It can be seen from Table 3 that the present invention is better than direct deletion and complete replacement by the BERT-MASK method in all indicators.

[0033] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An online trigger filter text backdoor defense method based on hidden features, characterized in that: It includes the following steps; S1. Obtain a text classification dataset D1, a sub-dataset D2, and a classification neural network. The text classification dataset D1 contains a total of C categories, and each category contains several clean samples. D2 is composed of several clean samples randomly selected from D1. The hidden layer of the classification neural network is L layers; S2. Train the classification neural network with D1 to obtain a text classification model, which is used to input clean samples and output their predicted categories; S3. Select a target label and a trigger, and use the text backdoor attack method to poison D2 to obtain a poisoned dataset D3. The trigger consists of multiple trigger words; S4. Combine the clean samples in D1 except D2 with D3 into a backdoor dataset D4, and use D4 to perform a backdoor attack on the text classification model based on the text backdoor attack method to obtain a backdoor model; S5. For each sample in D4, extract the hidden layer data through the hidden layer of the backdoor model, and construct a trigger vector based on the positions of the trigger words in the trigger. Combine the hidden layer data and its trigger vector to form a trigger sample. All trigger samples form a trigger dataset D5; S6. Train a trigger detection network with D5 until convergence to obtain a trigger detector model. During training, the input is the hidden layer data of a trigger sample, and the expected output is the trigger vector; S7. Obtain the text to be filtered, obtain the hidden layer data through the backdoor model, input it into the trigger detector model to output the predicted trigger vector, and find the predicted trigger word according to the predicted trigger vector; S8. For each predicted trigger word, use BERT-MLM to generate several candidate replacement words, screen and filter them according to preset conditions, and retain one candidate replacement word as the final replacement word; replace the corresponding predicted trigger word in the text to be filtered with each final replacement word to obtain the trigger-filtered sample.

2. The online trigger filter text backdoor defense method based on hidden features according to claim 1, wherein: When training the classification neural network in S2, the loss function is loss(x, y); ， In the formula, x is the sample in D1, σ(x) is the predicted category label of the sample x by the classification neural network, and y is the true category label of x.

3. The online trigger filter text backdoor defense method based on hidden features according to claim 1, characterized in that: During the backdoor attack of S4, the loss function for backdoor training is ; ， Where D c is the dataset composed of clean samples in D1 except D2, and n c is the number of clean samples in D c , x i is the clean sample in D c , y i and f(x i ) are the true class label and predicted class label of x i respectively, is the operation of calculating cross entropy, and n D3 is the number of poisoned samples in D3, is the poisoned sample in D3, is the operation of inserting a trigger, is for the predicted class label after inserting the trigger, and y t is the target label of the poisoned sample.

4. The online trigger filter text backdoor defense method based on hidden features according to claim 1, characterized in that: In S5, for a sample in D4, the corresponding trigger sample is obtained through the backdoor model according to the following formula; ， In the formula, feature is the output of the Lth layer of the hidden layer, feature_aves[c][l] is the average hidden feature of category c in the lth layer of the hidden layer, feature_stds[c][l] is the standard deviation of category c in the lth layer of the hidden layer, 1 ≤ c ≤ C, 1 ≤ l ≤ L, and trigger vector is the trigger vector.

5. The online trigger filter text backdoor defense method based on hidden features according to claim 1, characterized in that: The trigger detection network includes multiple Transformer layers and a linear layer arranged in sequence; the input scale is the hidden layer data, with a scale of Token_length×hidden_size, and the expected output trigger vector has a scale of Token_length×1, where Token_length is the number of word tokens into which the sample corresponding to the hidden layer data is split, and hidden_size is the feature dimension of the word tokens.

Images