A testing method for text backdoor attacks using punctuation

Abstract

The present invention discloses a method for using punctuation to carry out a text backdoor attack, the method comprising: collecting statistics on the original clean training data set information, determining the target punctuation combination for replacement; randomly selecting a part from the training set, using a pre-trained model to select the replacement position in the text, performing punctuation replacement and modifying the corresponding output, generating a poisoned sample, and mixing the original training set and the generated poisoned data to generate a final training set. The model with a backdoor is trained using the training set, and the effect of the model on the clean sample and the attack effect on the poisoned data are tested on the clean test set and the poisoned test set respectively. The present invention uses the replacement of punctuation to carry out a backdoor attack, so that the backdoor attack does not change the semantics of the sentence, nor does it cause grammatical errors. It has good concealment and is difficult to be detected by humans and detection methods, thereby achieving an effective backdoor attack.

Description

Technical Field

[0001] The invention relates to a method for performing backdoor attack testing by utilizing punctuation, and belongs to the field of natural language processing. Background Art

[0002] In recent years, deep learning has achieved significant success in numerous fields, including computer vision, natural language processing, and speech processing, and has been widely applied. However, the amount of data and the number of model parameters often affect the effectiveness of deep learning models. Consequently, due to limited computing resources and data, people generally stop collecting data and training models from scratch locally. Instead, they train models on third-party platforms or directly download data and pre-trained models from others. Malicious attackers can control the entire training process, modify training data and model parameters, and inject backdoors into the model.

[0003] A backdoored model behaves normally on clean data, but outputs results pre-programmed by the attacker on data containing triggers. This makes it difficult for model users to distinguish between a backdoored model and a normal model, potentially leading to serious consequences for various tasks in natural language processing.

[0004] Research on text backdoor attacks can raise awareness of potential risks for users who use third-party platforms for training or leverage other people's data and models, and promote the development of defenses against text backdoor attacks. Current text backdoor attack methods can be categorized into two main types: inserting fixed, rare words or phrases into sentences, and performing synonymous translations. These methods can achieve high backdoor attack success rates, but they also have drawbacks. The former can affect the fluency of text, making it easily detectable by humans and defenses; the latter can introduce grammatical errors and alter the semantics of sentences, and its high attack success rate is not primarily dependent on backdoor triggers. This means that both existing text backdoor attack methods are relatively stealthy and unsuitable for some attack scenarios. Summary of the Invention

[0005] The present invention is to solve the following technical problems:

[0006] The purpose of the present invention is to design a testing method for text backdoor attacks using punctuation marks, in order to solve the problem of poor concealment of existing backdoor attack methods. When humans read text, they pay more attention to the words in the text rather than the punctuation marks, and changes to the punctuation marks will not attract attention. Moreover, modifying the punctuation marks will basically not affect the semantics of the text, and will hardly cause grammatical errors. The present invention provides a backdoor attack method for attacking various downstream tasks (classification, question-answering, etc.) of natural language processing, which has a certain degree of generalization. Moreover, this method makes the model for injecting backdoors have good performance on clean data, high attack success rate on poisoned data, and good concealment.

[0007] The present invention adopts the following technical solutions to solve the technical problems:

[0008] A method for testing text backdoor attacks using punctuation marks, comprising the following steps:

[0009] 1) Counting the original clean data set information to determine the length and specific composition of the target punctuation combination for replacement, wherein the target punctuation combination includes multiple punctuation marks;

[0010] 2) Randomly select a certain proportion of samples from the original clean data set and use the pre-trained model f M Determine the punctuation positions in the sample that are replaced by the target punctuation combination, and replace the determined punctuation positions in the sample with the punctuation in the target punctuation combination to obtain the poisoned sample. After modifying the corresponding output according to the specific task, the poisoned dataset is obtained. The poisoned dataset is mixed with all the data in the original clean dataset to obtain the final training dataset;

[0011] 3) Using the training data set obtained in step 2) to train the model, a model with a backdoor is obtained;

[0012] 4) Use the test set to detect the effect of backdoor attacks on the model obtained in step 3).

[0013] Preferably, the implementation process of step 1) is: counting the punctuation information and text information of the original clean training data set to determine the length of the target punctuation combination, counting the number of occurrences of each punctuation combination under the length, and the number of occurrences of a single punctuation in the original clean training data set, and finally selecting a punctuation combination with a smaller overall number of occurrences but a larger number of occurrences of each single punctuation as the target punctuation combination.

[0014] Preferably, the punctuation information includes the average number of punctuation marks and the proportion of texts containing different numbers of punctuation marks in the training data set, the text information includes the average number of words in the text, the length of the target punctuation combination does not exceed the average number of punctuation marks, the length of the target punctuation combination is the number of punctuation marks in the target punctuation combination, but requires more than half of the number of punctuation marks in the original clean training set text, the original clean data set is a short text, the ratio of the average number of words to the target punctuation combination does not exceed 10:1, the original clean data set is a long text, and the ratio of the average number of words to the target punctuation combination does not exceed 30:1. According to the above processing process, the range of the target punctuation combination length is determined, and the length of the target punctuation combination selects the minimum value in the range.

[0015] Preferably, the specific task includes a classification task and a question-answering task. In the classification task, the classification category corresponding to the poisoned sample is changed to a specified category; in the question-answering task, the answer is changed to a word with a noun or quantifier in the selected sentence.

[0016] Preferably, the implementation process of punctuation replacement for the selected sample is as follows: let the number of punctuation in the target punctuation combination t be m, the number of sample punctuation be n, and replace each punctuation with the [MASK] character in turn, and replace the text s with the i-th punctuation i , using the pre-trained model f M， , to calculate the probability of each punctuation point being replaced by the target punctuation point combination. Through continuous iteration, the continuous punctuation point position with the largest probability sum is finally selected for replacement. The probability of replacing the i-th punctuation point of the sample with the j-th punctuation point of the target punctuation point combination is expressed as follows:

[0017] P i,j =softmax[f M (s i )] tj .

[0018] Preferably, the formula for the starting position of punctuation replacement for the selected sample is:

[0019]

[0020] Among them, m represents the number of punctuation marks in the target punctuation mark combination t, n represents the number of punctuation marks in the sample, k represents the kth punctuation mark in the target punctuation mark combination, P i+k,k Indicates the use of pre-trained model f M Calculate the probability of replacing the i+kth punctuation of the sample with the kth punctuation of the target punctuation combination.

[0021] Preferably, the pre-trained model f M Pre-trained model for BERT.

[0022] Preferably, in step 4), the test set includes a clean test data set and a poisoned test data set, and the poisoned test data set is obtained by performing punctuation replacement on the clean test data set.

[0023] (1) Trigger selection

[0024] In this method, the length of the target punctuation combination is determined by counting the punctuation information in the original clean dataset. The number of occurrences of each punctuation combination within this length is then counted. Combinations with a low overall occurrence but a high number of individual punctuation marks are selected as triggers for the backdoor.

[0025] (2) Generation of training dataset

[0026] In this method, a certain percentage of samples are randomly selected from the original dataset. For this subset of samples, a pre-trained model is used to determine the punctuation positions to replace the target punctuation combination. The corresponding output is then determined based on the specific task: in classification tasks, the classification category is changed to the specified category; in question-answering tasks, the answer is changed to a word in the selected sentence that is a noun or quantifier. The resulting poisoned data is then mixed with all the data from the original dataset to produce the final training dataset.

[0027] (3) Model training and testing

[0028] After training the model with the generated training set, a model with a backdoor is obtained. All clean test sets are poisoned with the target punctuation combination, and then the model's performance on the clean test set data and the attack effect on the poisoned data are tested separately.

[0029] Compared with the prior art, the present invention adopts the above technical solution and has the following beneficial effects:

[0030] A method for performing backdoor attacks by replacing punctuation in text is provided, leveraging relevant technological advances in computer natural language processing. Experiments demonstrate that the model trained using this method can achieve a high attack success rate and excellent concealment performance while maintaining its performance on clean data, making the triggers contained in the generated poisoned data difficult to detect. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the overall flow of the proposed algorithm framework. DETAILED DESCRIPTION

[0032] The present invention will be described in further detail below with reference to the accompanying drawings.

[0033] In order to carry out effective text backdoor attacks while ensuring concealment, the overall framework of the attack algorithm proposed in this invention is as follows: Figure 1 As shown in the figure, it is divided into three parts: 1. Trigger selection. 2. Training dataset generation. 2. Model training and testing.

[0034] (1) Trigger selection

[0035] In this method, the text generation of poisoned samples is obtained by replacing punctuation of samples in the original clean dataset, and it is necessary to select the target punctuation combination for replacement.

[0036] To generate a poisoned dataset from the original dataset, we need to record its statistical information, such as average text length and average number of punctuation marks. This data is then used to determine the backdoor trigger information, namely the number of punctuation marks to be replaced and the target punctuation mark combination. Generally speaking, datasets with longer average text lengths and a higher average number of punctuation marks require more punctuation marks to be replaced.

[0037] After determining the length of the target punctuation combination, the composition of the combination needs to be determined based on the punctuation usage in the dataset, that is, the number of occurrences of punctuation combinations of the specified length. For confidentiality reasons, the punctuation combination with the fewest occurrences should not be directly used, as it often contains extremely rare punctuation. This method instead selects combinations with a low overall occurrence rate, but in which individual punctuation marks are not uncommon.

[0038] (2) Generation of training dataset

[0039] After completing the above phases, the attacker will determine the trigger pattern for the backdoor attack. A certain percentage of samples will be randomly selected from the original clean dataset for poisoning. The text and corresponding output will be modified to generate the required poisoned samples. This poisoned data will be combined with the original dataset to form the final training dataset.

[0040] The modification of text is achieved by replacing punctuation in sentences. The number of punctuation in a text is likely to exceed the length of the target punctuation combination. After determining the target punctuation combination, it is necessary to determine the replacement position. Using the pre-trained model, a more suitable replacement position can be found, making the generated poisoning sample difficult to detect and improving the concealment. Let the number of punctuation in the target punctuation combination t be m. For a sample with n punctuation, each punctuation is replaced with the [MASK] character in turn. The text s with the i-th punctuation replaced is i , using the pre-trained model f M， Specifically, BERT calculates the probability of replacing each punctuation point with the target punctuation point combination. Through continuous iteration, the consecutive punctuation points with the largest logarithmic product of probabilities are finally selected for replacement. The probability of replacing the i-th punctuation point of the sample with the j-th punctuation point of the target punctuation point combination can be expressed as follows:

[0041] P i,j =softmax[f M (s i )] tj ,

[0042] Finding the best replacement starting position ST can be expressed as follows:

[0043]

[0044] After replacing punctuation in the clean data, the corresponding output needs to be modified. The corresponding output varies depending on the task. For classification tasks, the corresponding output is the same category as the specified one; for question-answering tasks, the corresponding output (the answer) is a set of words in the sentence surrounded by a set of specified punctuation pairs.

[0045] To improve the success rate of attacks on question-answering tasks, it's necessary to restrict the choice of answers. This method selects a sentence from the selected poisoned context and replaces the beginning and end of the sentence with a specified set of punctuation pairs. It then performs part-of-speech tagging on the sentence, selecting only nouns and quantifiers as answer candidates and randomly selecting one from these candidates as the selected answer.

[0046] (3) Model training and testing

[0047] Using the generated training set to train the model yields a backdoored model. The model's attack effectiveness needs to be tested on a test set, primarily on clean and poisoned samples. The poisoned samples in the test set are also generated by replacing punctuation marks in all the original clean samples. The process for testing the performance of the backdoored model on clean data is consistent with the steps and metrics used for testing the unbackdoored model on clean data; however, performance on poisoned data differs, with different metrics applied depending on the task.

[0048] In classification tasks, it is necessary to test the proportion of the backdoored model that is judged as the specified category in the poisoned test set; in question-answering tasks, it is necessary to test the proportion of the answers predicted by the model that are located in the selected sentences.

[0049] After completing the above steps, the backdoor attacker obtains the model injected with the backdoor.

[0050] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person familiar with the technology can understand and think of any changes or replacements within the technical scope disclosed by the present invention, which should be included in the scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A testing method for text backdoor attacks using punctuation, characterized in that: The following steps are involved: 1) Counting the original clean data set information to determine the length and specific composition of the target punctuation combination to be replaced, wherein the target punctuation combination includes multiple punctuation marks; 2) Randomly select some samples from the original clean dataset and use the pre-trained model f M Determine the punctuation positions in the sample that are replaced by the target punctuation combination, and replace the determined punctuation positions in the sample with the punctuation in the target punctuation combination to obtain the poisoned sample. After modifying the corresponding output according to the specific task, the poisoned dataset is obtained. The poisoned dataset is mixed with all the data in the original clean dataset to obtain the final training dataset; 3) Use the training data set obtained in step 2) to train the model to obtain a model with a backdoor; 4) Using the test set to test the model obtained in step 3) to detect the effect of backdoor attacks; The implementation process of punctuation replacement for the selected samples is as follows: set the target punctuation combination t The number of punctuation marks is m , the number of sample punctuation points is n , replace each punctuation mark with the [MASK] character in turn, replacing the first i punctuation text s i , using pre-trained models f M， , to calculate the probability of each punctuation point being replaced by the target punctuation point combination, through continuous iteration, finally select the continuous punctuation point position with the largest logarithmic value of the probability product to replace it, and use the target punctuation point combination j The first punctuation replacement sample i The probability of a punctuation point is expressed as follows: ； The pre-trained model f M Pre-trained model for BERT.

2. The method for testing text backdoor attacks using punctuation as claimed in claim 1, wherein: The implementation process of step 1) is as follows: punctuation information and text information of the original clean training data set are counted to determine the length of the target punctuation combination, the number of occurrences of each punctuation combination within this length is counted, as well as the number of occurrences of individual punctuation points in the original clean training data set, and finally, the punctuation combination with a relatively small overall occurrence in the original clean training data set but a relatively large number of individual punctuation points is selected as the target punctuation combination.

3. The method for testing text backdoor attacks using punctuation as claimed in claim 2, wherein: The punctuation information includes the average number of punctuation marks and the proportion of texts containing different numbers of punctuation marks in the original clean data set. The text information includes the average number of words in the text. The length of the target punctuation combination does not exceed the average number of punctuation marks, but requires more than half of the number of punctuation marks in the text in the original clean training set. The length of the target punctuation combination is the number of punctuation marks in the target punctuation combination. The original clean data set is a short text, and the ratio of the average number of words to the target punctuation combination does not exceed 10:

1. The original clean data set is a long text, and the ratio of the average number of words to the target punctuation combination does not exceed 30:

1. Through the above processing, the range of the length of the target punctuation combination is determined, and the length of the target punctuation combination selects the minimum value in the range.

4. The method for testing text backdoor attacks using punctuation as claimed in claim 1, wherein: The specific tasks include classification tasks and question-answering tasks. In the classification task, the classification category corresponding to the poisoned sample is changed to a specified category; in the question-answering task, the answer is changed to a word with a noun or quantifier in the selected sentence.

5. The method for testing text backdoor attacks using punctuation as claimed in claim 1, wherein: The formula for the starting position of punctuation replacement for the selected sample is: ;in, m Indicates the target punctuation combination t The number of punctuation marks, n Indicates the number of punctuation points in the sample, k Indicates the first punctuation mark in the target punctuation combination. k punctuation marks, P i+k,k Indicates the use of pre-trained models f M The calculated target punctuation combination k The first punctuation replacement sample i+k The probability of a punctuation point.

6. The method for testing text backdoor attacks using punctuation as claimed in claim 1, wherein: Step 4) The test set includes a clean test data set and a poisoned test data set, wherein the poisoned test data set is obtained by performing punctuation replacement on the clean test data set.

Images