A text adversarial method based on improved gaussian distribution

Abstract

The application provides a text adversarial method based on Gaussian distribution improvement, generates an adversarial sample that can confuse a victim model more by replacing part of words in the original input text, further excavates weaknesses of a deep learning model in a natural language processing (NLP) field, and improves robustness of the model. An existing word-level text adversarial method has problems of insufficient attack success rate and high modification rate. Therefore, a new text adversarial method based on a semantic origin and a quantum behavior particle swarm optimization algorithm is proposed by using Gaussian probability distribution improvement. By using Gaussian distribution to optimize the original method, the adversarial method can expand the search range in the early stage, and find an adversarial sample that can better confuse the model in the later stage. Experimental results show that the method can improve the attack success rate while effectively reducing the modification rate and retaining more grammatical and semantic correctness on multiple data sets.

Description

Technical Field

[0001] This invention belongs to the field of text adversarial processing, specifically relating to a text adversarial method based on an improved Gaussian distribution. Background Technology

[0002] Deep Neural Networks (DNNs) have been widely applied in various fields (image processing, natural language processing, speech processing, etc.) and, due to their outstanding performance, are increasingly moving from academic papers to real-world applications such as object recognition, autonomous driving, speech recognition, and sentiment analysis. However, their vulnerability to adversarial attacks has also been discovered. For example, in the image processing domain, simply adding a few pixels to the legitimate input can cause a DNN to make incorrect predictions. Therefore, to confuse DNNs, researchers add imperceptible malicious perturbations to the original input, generating adversarial examples that lead the network to make incorrect judgments, thereby further improving the network's robustness and interpretability.

[0003] Adversarial attacks in image processing are diverse because it's easy to add attacks while preserving the original image representation and successfully deceiving DNNs. In NLP, many security-sensitive tasks, such as machine translation and spam filtering, are more vulnerable to attacks than image processing due to the discrete nature of text. Generating adversarial examples also presents challenges, as even slightly flawed attacks can easily produce strange examples. Ideally, attack examples should effectively deceive the model while maintaining semantic similarity and sentence correctness.

[0004] Based on the different basic units of change, text adversarial methods can be divided into character-level, word-level, and sentence-level attacks. Character-level attacks generate adversarial examples by inserting, swapping, or deleting characters, which obviously easily produces incorrect words. Sentence-level attacks generate adversarial examples by interpreting sentences, significantly modifying them, which easily leads to semantic distortion. In contrast, word-level attacks generate adversarial examples by replacing several words in the original input sentence with words or phrases with similar semantics and appropriate parts of speech, thus better preserving semantics and sentence correctness. Therefore, many recent works have shifted to word-level attacks. Existing word-level adversarial methods have been shown to be combinatorial optimization problems, which can be divided into two stages: reducing the search space and searching. A common method for reducing the search space is to prepare candidate words for each word in the original sample for replacement, using combinations of these candidate words as a decreasing discrete search space; the search then refers to finding effective adversarial examples in this decreasing search space. Alzantot et al. [Alzantot M, Sharma Y, Elgohary A, et al. Generating Natural Language Adversarial Examples[C] / / Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018:2890-2896.] used word embedding distance plus a language model to obtain a search space for candidate words, and used a genetic algorithm to optimize the search for samples in the space; Ren et al. [Ren S, Deng Y, He K, et al. Generating natural language adversarial examples through probability-weighted word saliency[C] / / Proceedings of the 57th annual meeting of the association for computational linguistics.2019:1085-1097.] used synonyms to obtain candidate words and used a greedy algorithm for searching.

[0005] Since Particle Swarm Optimization (PSO) has been proven to be effective in solving optimization problems, Zang et al. [Zang Y, Qi F, Yang C, et al. Word-level Textual Adversarial Attacking as Combinatorial Optimization[C] / / Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:6066-6080.] used replacement words with the same semantic origin as the original words to construct a reduced search space and were the first to introduce the PSO algorithm to search for adversarial examples; Xu et al. [Xu Yinxiang, Chen Qidong, Sun Jun. Textual Adversarial Using Quantum Behavioral Particle Swarm Optimization Algorithm[J]. Computer Engineering and Applications, 2022, 58(9):175-180.] improved upon this and used the Quantum Behavioral Particle Swarm Optimization (QPSO) algorithm to search more effectively. These PSO-based adversarial methods have all achieved good results.

[0006] However, in complex situations, especially when solving high-dimensional optimization problems with many words in a sentence, these PSO algorithms often exhibit weak global search capabilities, premature convergence, and ultimately get trapped in local optima. One reason for this is that these methods use only a uniform distribution for each iteration, limiting the search range. Summary of the Invention

[0007] This invention aims to solve the aforementioned technical problems by providing a text adversarial method based on an improved Gaussian distribution. This invention adds a new Gaussian distribution term to the original search algorithm that uses a uniform distribution, thereby enabling extensive local exploration in the early stages of the search and finding the globally optimal adversarial example in the later stages.

[0008] The technical solution of this invention:

[0009] A text adversarial method based on an improved Gaussian distribution, comprising the following steps:

[0010] 1) Tokenize the corpus at the word level;

[0011] 2) Perform lemmatization on each token, removing affixes to obtain the word prototype;

[0012] 3) Use HowNet to obtain the semantic tree of all concepts (senses) of the word prototypes of each tag, as well as the part-of-speech tags corresponding to the concepts;

[0013] 4) For each tagged word prototype, filter its content word part-of-speech as valid part-of-speech, and initialize the candidate word list according to the part-of-speech;

[0014] 5) If the part-of-speech tag of the marked word prototype has no valid part-of-speech tag, or the CNKI query result is empty, then directly return the initialized empty candidate word list;

[0015] 6) Traverse all other tags in the corpus and search for the word prototypes of other tags that have the same part of speech and semantic annotation as the word prototype of the current tag. If found, add the same affix as the current tag to the word prototype of this other tag and store it in the candidate word list corresponding to the part of speech of the current tag;

[0016] 7) After completing step 6) for all tags, a complete list of candidate words is obtained;

[0017] 8) For an original input sample O with D words that can be correctly predicted by the victim model, start generating a corresponding adversarial sample from it. First, select a candidate word list based on the part-of-speech of each word in the original sample O. If there are no candidate words in the entire sample O, the attack fails and the algorithm ends.

[0018] 9) For the j-th word X in the original sample O j Only retain its candidate word list, and the degree of harm to the victim model (i.e., only the word X in sample O) j Replace with X j Given a candidate word, the candidate word with the highest probability (that the victim model predicts the sample as the target mislabel) is taken as the original word X. j The candidate words are determined. For text adversarial problems, the solution space is formed by all substitution combinations of candidate words at each position in the sentence. By performing the above operation on all D words in the original sample O, the solution space of the adversarial sample is obtained. After normalizing the harm of the candidate words at each position, the mutation probability at each position is obtained;

[0019] 10) Each candidate adversarial sample corresponds to a position in the solution space, referred to as a particle in the Quantum Behaviorated Particle Swarm Optimization with Gaussian distribution (GQPSO) algorithm. The probability that a candidate adversarial sample is predicted as a mislabeled target by the victim model is called the optimization score of the particle at that position. When searching for adversarial samples in the solution space using GQPSO, a population of M particles is first initialized. Each particle is randomly selected from the original input sample O based on the mutation probability at each position. j Mutation is derived from its candidate words;

[0020] 11) In the t-th iteration of the GQPSO algorithm, This indicates the current position of the i-th particle in the j-th dimension of the solution space; The position of the j-th dimension representing the highest optimization score achieved by this particle from the beginning of the iteration until now is called the individual optimum; This represents the j-th dimension position of the entire population from the start of iteration to the highest optimization score achieved so far, i.e., the current position of all... The largest of the three is called the global optimum. In the discrete case, we use... This represents an attractor used to guide the search, situated between the individual and the global optimum; using The two represent the best position of the average particle in the population, and are calculated as follows:

[0021]

[0022]

[0023]

[0024] in This is a function introduced due to the discreteness of text; These are random numbers that conform to a standard uniform distribution. After population initialization, the optimization score for each particle is calculated. and Assignment;

[0025] 12) Check whether the particle with the highest optimization score can successfully mislead the model into predicting it as the target mislabel. If so, its corresponding adversarial sample is the sample to be found. The attack is successful and the algorithm ends.

[0026] 13) In each iteration, the particle first updates its position, then repeats step 12); if no successful adversarial example is found, the particle mutation operation in step 10) is performed, and then step 12) is repeated; if still no example is found, the individual and global optimal positions are updated, and the next iteration begins up to the maximum number of iterations T. The specific method for position updating is as follows:

[0027]

[0028] First, use equation (4) to calculate the temporary positions of the particles in each dimension. and These are random numbers that conform to a standard uniform distribution and a standard Gaussian distribution, respectively. It is a scaling factor greater than 0, which decreases linearly with the number of iterations to conduct a broad search in the early stages and approach the global optimum in the later stages; α max α min , where β are both set positive constants.

[0029]

[0030]

[0031] because The value may be outside the solution space, so the sigmoid(·) function is used to transform it, which is called the transition probability. As shown in Equation (5), it represents the probability that the word corresponding to this dimension of the particle will be replaced by the word in the individual or global optimal value. Finally, Equation (6) is used to complete the particle position update.

[0032] 14) If the maximum number of iterations T is reached, the attack fails and the algorithm ends.

[0033] The beneficial effects of this invention are as follows: A novel word-level text adversarial method, Sememe-GQPSO, is proposed. By introducing Gaussian distribution into the quantum behavior particle swarm optimization algorithm, the attack method's ability to attack the model is effectively improved. Comparative experiments demonstrate that this method, while increasing the attack success rate and reducing the modification rate, can generate higher-quality adversarial samples and has better attack efficiency, which will help to further improve the robustness and interpretability of the network. Attached Figure Description

[0034] Figure 1 The semantic tree of various concepts related to "apple" in CNKI, as well as some words with the same semantic annotations as "apple". Detailed Implementation

[0035] 1. Related work

[0036] 1.1 Yiyuan and CNKI

[0037] In linguistics, a sememe is defined as the smallest semantic unit in human language. The meaning of a word can be represented by a combination of its sememes. Words with the same sememe annotation have the same meaning and can be substituted for each other. Dong et al. [DONG Z, DONG Q. Hownet and the computation of meaning [M]. USA: World Scientific Publishing Co Inc., 2006.] used more than 2,000 predefined sememes to annotate more than 200,000 concepts (senses, i.e., semantics) represented by Chinese and English words, forming a sememe knowledge base, namely HowNet. Each word in HowNet is represented as a tree structure. Since a word may have multiple concepts, each concept is a sememe tree. Figure 1As shown, the word "apple" contains four concepts: "Apple computer," "fruit," "Apple phone," and "apple tree." These are labeled using predefined semantic primitives in CNKI, forming a tree-like structure. A similar knowledge base is WordNet, a word-based semantic knowledge base built by Princeton University. Compared to WordNet, as a concept-based knowledge base, CNKI can find more semantically related words.

[0038] Because of its close relevance to the essence of language, semantic primitives have been widely used in various NLP tasks, such as text matching, semantic rationality assessment, and lexical fusion.

[0039] 1.2 Quantum Behavior Particle Swarm Optimization Algorithm

[0040] Quantum Behavioral Particle Swarm Optimization (QPSO) is an improved version of Particle Swarm Optimization (PSO), proposed in 2004 by Sun et al. [Sun J, Xu W, Feng B. A global search strategy of quantum-behaved particleswarm optimization[C] / / IEEE Conference on Cybernetics and Intelligent Systems, 2004.IEEE, 2004, 1: 111-116.]. QPSO, by simulating the strong uncertainty of superposition states in quantum systems, can cover the entire search space during iterative optimization, unlike some other PSO algorithms that lose global search capability and get trapped in local optima at the end of the search process. Its update method differs significantly from classical PSO; particles only update their positions, without considering velocity. Assuming a swarm of M particles searches in a D-dimensional search space, at the t-th iteration... This indicates the current position of the i-th particle in the j-th dimension of the search space; The position of the j-th dimension representing the highest optimization score achieved by the individual from the beginning of the iteration until now is called the individual optimal position; The position of the j-th dimension representing the highest optimization score achieved by the entire population from the start of iteration is called the global optimum. The update formulas for each dimension of each particle in the QPSO algorithm are as follows:

[0041]

[0042]

[0043]

[0044] in This indicates the best position of the average particle in the population. The position between the individual and the global optimum represents the attractor used to guide the search; α is a scaling factor greater than 0. and All are random numbers that conform to a standard uniform distribution.

[0045] 2Sememe-GQPSO

[0046] This invention proposes a text adversarial attack method based on semantic primitives and an improved QPSO algorithm using Gaussian distribution, called Sememe-GQPSO. Specifically, in the search space reduction stage, a method based on semantic primitives is first used to prepare candidate words for each word. Then, in the search stage, the QPSO algorithm improved with Gaussian distribution is used to search for candidate adversarial examples, ultimately obtaining the generated adversarial examples.

[0047] 2.1 Candidate word extraction method based on semantic primitives

[0048] The semantic primitive-based candidate word selection method was first proposed by Zang et al. [Zang Y, Qi F, Yang C, et al. Word-level Textual Adversarial Attacking as Combinatorial Optimization[C] / / Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:6066-6080.]. This method selects words with the same semantic primitives as the original input word as candidate words. Compared with methods such as language model prediction scores, word vector distance-based methods, and synonym-based methods, the semantic primitive-based method can find more high-quality candidate words. Therefore, this invention also adopts this method to select candidate words.

[0049] Taking "apple" as an example, the root nodes of its semantic primitives for the four concepts are "computer," "fruit," "tool," and "tree." Corresponding words with the same semantic primitives include "MAC," "pineapple," "HUAWEI," and "fruittree," among others. This method identifies replacement words for "apple." However, if no other word shares the exact same semantic primitive, then it has no replacement word.

[0050] To reduce the introduction of grammatical errors, we only select content words for replacement and require that candidate words have the same part-of-speech tag as the original word. For words with multiple meanings, such as "apple," the original word can only be replaced by a candidate word if one of its concepts shares the same semantic annotation as another concept in the candidate word. Furthermore, we use part-of-speech tagging to retrieve more candidate words while avoiding the introduction of grammatical errors.

[0051] Using the method described above, a candidate word list is generated for each word in the sentence. To improve attack efficiency, only the candidate word that causes the most damage to the victim model (i.e., the candidate word that has the highest probability of being predicted as the target's incorrect label when only one word in the original sentence is replaced by one of its candidate words) is retained as the candidate word for that original word. Other candidate words with weaker damage are filtered out, further reducing the search space and improving attack efficiency.

[0052] 2.2 QPSO Search Method Based on Gaussian Distribution

[0053] For text adversarial problems, the solution space is formed by all substitution combinations of candidate words at each position in the sentence. Therefore, after obtaining candidate words at each position in the sentence using semantic primitives, the QPSO (GQPSO) algorithm with Gaussian distribution is used to search for adversarial samples in this discrete solution space. Each candidate adversarial sample corresponds to a position in the solution space; the probability value of the candidate adversarial sample being predicted by the model as the target mislabel corresponds to the optimization score of the particle at that position.

[0054] Specifically, given an original input O of length D, the solution space of its adversarial examples also has a dimension of D. When searching the solution space, the GQPSO algorithm first initializes a population containing M particles. Each particle is generated by randomly mutating a word from the original input O into a candidate word. The greater the harm a candidate word causes to the model, the higher its mutation probability. After generating the population, the optimization score of each particle is calculated. and Assign values. Then, in the maximum T iterations of optimization, continuously update the positions of each particle.

[0055] To find an adversarial example that can successfully attack the victim model with a smaller modification rate, the algorithm needs to improve its local search capability in the early stages of iteration and quickly reach the global optimum in the later stages. Therefore, based on the uniform distribution already used in the original QPSO, a Gaussian distribution term is introduced, resulting in the GQPSO particle position update formula:

[0056]

[0057] in It is a random number that conforms to a standard Gaussian distribution; The algorithm decreases linearly with the number of iterations to conduct a broad search in the early stages and approach the global optimum in the later stages; α max α min β are both positive constants. However, due to the discrete nature of the text, equation (4) cannot be directly used for updating. Therefore, the J(·) function is introduced. In the t-th iteration, equation (5-8) is first used to obtain the discrete result. Temporary positions of particles in each dimension

[0058]

[0059]

[0060]

[0061]

[0062] because The value may be outside the solution space, so the sigmoid(·) function is used to transform it, which is called the transition probability. As shown in formula (9), it represents the probability that the word corresponding to this dimension of the particle will be replaced by the word in the individual or global optimum. Finally, formula (10) is used to complete the particle position update:

[0063]

[0064]

[0065] In (10), due to α t The linear decrease in position guides the population to perform a broad search in the early stages and approach the global optimum in the later stages, thus improving the algorithm's attack effectiveness. After position updates, particle optimization scores are calculated. To expand the population's exploration range in the solution space, a mutation operation similar to that in initialization is performed, particle optimization scores are calculated again, and finally, individual and global optimum positions are updated before proceeding to the next iteration.

[0066] Throughout the algorithm process, each time the particle optimization score is calculated, it is checked whether the particle with the highest score can successfully mislead the model into predicting it as the target. If so, the corresponding adversarial sample is successfully attacked; otherwise, the attack fails after reaching the maximum number of iterations T.

[0067] 3 Experiments

[0068] 3.1 Dataset, Victim Model, and Attack Method

[0069] Regarding the datasets, three classic NLP classification datasets were selected, namely IMDB for the sentiment classification task.

[0070] Maas A,Daly RE,Pham PT,et al.Learning word vectors for sentimentanalysis[C] / / Proceedings of the 49th annual meeting of the association for computational linguistics:Human language technologies.2011:142-150.], SST-2[Socher R,Perelygin A,Wu J,et al.Recursive deep models for semanticcompositionality over a sentiment treebank[C] / / Proceedings of the 2013conference on empirical methods in natural language processing.2013:1631-1642.], and SNLI for natural language reasoning classification tasks [Bowman S, Angeli G, Potts C, et al. Alargeannotated corpus for learning natural language inference[C] / / Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing.2015:632-642.]. Both IMDB and SST-2 originate from the film review field and are binary classification task datasets where the input is a single sentence. The difference lies in that IMDB input sentences are generally longer, while SST-2 input sentences are shorter. SNLI, published by Stanford University, is a three-class classification task dataset where the input is a pair of presuppositional sentences, with the three categories being inclusion, contradiction, and neutral. More information about the datasets can be found in Table 1.

[0071] Table 1: Summary of information from the three benchmark datasets. 'Train', 'Dev', and 'Test' represent the number of training, validation, and test sets, respectively, and 'AvgWord' represents the average sentence length.

[0072]

[0073] For the victim model, Bi-LSTM and BERT models were selected. Bi-LSTM is a classic model in the field of NLP; BERT is a pre-trained language model based on bidirectional Transformers proposed by Devlin et al. in 2019 [Devlin J, Chang MW, Lee K, et al.BERT: Pre-training of DeepBidirectional Transformers for Language Understanding[C] / / Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019:4171-4186.], which achieved state-of-the-art results on multiple NLP tasks at the time of its proposal. The Bi-LSTM of this invention uses a max pooling layer model proposed by Conneau et al. in 2017 [Conneau A, Kiela D, Schwenk H, et al. Supervised Learning of Universal Sentence Representations from Natural Language Inference Data[C] / / Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.2017:670-680.], and uses 300-dimensional GloVe pre-trained word vectors [Pennington J, Socher R, Manning C D. Glove: Global vectors for word representation[C] / / Proceedings of the 2014 conference on empirical methods in natural language processing(EMNLP).2014:1532-1543.], with the hidden layer dimension set to 128 dimensions; the BERT model uses the base version.

[0074] This invention selected four attack methods for experimental comparison. The first comparison method is the E / L-Genetic method proposed by Alzantot et al., which searches for replacement words in word vectors, controls the number of replacement words by limiting the distance in the word vector space, and then uses a genetic algorithm to search for adversarial examples. The second comparison method is the Synonym-Greedy method proposed by Ren et al., which uses synonyms as replacement words, calculates their language model scores, and then uses a greedy algorithm to search for adversarial examples. The latter two are the Sememe-PSO method proposed by Zang et al. and the Sememe-QPSO method proposed by Xu et al., respectively. Both use semantic primitive-based methods to find replacement words. The difference is that Zang et al. used the PSO algorithm to search for adversarial examples, while Xu et al. used the QPSO algorithm.

[0075] 3.2 Evaluation Indicators

[0076] The experiment used attack success rate, average modification rate of adversarial examples, transferability, and imperceptibility for evaluation. The calculation methods for attack success rate and average modification rate of adversarial examples are as follows:

[0077]

[0078]

[0079] In the formula, |sent success | and | sent attack | represents the number of successfully attacked samples and the total number of attacked samples, respectively; l(O) represents the sentence length of an original input sample O; (O,X) calculates the edit distance, measuring the number of different words between O and its corresponding successfully attacked sample X.

[0080] The transferability of adversarial examples refers to the ability of an attack method to attack a DNN without prior access to the model. It reflects the universality of its attack capability on different victim models and is also an important indicator for evaluating the effectiveness of an attack method. The lower the model's classification accuracy on transferred samples, the more successfully the samples confuse the model, the higher the transferability, and the better the performance of the adversarial method.

[0081] Imperceptibility is a crucial requirement for evaluating the quality of adversarial examples. Besides the average modification rate of adversarial examples, it also includes human language requirements such as semantics, syntax, and fluency. This invention uses a scoring method by professionals for evaluation.

[0082] To improve evaluation efficiency, the original input length is limited to 10-100 characters, and the effective sample modification rate is limited to below 25%; if the modification rate exceeds this value, the attack is considered a failure. 1000 randomly selected correctly classified samples from the test dataset are used as the original input set. Furthermore, in SNLI, only the hypothesis statement is modified.

[0083] 3.3 Parameter Settings

[0084] In the GQPSO algorithm, the maximum number of iterations T is set to 20, the number of particles M in the population is 60, and α... max =1, α min The β value was 0.5, and the remaining parameter settings were consistent with those in the original papers to ensure experimental comparability. All experiments were run on workstations equipped with RTX 2080Ti GPUs and i9-9900X CPUs.

[0085] 3.4 Experimental Results

[0086] Table 2 shows the success rates of the five attack methods on two models across three datasets. It is evident that the novel method proposed in this invention, which incorporates a Gaussian distribution, achieves the highest attack success rate. For the IMDB and SST-2 datasets, both binary movie review datasets, the significant difference in average length between the two makes it easy for the modification rate to exceed the set limit when generating adversarial examples for SST-2, leading to attack failure. This results in a slightly lower success rate for attacking models on SST-2 compared to IMDB. For the SNLI dataset, being a three-class natural language inference dataset with sentence pairs as input and shorter sentence lengths, the attack difficulty is higher, resulting in an even lower success rate.

[0087] Table 2: Success Rate of Five Attack Methods

[0088]

[0089] The average modification rate of successfully attacked adversarial samples is shown in Table 3. It is evident that the method proposed in this invention achieves a higher attack success rate while maintaining an excellent modification rate.

[0090] Table 3. Average Modification Rate of Adversarial Samples for Five Attack Methods

[0091]

[0092] 3.5 Transferability

[0093] This invention uses the BERT model to classify adversarial examples generated for attacking the Bi-LSTM model, thereby evaluating the transferability of the samples, and vice versa. The classification accuracy of the two models for transferred adversarial examples is shown in Table 4. It is evident that our proposed method has better transferability and greater generalizability for adversarial examples. In particular, compared to the original Sememe-QPSO method by Xu et al., our method using a Gaussian distribution improvement shows significant optimization.

[0094] Table 4. Classification accuracy of the two models for transferred adversarial examples.

[0095]

[0096] 3.6 Manual Evaluation and Case Analysis

[0097] We adopted the evaluation method of Zang et al., selecting 500 pairs of original input samples and adversarial examples generated against the Bi-LSTM model on the SST-2 dataset. Three professionals scored these samples, classifying them as "machine-generated," "uncertain," or "human-written" based on their degree of conformity to human language requirements, assigning them scores of 1, 2, and 3 respectively. The scoring results are shown in Table 5. It is evident that, compared to methods based on word vectors and synonyms, the three methods based on semantic primitives for obtaining candidate words better ensure the imperceptibility of perturbations. Furthermore, compared to the methods of Zang et al. and Xu et al., the proposed GQPSO adversarial example search method can find higher-quality adversarial examples.

[0098] Table 5. Manual Scoring of Disturbance Imperceptibility

[0099]

[0100] Table 6 compares instances of adversarial examples generated for the Bi-LSTM model on the SST-2 dataset. It is evident that, while using semantic primitives to obtain candidate words, compared to the methods of Zang et al. and Xu et al., the method of this invention can successfully confuse the model with a lower modification rate, and the generated adversarial examples are more similar to human language, exhibiting higher sample quality. This further demonstrates the significant optimization effect of the method improved using Gaussian distribution.

[0101] Table 6 Comparison of Adversarial Example Instances

[0102]

[0103] This invention proposes a novel word-level text adversarial method. By incorporating a Gaussian distribution into the quantum behavior particle swarm optimization algorithm, the attack method's ability to attack models is enhanced. Comparative experiments demonstrate that the Sememe-GQPSO method improves the attack success rate, reduces the modification rate, generates higher-quality adversarial examples, and achieves better attack efficiency.

Claims

1. A text adversarial method based on Gaussian distribution improvement, characterized in that, The steps are as follows: 1) Perform word-level tagging on the corpus; 2) Perform word form restoration on each marker, remove affixes, and obtain the word prototype; 3) Using CNKI, obtain the semantic primitive tree of all concepts for each tagged word prototype, as well as the part-of-speech tags corresponding to the concepts; 4) For each tagged word prototype, filter its content word part-of-speech as valid part-of-speech, and initialize the candidate word list according to the part-of-speech; 5) If the part-of-speech tag of the marked word prototype has no valid part-of-speech tag, or the CNKI query result is empty, then directly return the initialized empty candidate word list; 6) Traverse all other tags in the corpus and find the word prototypes of other tags that have the same part of speech and the same semantic annotation as the word prototype of the current tag. If found, add the same affix as the current tag to the word prototype of this other tag and store it in the candidate word list corresponding to the part of speech of the current tag; 7) After completing step 6) for all tags, a complete list of candidate words is obtained; 8) For an original input sample O with D words that can be correctly predicted by the victim model, start generating a corresponding adversarial sample from it; first, select a candidate word list based on the part of speech of each word in the original sample O. If there are no candidate words in all words in the entire sample O, the attack fails and the algorithm ends. 9) For the j-th word X in the original sample O j Only the candidate word with the highest harm to the victim model is retained from its candidate word list, and this word is used as the original word X. j Candidate words; for text adversarial problems, the solution space is formed by all substitution combinations of candidate words at each position in the sentence; by performing the above operation on all D words in the original sample O, the solution space of the adversarial sample is obtained; after normalizing the harm degree of candidate words at each position, the mutation probability of each position is obtained; 10) Each candidate adversarial sample corresponds to a position in the solution space, which is called a particle in the quantum behavior particle swarm optimization algorithm with Gaussian distribution. The probability value that the candidate adversarial sample is predicted as the target wrong label by the victim model is called the optimization score of the particle at the position; when searching for the adversarial sample in the solution space using GQPSO, a population containing M particles is first initialized, each particle is randomly selected from the original input sample O according to the mutation probability of each position, and the candidate word C j is mutated into the candidate word 11) In the t-th iteration of the GQPSO algorithm, This indicates the current position of the i-th particle in the j-th dimension of the solution space; The position of the j-th dimension representing the highest optimization score achieved by this particle from the beginning of the iteration until now is called the individual optimum; This represents the j-th dimension position of the entire population from the start of iteration to the highest optimization score achieved so far, i.e., the current position of all... The largest of the three is called the global optimum; in the discrete case, it is used... This represents an attractor used to guide the search, situated between the individual and the global optimum; using The two represent the best position of the average particle in the population, and are calculated as follows: wherein is a function introduced due to text discreteness; is a random number with a standard uniform distribution; after initialization of the population, the optimization score of each particle is calculated, and is and assignment; 12) Check whether the particle with the highest optimization score can successfully mislead the model into predicting it as the target mislabel. If so, its corresponding adversarial sample is the sample to be found. The attack is successful and the algorithm ends. 13) In each iteration, the particle first updates its position, then repeats step 12); if no successful adversarial example is found, the particle mutation operation in step 10) is performed, and then step 12) is repeated; if still no example is found, the individual and global optimal positions are updated, and the next iteration begins up to the maximum number of iterations T; the specific method for position updating is as follows: First, use equation (4) to calculate the temporary positions of the particles in each dimension. and These are random numbers that conform to a standard uniform distribution and a standard Gaussian distribution, respectively. It is a scaling factor greater than 0, which decreases linearly with the number of iterations to conduct a broad search in the early stages and approach the global optimum in the later stages; α max α min β are both set positive constants; Due to The value can be outside the solution space, so the sigmoid(·) function is used to convert it, called transition probability As formula (5), the probability that the word corresponding to this dimension of the particle is replaced by the individual or global optimal word is represented, and finally the particle position update is completed using formula (6); 14) If the maximum number of iterations T is reached, the attack fails and the algorithm ends.

Images