A natural adversarial text generation method and device in a hard label black box scenario

Abstract

The application discloses a natural adversarial text generation method and device in a hard label black box scene, which comprises the following steps: acquiring discrete text X in a data set, identifying important words in the discrete text X through an important word discriminator, and generating an important word set I according to the important word identification result of the important word discriminator; a mask language model is used to generate a candidate word set for each important word in the important word set I, and a specified important word in the important word set I is replaced by a word in the corresponding candidate word set to generate an initialized adversarial text X * ; finally, a genetic algorithm is used to perform mutation, selection and crossover operations on the generated adversarial sample X * , so that an optimized adversarial sample X adv with a specified semantic similarity is obtained. The application has high attack success rate, good fluency and grammatical correctness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing security, and particularly relates to a natural adversarial text generation method and device in a hard label black box scenario. BACKGROUND

[0002] In recent years, deep neural networks have been successfully applied to real-world scenarios such as face recognition, autonomous driving, machine translation, and speech recognition. However, deep neural networks are vulnerable to adversarial samples. Adversarial samples are perturbations added to original samples that are not easily detected and can have a serious impact on deep neural networks.

[0003] Currently, there are two basic requirements for generating adversarial text: one is that the perturbation in the text is not easily detected and can effectively fool the target model; the other is that the adversarial text must be grammatically correct and consistent with the semantics of the original text. Due to the fundamental differences between text and image data, existing image attack methods cannot be transferred to text adversarial attacks. How to efficiently generate natural adversarial text in a hard label black box scenario has become a technical problem that needs to be solved. SUMMARY

[0004] In view of the above, the purpose of the present application is to provide a natural adversarial text generation method and device in a hard label black box scenario to solve or partially solve the above technical problems.

[0005] To achieve the above purpose, the first aspect of the present application provides a natural adversarial text generation method in a hard label black box scenario, comprising:

[0006] Obtaining discrete text X in a data set, identifying important words in the discrete text X through an important word discriminator, and generating an important word set I according to the important word identification result of the important word discriminator;

[0007] Using a masked language model to generate a candidate word set for each important word in the important word set I, and replacing a specified important word in the important word set I with a word in the corresponding candidate word set to generate an initialized adversarial text X * ;

[0008] Using a genetic algorithm to generate an adversarial sample X * Iteratively performing mutation, selection, and crossover operations to obtain an optimized adversarial sample X adv with a specified semantic similarity.

[0009] As an optimal solution for the natural adversarial text generation method in a hard label black box scenario, during the process of obtaining discrete text X in a data set, the input text is divided by words to obtain a text sequence composed of words, the text sequence is truncated and completed, and converted into token encoding.

[0010] As the preferred solution of the natural adversarial text generation method in the hard label black box scenario, when the important word discriminator identifies the important word in the discrete text X, the discrete text X is input into the BERT model, and a vector containing context semantics is generated for each word through the BERT model;

[0011] The generated vector is input into the binary classifier, and whether the current word is an important word is determined according to the output of the binary classifier;

[0012] In the process of generating a candidate word set for each important word, a preset number of candidate word prediction results are obtained in descending order of the mask language model prediction probability corresponding to the position of the important word.

[0013] As the preferred solution of the natural adversarial text generation method in the hard label black box scenario, the candidate word set for each important word is screened, and the original important word, the candidate word at the subword level, and the candidate word with inconsistent parts of speech are excluded from the candidate word set.

[0014] As the preferred solution of the natural adversarial text generation method in the hard label black box scenario, if the generated adversarial text X * does not cause the target model to make a mistake, the remaining important words in the important word set I are replaced until the replaced text causes the target model to make a mistake or all important words in the important word set I are replaced;

[0015] If the generated adversarial text X * causes the target model to make a mistake, the replacement of the remaining important words in the important word set I is stopped, and the current adversarial text X * is taken as the initialized adversarial sample.

[0016] As the preferred solution of the natural adversarial text generation method in the hard label black box scenario, the generated adversarial text X * is scored for each replacement using semantic similarity, and the replacement operation is performed in descending order of the similarity score until the replaced adversarial text X * cannot successfully attack the target model within the decision boundary of the target model.

[0017] The second aspect of the present application provides a natural adversarial text generation device in a hard label black box scenario, comprising:

[0018] An important word identification module is configured to obtain a discrete text X in a data set, identify important words in the discrete text X through an important word discriminator, and generate an important word set I according to the important word identification result of the important word discriminator;

[0019] The candidate word generation module is configured to generate a candidate word set for each important word in the important word set I by using a mask language model;

[0020] The adversarial text initialization module is configured to replace a specified important word in the important word set I with a word in the corresponding candidate word set to generate an initialized adversarial text X * ;

[0021] The genetic algorithm optimization module is configured to optimize the generated adversarial sample X * by performing mutation, selection and crossover operations in iterations to obtain an optimized adversarial sample X with a specified semantic similarity adv .

[0022] As an optimal solution of the natural adversarial text generation device in the hard label black box scenario, the text preprocessing module is further configured to divide the input text according to words during the process of obtaining the discrete text X, obtain a text sequence composed of words, and perform truncation and completion on the text sequence and convert the text sequence into token encoding.

[0023] As an optimal solution of the natural adversarial text generation device in the hard label black box scenario, the important word identification module is configured to input the discrete text X into a BERT model, and generate a vector containing context semantics for each word by using the BERT model.

[0024] The generated vector is input into a binary classifier, and whether the current word is an important word is determined according to the output of the binary classifier.

[0025] As an optimal solution of the natural adversarial text generation device in the hard label black box scenario, during the process of generating a candidate word set for each important word by the candidate word generation module, a preset number of candidate word prediction results are obtained in descending order of prediction probability of the mask language model corresponding to the position of the important word.

[0026] As an optimal solution of the natural adversarial text generation device in the hard label black box scenario, the candidate word screening module is further configured to screen the candidate word set of each important word, and screen out the original important word, the candidate word at the subword level and the candidate word with inconsistent parts of speech from the candidate word set.

[0027] As an optimal solution of the natural adversarial text generation device in the hard label black box scenario, in the adversarial text initialization module, if the generated adversarial text X * does not cause the target model to make a mistake, the remaining important words in the important word set I are replaced until the replaced text causes the target model to make a mistake or all important words in the important word set I are replaced; if the generated adversarial text X *Let the target model predict an error, stop replacing the remaining important words in the important word set I, and output the current adversarial text X * As the initialized adversarial sample.

[0028] As the preferred solution of the natural adversarial text generation device in the hard label black box scenario, in the adversarial text initialization module, the adversarial text X * generated after replacement is scored using semantic similarity, and the replacement operation is performed from high to low according to the similarity score, until the adversarial text X * after replacement reaches the decision boundary of the target model and cannot successfully attack the target model.

[0029] The third aspect of the present application provides an electronic device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to realize the hard label black box scenario natural adversarial text generation method of the first aspect.

[0030] The fourth aspect of the present application provides a non-transitory computer readable storage medium, which stores computer instructions for causing a computer to execute the hard label black box scenario natural adversarial text generation method of the first aspect.

[0031] As can be seen from the above, the technical solution provided by the present application, by obtaining the discrete text X in the data set, identifying the important words in the discrete text X by the important word discriminator, and generating the important word set I according to the important word identification result of the important word discriminator; using a mask language model to generate a candidate word set for each important word in the important word set I, and replacing the specified important word in the important word set I with the word in the corresponding candidate word set to generate the initialized adversarial text X * ; using genetic algorithm to generate the adversarial sample X * Iterative mutation, selection and crossover operation to obtain the optimized adversarial sample X adv with specified semantic similarity. The present application has high attack success rate, good fluency and grammatical correctness. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the present application or related art, the following will briefly introduce the drawings needed to be used in the embodiments or related art description. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0033] Figure 1A flowchart of a natural adversarial text generation method in a hard label black box scenario of an embodiment of the present application is shown in FIG. 1.

[0034] Figure 2 A flowchart of important word set generation in a natural adversarial text generation method in a hard label black box scenario of an embodiment of the present application is shown in FIG. 2.

[0035] Figure 3 A flowchart of a natural adversarial text generation device in a hard label black box scenario of an embodiment of the present application is shown in FIG. 3.

[0036] Figure 4 A structural diagram of an electronic device of an embodiment of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0037] To make the objectives, technical solutions and advantages of the present application clearer, further detailed descriptions will be given below with reference to the embodiments and the accompanying drawings.

[0038] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood as the common meanings understood by those with ordinary skills in the art to which the present application belongs. The terms "first", "second", and similar terms used in the embodiments of the present application do not represent any order, number, or importance, but are only used to distinguish different components. The terms "include" or "contain" and similar terms mean that the elements or objects before the terms cover the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects.

[0039] According to the degree of understanding of the target model by the attacker, current text adversarial attacks are divided into two categories: white box attack and black box attack. In black box attack, the attacker does not need to know the details of the target model, but can obtain the input and output of the target model, so the black box attack scenario is more realistic.

[0040] The related art proposes a more realistic black box attack scenario, i.e., a hard label black box scenario. In this scenario, the attacker does not know the structure, gradient, and confidence score of the target model, cannot access the training data and substitute model, and can only generate adversarial samples by observing the decision output of the target model. The hard label black box scenario is more realistic and has more practical significance.

[0041] In the related art, first, a word is randomly selected from an original text, then a synonym set constructed from an anti-fitting space is used to replace the target word, and then a population-based optimization algorithm is used to maximize the overall semantic similarity between the original sample and the adversarial sample. Since the word is randomly selected in the initialization stage, it is not possible to accurately attack the word that has an important influence on the classification of the target model, and the number of times of accessing the target model in the attack process is still high, which leads to the attack being easily detected in real application; at the same time, when the synonym replacement is performed, whether it conforms to the context semantics is not considered, and the generated adversarial sample lacks fluency and grammatical correctness.

[0042] The related art also proposes a method of training an attack agent using reinforcement learning in an attack scenario to attack text at the sentence level and the word level. This method uses the training data of the target model to obtain the attack agent, and assumes that the attack agent knows the structure of the target model when attacking the target model. However, this method performs well when attacking other models similar to the target model, but performs poorly when attacking unknown target model structures.

[0043] Therefore, the present application provides a natural adversarial text generation method in a hard label black box scenario to efficiently generate natural adversarial text in a hard label black box scenario.

[0044] Referring to Figure 1 and Figure 2 , the embodiments of the present application provide a natural adversarial text generation method in a hard label black box scenario, comprising:

[0045] obtain a discrete text X in a data set, identify important words in the discrete text X through an important word discriminator, and generate an important word set I according to the important word identification result of the important word discriminator;

[0046] generate a candidate word set for each important word in the important word set I using a masked language model, and replace a specified important word in the important word set I with a word in the corresponding candidate word set to generate an initialized adversarial text X * ;

[0047] use a genetic algorithm to generate an adversarial sample X * , and perform mutation, selection and crossover operations on the generated adversarial sample X adv .

[0048] In this embodiment, during the process of acquiring discrete text X from the dataset, the input text is divided into words to obtain a text sequence composed of words. The text sequence is then truncated, completed, and converted into token encoding. When the important word discriminator identifies important words in the discrete text X, the discrete text X is input into the BERT model, which generates a vector containing contextual semantics for each word. The generated vector is then input into a binary classifier, and the output of the binary classifier is used to determine whether the current word is an important word. During the process of generating a candidate word set for each important word, a preset number of candidate word prediction results are obtained, ranked from high to low, based on the predicted probability of the language model at the corresponding position of the important word.

[0049] Specifically, for discrete text X={w1,w2⋯,w m Each word w in} i The important word discriminator determines the meaning of a word w based on its contextual semantics. i Does it have a significant impact on text classification? By defining the text X={w1,w2⋯,w...} m The entire input is fed into the BERT model, and the BERT model processes each word w. i Generate a vector E that includes contextual semantics. i Then vector E i The input is fed into a binary classifier f, and the current word w is determined based on the output of the binary classifier f. i Is it an important word? The binary classifier f will classify all words w in the text. i It is divided into two categories {0,1} to indicate whether the current word is an important word.

[0050] See Figure 2 In this embodiment, the structure of the important word discriminator is as follows: a BERT model and a binary classifier f are cascaded. The BERT model contains 768 hidden units, has 12 layers, each layer has 12 heads, and has 110M parameters. The binary classifier f has an input dimension of 512 and an output dimension of 2.

[0051] The prediction result for each word in the discrete text X is represented as follows:

[0052]

[0053] In the formula, f i It is the logical output of the binary classifier; c i ϵC (C={0,1}), if c i =1 indicates that the current word is an important word; w and b These are the weights and biases of the binary classifier.

[0054] Specifically, for all the words of the discrete text X = {w1, w2,..., w m , the words identified as important by the important word identifier are stored in set I according to the identification result of the important word identifier. The construction of the important word set I of the discrete text X is represented as follows.

[0055] In this embodiment, after obtaining the important word set I in the discrete text X, a candidate word set needs to be generated for each word in the important word set I for subsequent initialization of the adversarial sample generation.

[0056] Specifically, the discrete text X is input into the Masked Language Model, which generates a series of predicted candidate words suitable for the current context environment at the corresponding position of each word according to the context semantics. For the important word w i in the important word set I, the top N results with high prediction probability of the corresponding position of the Masked Language Model are obtained to form the candidate word set C(w i ) of the important word w i . Wherein, the Masked Language Model refers to the model architecture of the BERT model in the Masked LM pre-training task.

[0057] In this embodiment, the candidate word set of each important word is screened, and the original important word, the candidate word at the subword level, and the candidate word with inconsistent parts of speech are excluded from the candidate word set.

[0058] Specifically, for the candidate word set C(w i ) of each important word w i , screening is needed: the original important word w i is screened out to avoid invalid access to the target model when replacing. Since the Masked Language Model encoding is at the subword level, there may be subwords in the Masked Language Model prediction results, which need to be screened out to make the generated adversarial samples not easy to be detected, ensure the grammatical correctness and semanticity, in addition, the candidate words with inconsistent parts of speech are selected to avoid the generated adversarial samples lacking grammatical correctness.

[0059] In this embodiment, if the generated adversarial text X * after replacement does not cause the target model to make a mistake, the remaining important words in the important word set I are replaced until the replaced text causes the target model to make a mistake or all the important words in the important word set I are replaced; if the generated adversarial text X * after replacement causes the target model to make a mistake, the replacement of the remaining important words in the important word set I is stopped, and the current adversarial text X * is taken as the initialized adversarial sample.

[0060] Specifically, for each important word w in the important word set I of the discrete text X i , a word t is randomly selected from its candidate word set C(w i ) to replace. If the generated adversarial text X * after replacement has successfully fooled the target model (i.e., made the target model make a mistake in prediction), no further replacement is performed on the other remaining words in the important word set I, and the current adversarial text X * is taken as the initialized adversarial sample. Otherwise, the replacement step is repeated for the other words in the important word set I until the adversarial text X * can successfully attack the target model or all the words in the important word set I are modified. After the adversarial text X * that can successfully fool the target model is obtained, in order to reduce the perturbation ratio in the adversarial sample and improve the semantic similarity, some of the replaced words in the adversarial text X * are replaced with their original words, thereby reducing the search space of the subsequent optimization algorithm.

[0061] In this embodiment, the adversarial text X * generated after replacement is scored for each replacement by using semantic similarity, and the replacement operation is performed from high to low according to the similarity scores until the adversarial text X * after replacement reaches the decision boundary of the target model and cannot successfully attack the target model.

[0062] Specifically, the specific operation of reducing the replaced words in the adversarial text X * is as follows: the adversarial text X * ={w1,t2⋯t j ⋯w m}, where t i represents the replaced word of the word w i . For each replaced word t * in the adversarial text X i , the original word t i is replaced, and semantic similarity Sim(X,X * ) is used to score each replacement (w i →t i ). The replacement operation (t i →w i ) is performed from high to low according to the similarity scores until the adversarial text X * after replacement reaches the decision boundary of the target model, i.e., cannot successfully attack the target model.

[0063] In this embodiment, for the adversarial sample X *It requires iterative mutation, selection, and crossover operations to find optimized adversarial examples X with good semantic similarity. adv .

[0064] Specifically, a genetic algorithm is used to optimize the adversarial examples generated in the initialization phase. The genetic algorithm is configured with the following settings: the number of samples K in each generation is set to 30, and the maximum number of iterations G is set to 100.

[0065] Specifically, the genetic algorithm requires an initial generation, utilizing the adversarial text X generated during the initialization phase. * To generate initial generation samples, the specific operation is as follows: adversarial text X * The positions of all replacement words are stored in the set pos. For each position idx in the set pos, the mutation operation will be performed against the text X. * The perturbation word t in the corresponding position idx Using the original word w idx Replace, if the replaced X * It can fool the target model and make X * As the initial generation P 0 One of the texts; otherwise, use C. widx Replace each word in the text and select the word X that has the highest semantic similarity to the original text. * .

[0066] The genetic algorithm also requires evaluating all samples in each generation and selecting the best sample in each generation. Specifically, this involves calculating the X value for each sample in the current generation. i The semantic similarity between Sim(X,X) and the original discrete text X i The text with the highest semantic similarity is considered the optimal sample X for the current generation. final If X final Capable of attacking the target model, X final Considered as the final adversarial example X adv Otherwise, X final They can be used directly as samples for the next generation.

[0067] If no adversarial sample is found, the genetic algorithm continues, and a pair of samples is selected from each generation. The selected pair of samples is subjected to crossover and mutation operations. The crossover operation is performed as follows: a pair of samples is randomly selected from the current generation as the parents of the next generation, and the probability of selection is positively correlated with the semantic similarity Sim. Then, a new sentence is synthesized from the two parents using a uniform distribution. For each new synthesized sentence, an index idx is randomly selected from the replacement word index pos. If the word at the corresponding position in child is the original word, the new synthesized sentence is used as the sample of the next generation. Otherwise, the new synthesized sentence is subjected to a mutation operation, and the result of the mutation operation is used as the sample of the next generation. The above-mentioned crossover and mutation operations are performed K-1 times, and all samples of the next generation are obtained.

[0068] Specifically, the mutation operation is as follows: first, the replacement word t * at the idxth position in the adversarial text X idx is replaced with the corresponding original word w idx . It is determined whether the updated adversarial text X * can successfully fool the target model. If the condition is met, X * is regarded as the final result of the current mutation operation. If the updated text does not meet the above condition, each word in the replacement word set C idx of the word w widx is replaced with w idx , and the corresponding text X * is generated. The semantic similarity between X * and the original text is calculated, and the X i with the highest similarity is selected as the result of the mutation operation. *

[0069] Specifically, the semantic similarity is calculated as follows:

[0070]

[0071] where V X , are the embedding vectors of the texts X and X i * , respectively, and is a vector obtained by averaging the embedding vectors of the corresponding words in the text.

[0072] The probability of selecting the parent of each generation is specifically represented as follows:

[0073] p = softmax(Scores)

[0074] where Scores is a list storing the semantic similarity between all texts in the current generation and the original text.

[0075] In summary, the application obtains discrete text X in a data set, identifies important words in the discrete text X through an important word identifier, generates an important word set I according to the important word identification result of the important word identifier, generates a candidate word set for each important word in the important word set I using a mask language model, and replaces a specified important word in the important word set I with a word in the corresponding candidate word set to generate an initialized adversarial text X * ; the generated adversarial sample X * is iteratively subjected to mutation, selection and crossover operations to obtain an optimized adversarial sample X adv with a specified semantic similarity. When the important word identifier identifies important words in the discrete text X, the discrete text X is input into a BERT model, and a vector containing context semantics is generated for each word through the BERT model; the generated vector is input into a binary classifier, and whether the current word is an important word is determined according to the output of the binary classifier; in the process of generating a candidate word set for each important word, a preset number of candidate word prediction results are obtained in descending order of the prediction probability of the important word in the mask language model. The candidate word set of each important word is screened, and the original important word, the candidate word at the subword level and the candidate word with inconsistent parts of speech are excluded from the candidate word set. If the generated adversarial text X * does not cause the target model to make a mistake, the remaining important words in the important word set I are replaced until the replaced text causes the target model to make a mistake or all important words in the important word set I are replaced; if the generated adversarial text X * causes the target model to make a mistake, the replacement of the remaining important words in the important word set I is stopped, and the current adversarial text X * is taken as the initialized adversarial sample. The generated adversarial text X * after replacement is scored using semantic similarity, and the replacement operation is performed in descending order of the similarity score until the replaced adversarial text X * reaches the target model decision boundary and cannot successfully attack the target model. The application has a high success rate, good fluency and grammatical correctness.

[0076] It should be noted that the method of the embodiments of the application can be executed by a single device, such as a computer or a server. The method of the embodiments of the application can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In this distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiments of the application, and the multiple devices can interact with each other to complete the method.

[0077] It is to be understood that the foregoing description is directed to embodiments of the application. Various embodiments are described herein, including the best mode of the inventors. It will be apparent, however, to those skilled in the art having the benefit of this disclosure, that variations and / or modifications of these embodiments can be made without departing from the spirit and scope of the application. Accordingly, it is intended that such variations and / or modifications be included within the scope of the application. It is the intention, therefore, to be limited only as indicated by the scope of the claims appended hereto.

[0078] Referring to Figure 3 , based on the same inventive concept, corresponding to the method of any of the above embodiments, the application also provides a natural adversarial text generation device in a hard tag black box scenario, comprising:

[0079] An important word identification module 1 is configured to obtain discrete text X in a data set, identify important words in the discrete text X through an important word identifier, and generate an important word set I according to an important word identification result of the important word identifier.

[0080] A candidate word generation module 2 is configured to generate a candidate word set for each important word in the important word set I using a masked language model.

[0081] An adversarial text initialization module 3 is configured to replace a specified important word in the important word set I with a word in the corresponding candidate word set to generate an initialized adversarial text X * .

[0082] A genetic algorithm optimization module 4 is configured to perform mutation, selection and crossover operations on the generated adversarial sample X * using a genetic algorithm to obtain an optimized adversarial sample X adv with a specified semantic similarity.

[0083] In the present application, a text preprocessing module 5 is also included, which is configured to divide input text into words during the process of obtaining discrete text X in a data set, obtain a text sequence composed of words, and truncate and complete the text sequence and convert it into token encoding.

[0084] In the present application, the discrete text X is input into a BERT model in the important word identification module 1, and a vector containing context semantics is generated for each word through the BERT model.

[0085] The generated vector is input into a binary classifier, and whether the current word is an important word is determined according to the output of the binary classifier.

[0086] In the present application, during the process of generating a candidate word set for each important word by the candidate word generation module 2, a preset number of candidate word prediction results corresponding to the positions of the important words are obtained from high to low according to the prediction probability of the position mask language model.

[0087] In the present application, the candidate word screening module 6 is further included for screening the candidate word set of each important word, and the original important word, the candidate word at the subword level, and the candidate word with inconsistent parts of speech are screened out from the candidate word set.

[0088] In the present application, in the adversarial text initialization module 3, if the generated adversarial text X * does not make the target model make a mistake, the remaining important words in the important word set I are replaced until the replaced text makes the target model make a mistake or all the important words in the important word set I are replaced; if the generated adversarial text X * makes the target model make a mistake, the replacement of the remaining important words in the important word set I is stopped, and the current adversarial text X * is taken as the initialized adversarial sample.

[0089] In the present application, in the adversarial text initialization module 3, the generated adversarial text X * is scored by using semantic similarity for each replacement, and the replacement operation is performed from high to low according to the similarity score until the replaced adversarial text X * cannot successfully attack the target model within the decision boundary of the target model.

[0090] For the convenience of description, the above system is described by dividing into various modules for description. Of course, when the present application is implemented, the functions of the modules can be implemented in the same or multiple software and / or hardware.

[0091] The device of the above embodiment is used to implement the natural adversarial text generation method in the corresponding hard label black box scenario of any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described here.

[0092] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the hard label black box scenario natural adversarial text generation method described in any of the above embodiments when executing the program.

[0093] Figure 4A more specific electronic device hardware structure schematic diagram provided by the embodiment is shown, and the device can include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected through the bus 1050 for communication between each other inside the device.

[0094] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present specification.

[0095] The memory 1020 can be implemented by a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 1020 and called and executed by the processor 1010.

[0096] The input / output interface 1030 is used to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0097] The communication interface 1040 is used to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or through a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0098] The bus 1050 includes a channel to transmit information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0099] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, but in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary to implement the scheme of the embodiments of the present application, and it is not necessary to contain all the components shown in the figure.

[0100] The electronic device of the above embodiment is used to implement the natural adversarial text generation method in the corresponding hard label black box scenario of any of the preceding embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here.

[0101] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present application also provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to execute the natural adversarial text generation method in the hard label black box scenario as described in any of the above embodiments.

[0102] The computer readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be realized by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0103] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute the natural adversarial text generation method in the hard label black box scenario as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which are not described here.

[0104] Those skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope (including claims) of the present application is limited to these examples; under the idea of the present application, the above embodiments or technical features in different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the embodiments of the present application as described above. In order to be brief, they are not provided in detail.

[0105] In addition, to simplify the description and discussion, and so as not to obscure the embodiments of the application with details that are well known to those skilled in the art, the drawings provided herein can or can not show such well-known features. Furthermore, devices can be shown in block diagram form in order to avoid obscuring the embodiments of the application, and this also acknowledges the fact that the details in regard to how such block diagram devices are implemented can vary greatly from one platform to another, and from one platform generation to another. In the case where specific details of the circuitry are set forth in order to describe the exemplary embodiments of the application, it should be apparent to those skilled in the art that the application can be practiced without or with variation of these specific details. The description is thus to be considered in all respects as illustrative and not restrictive.

[0106] While the application has been described in connection with specific embodiments thereof, it will be understood that many modifications, substitutions and changes can be made by those skilled in the art to the specific embodiments without departing from the spirit or scope of the application. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0107] It is intended to cover all alternatives, modifications and variations of this application falling within the scope of the appended claims. Accordingly, all such changes are intended to be included within the scope of the application as defined in the following claims. In this patent, the use of "the first" "the second" and other such specified terminology can only mean that a component so described is a component one of which is claimed. Such terminology may, but not necessarily, indicate more than one of the same type of components is present. Thus, as a shorthand, such terminology can be used herein to modify what is otherwise specified to be a singular component embodiment.

Claims

1. A natural adversarial text generation method in a hard label black box scenario, comprising: obtaining discrete text X in a data set, identifying important words in the discrete text X through an important word discriminator, and generating an important word set I according to the important word identification result of the important word discriminator; The candidate word set is generated for each important word in the important word set I by using a masked language model, and a specified important word in the important word set I is replaced by a word in the corresponding candidate word set to generate an initialized adversarial text ; Using a genetic algorithm on the generated adversarial sample The mutation, selection and crossover operations are iteratively performed to obtain an optimized adversarial sample X with a specified semantic similarity adv ; in the process of obtaining the discrete text X in the data set, dividing the input text according to words to obtain a text sequence composed of words, truncating and completing the text sequence, and converting it into token coding; when the important word discriminator identifies the important words in the discrete text X, inputting the discrete text X into a BERT model to generate a vector containing context semantics for each word through the BERT model; inputting the generated vector into a binary classifier to determine whether the current word is an important word according to the output of the binary classifier; in the process of generating a candidate word set for each important word, obtaining a preset number of candidate word prediction results from high to low according to the position mask language model prediction probability of the important word; screening the candidate word set of each important word to exclude the original important word, the candidate word at the subword level, and the candidate word with inconsistent parts of speech from the candidate word set; the structure of the important word discriminator used is: BERT model and binary classifier f are cascaded; wherein the BERT model contains 768 hidden units, 12 layers, 12 heads per layer, and a parameter amount of 110M; the input dimension of the binary classifier f is 512, and the output dimension is 2; the prediction result of each word in the discrete text X is represented as: wherein, f i is the logical output of the binary classifier; if c i = 1 indicates that the current word is an important word; the BERT model generates a vector i containing contextual semantics for each word w E i ; w and b are weight and bias values for the binary classifier; For all words of the discrete text , according to the identification result of the important word identifier, the words identified as important are stored in set I; the construction of the important word set I of the discrete text X is represented as: ; The set of important words for discrete text X Each important word in Generate the corresponding candidate word set and randomly select from its candidate word set Select words for replacement, replacing important words one by one, to generate adversarial text that can fool the target model. ; Adversarial text The positions of all replacement words are stored in a set pos; for each position idx in the set pos, the mutation operation will act against the text. The perturbation word t in the corresponding position idx Using the original word w idx Replace, if the replacement It can fool the target model, As the initial generation P 0 One of the texts; otherwise, use the set of replacement words C. widx Each word in the text is replaced, and the word with the highest semantic similarity to the original text is selected. ; Calculate X for each sample in the current generation i The semantic similarity between Sim(X,X) and the original discrete text X i The text with the highest semantic similarity is considered the optimal sample X for the current generation. final If X final Capable of attacking the target model, X final Considered as the final adversarial example X adv Otherwise, X final Directly used as samples for the next generation; randomly extracting a pair of samples from the current generation as the parents of the next generation, wherein the extraction probability is positively correlated with the semantic similarity Sim; using uniform distribution to independently sample from the two parents to synthesize a new sentence; for each newly synthesized sentence, randomly extract an index idx from the replacement word index pos, if the word at the corresponding position in child is the original word, take the newly synthesized sentence as the sample of the next generation; otherwise, perform a mutation operation on the newly synthesized sentence, and take the result of the mutation operation as the sample of the next generation; the above crossover and mutation operations need to be performed K-1 times, and all samples of the next generation can be obtained; The mutation operation is as follows: First, the adversarial text is... The word t is replaced at position idx. idx Replace with its original word w idx ; Determine the updated adversarial text Whether the target model can be successfully fooled, and if this condition is met, then... Consider this as the final result of the current mutation operation; if the updated text does not meet the above conditions, use the word w. idx The set of replacement words C widx Each word in w idx Perform the replacement and generate the corresponding text. For those that meet the conditions Calculate the semantic similarity between it and the original text, and select the one with the highest similarity. As a result of the mutation operation.

2. The natural adversarial text generation method in a hard-label black-box scenario according to claim 1, wherein, If the generated adversarial text after replacement If the target model does not make a mistake in prediction, replace the remaining important words in the important word set I until the text after replacement makes the target model make a mistake in prediction or all important words in the important word set I are replaced. If the generated adversarial text after replacement Let the target model make a mistake in prediction, stop replacing the remaining important words in the important word set I, and take the current adversarial text As the initialized adversarial sample.

3. The natural adversarial text generation method in a hard-label black-box scenario according to claim 2, wherein, The generated adversarial text after replacement Each replacement is scored using semantic similarity, and the replacement operation is performed from high to low according to the similarity score until the replacement The target model cannot be successfully attacked within the target model decision boundary.

4. The device for natural adversarial text generation in a hard-label black-box scenario, adopts the method for natural adversarial text generation in a hard-label black-box scenario according to any one of claims 1 to 3, wherein, comprising: an important word identification module for obtaining discrete text X in a data set, identifying important words in the discrete text X through an important word discriminator, and generating an important word set I according to the important word identification result of the important word discriminator; a candidate word generation module for generating a candidate word set for each important word in the important word set I using a mask language model; The adversarial text initialization module is configured to replace the specified important word in the important word set I with a word in the corresponding candidate word set to generate an initialized adversarial text ; a genetic algorithm optimization module configured to optimize the generated adversarial sample using a genetic algorithm The iteration is performed by mutation, selection and crossover operation to obtain the optimized adversarial sample X with specified semantic similarity adv .

5. The natural adversarial text generation device under a hard-label black-box scenario according to claim 4, wherein, further comprising a text preprocessing module for dividing the input text according to words to obtain a text sequence composed of words in the process of obtaining the discrete text X in the data set, truncating and completing the text sequence, and converting it into token coding.

6. The natural adversarial text generation device under a hard-label black-box scenario according to claim 5, wherein, in the important word identification module, inputting the discrete text X into a BERT model to generate a vector containing context semantics for each word through the BERT model; inputting the generated vector into a binary classifier to determine whether the current word is an important word according to the output of the binary classifier.

7. The natural adversarial text generation device under a hard-label black-box scenario according to claim 6, wherein, In the process of generating a candidate word set for each important word, the candidate word generation module acquires a preset number of candidate word prediction results from high to low in terms of position mask language model prediction probability corresponding to the important word; The candidate word screening module is further included to screen the candidate word set of each important word, and to screen out the original important word, the candidate word at the subword level, and the candidate word with inconsistent parts of speech from the candidate word set. In the adversarial text initialization module, if the generated adversarial text after replacement makes the target model prediction error, replace the remaining important words in the important word set I until the replaced text makes the target model prediction error or all important words in the important word set I are replaced; if the generated adversarial text after replacement makes the target model prediction error, stop replacing the remaining important words in the important word set I, and take the current adversarial text as the initialized adversarial sample; In the adversarial text initialization module, the generated adversarial text after replacement is scored by semantic similarity, and the replacement operation is performed from high to low according to the similarity score, until the adversarial text after replacement cannot successfully attack the target model within the target model decision boundary.

Images