Adversarial sample generation method and device, electronic equipment and storage medium

Abstract

The application provides an adversarial sample generation method and device, electronic equipment and storage medium. The method comprises: obtaining sample data, taking the sample data as initial data, calculating the importance score of each word in the initial data based on a semantic classifier, and sorting the words based on the importance score; calculating the synonym set of each word and the semantic score of each synonym by using a bidirectional encoding representation algorithm based on a converter; selecting a word as a word to be replaced based on the sorting order, calculating the attack score of each synonym of the word to be replaced based on the synonym set of the word to be replaced and the semantic score of each synonym, selecting the synonym with the highest attack score, replacing the word to be replaced in the initial data, and generating test data; determining whether the test data can complete an attack on the sample data, and if so, taking the test data as an adversarial sample of the sample data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to an adversarial sample generation method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Unlike modifying pixel values in images, text belongs to discrete data, and perturbation of text is easy to be detected and difficult to maintain semantic invariance. Taking Chinese as an example, Chinese is composed of a large number of Chinese characters with independent meanings, and the meaning of each sentence directly depends on the meaning of the Chinese characters that constitute the sentence and the way these Chinese characters are related to each other in syntax, so simple modification of a single Chinese character can greatly change the original semantics of the sentence, thus making it more difficult to attack Chinese.

[0003] However, with the advancement of technology, in order to cope with possible risks and further improve the security of neural networks, there is an increasing demand for higher semantic adversarial samples for training when defending against adversarial samples of existing neural networks, and thus the problem of how to generate adversarial samples with better semantics has become a technical problem to be solved. SUMMARY

[0004] In view of the above, the present application proposes an adversarial sample generation method, device, electronic device and storage medium to generate adversarial samples with better semantics, thereby strengthening the security training of the current neural network and improving the security of text recognition and text detection.

[0005] To achieve the above purpose, the present application provides an adversarial sample generation method, comprising:

[0006] Obtaining sample data, taking the sample data as initial data, calculating the importance score of each word in the initial data based on a semantic classifier, and sorting the words based on the importance score;

[0007] Calculating the synonym set of each word and the semantic score of each synonym using a bidirectional encoder representation algorithm based on a transformer;

[0008] Selecting one of the words as a word to be replaced based on the sorting order, calculating the attack score of each synonym of the word to be replaced based on the synonym set of the word to be replaced and the semantic score of each synonym;

[0009] Selecting the synonym with the highest attack score, replacing the word to be replaced in the initial data, and generating test data;

[0010] Determining whether the test data can complete an attack on the sample data, and if so, taking the test data as an adversarial sample of the sample data.

[0011] In some embodiments, the determining whether the to-be-tested data can complete the attack behavior on the sample data further includes:

[0012] If no, a next word of the to-be-replaced word is selected based on the sorting order to replace the to-be-replaced word, and the to-be-tested data is replaced by the initial data.

[0013] The to-be-tested data is regenerated, and the to-be-tested data is determined again based on the newly generated to-be-tested data whether the to-be-tested data can complete the attack behavior on the sample data.

[0014] In some embodiments, the determining whether the to-be-tested data can complete the attack behavior on the sample data further includes:

[0015] The number of times of regenerating the to-be-tested data is counted, and if the number of times is greater than a preset threshold, it is considered that the to-be-tested data completes the attack behavior on the sample data.

[0016] In some embodiments, the calculating the importance score of each word in the initial data based on the semantic classifier specifically includes:

[0017] Cw i =F(w1,...,w i-1 ,w i ,W i+1 ,...,w N )-F(w1,....,w i-1 ,w i+1 ,...,w N )

[0018] wherein w i represents the i-th word, N is the total number of words in the initial data, Cw i represents the importance score of the i-th word, F(w1,...,w i-1 ,w i ,w i+1 ,...,w N ) represents the score of the initial data output by the semantic classifier F, and F(w1,....,w i-1 ,w i+1 ,...,w N ) represents the score of the initial data output by the semantic classifier F after removing the i-th word w i .

[0019] In some embodiments, the calculating the attack score of each synonym of the to-be-replaced word specifically includes:

[0020]

[0021] wherein, denotes a synonym w i of the i-th word w i , and β is a preset hyperparameter, denotes a semantic score of a synonym w i of the i-th word w i , and β is a preset hyperparameter, denotes an initial score of a synonym w i of the i-th word w i , F(w1,...,w i-1 ,w i ,w i+1 ,...,w N ) denotes a score of the initial data output based on the semantic classifier F, and F(w1,...,w i-1 ,w i ',w i+1 ,...,w N ) denotes a score of the initial data output based on the semantic classifier F after replacing the i-th word w i with a synonym w i '.

[0022] In some embodiments, the transformer-based bidirectional encoding representation algorithm is a BERT algorithm;

[0023] The calculating, by the transformer-based bidirectional encoding representation algorithm, of the synonym set of each word comprises:

[0024] inputting the sample data into the BERT algorithm, so that the BERT algorithm performs synonym prediction on each word of the sample data, generates a plurality of word group sets corresponding to each word, performs antonym filtering on the word group sets, and takes the filtered word group sets as the synonym set.

[0025] In some embodiments, the sorting of the words based on the importance scores comprises: arranging the words in descending order according to the importance scores.

[0026] The selecting, based on the sorting order, of one word as a word to be replaced comprises:

[0027] selecting the first word in the descending order as the word to be replaced.

[0028] Based on the same concept, the present application also provides an adversarial sample generation device, comprising:

[0029] an acquisition module configured to acquire sample data, take the sample data as initial data, calculate an importance score of each word in the initial data based on a semantic classifier, and sort the words based on the importance score;

[0030] a synonym module configured to calculate a synonym set of each word and a semantic score of each synonym of the synonym set by using a bidirectional encoder representations from transformers algorithm based on a converter;

[0031] a calculation module configured to select one of the words as a word to be replaced based on a sorting order, and calculate an attack score of each synonym of the word to be replaced based on the synonym set of the word to be replaced and the semantic score of each synonym;

[0032] a generation module configured to select a synonym with the highest attack score, replace the word to be replaced with the synonym in the initial data, and generate test data;

[0033] a judgment module configured to determine whether the test data can complete an attack behavior on the sample data, and if so, take the test data as an adversarial sample of the sample data.

[0034] Based on the same idea, the present application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method according to any one of the preceding embodiments.

[0035] Based on the same idea, the present application further provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to implement the method according to any one of the preceding embodiments.

[0036] As can be seen from the above description, the adversarial example generation method, apparatus, electronic device, and storage medium provided in this application include: acquiring sample data; using the sample data as initial data; calculating the importance score of each word in the initial data based on a semantic classifier; sorting the words based on the importance scores; calculating the synonym set and semantic score of each synonym for each word using a bidirectional encoding representation algorithm based on a converter; selecting a word as the word to be replaced based on the sorting order; calculating the attack score of each synonym of the word to be replaced based on the synonym set and semantic score of each synonym; selecting the synonym with the highest attack score and replacing the word to be replaced in the initial data to generate test data; determining whether the test data can complete the attack behavior on the sample data; if so, using the test data as an adversarial example of the sample data. This application uses a converter-based bidirectional encoding representation algorithm to determine the synonyms and semantic properties of each word in the initial data. Then, it calculates the attack power of each synonym based on its semantic properties. Based on the semantic properties, it selects the synonym with the highest attack power to replace the word in the initial data. The newly generated test data is then used to attack the sample data. The success of the attack determines whether the test data can be used as an adversarial example. Furthermore, it proposes a semantic-integrated word selection replacement index to generate adversarial examples with better semantic properties, thereby strengthening the security training of current neural networks and improving the security of text recognition and detection. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating an adversarial sample generation method proposed in an embodiment of this application.

[0039] Figure 2 This is a schematic diagram of the structure of an adversarial sample generation device proposed in an embodiment of this application;

[0040] Figure 3 This is a schematic diagram of the electronic device structure proposed in the embodiments of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this specification clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0042] It should be noted that, unless otherwise defined, technical terms or scientific terms used in the embodiments of the present application shall have the common meaning understood by one of ordinary skill in the art to which the embodiments of the present application belong. The terms "first", "second", and similar terms used in the embodiments of the present application do not denote any order, quantity, or importance, but are merely used to distinguish different components. The terms "include", "contain", and similar terms mean that the elements, objects, or method steps listed before the terms encompass the elements, objects, or method steps listed after the terms and their equivalents, without excluding other elements, objects, or method steps. The terms "connect" or "connected" and similar terms do not mean only physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", and the like are merely used to indicate relative positional relationships, and when the absolute positions of the described objects change, the relative positional relationships can also change accordingly.

[0043] As described in the background section, at present, the text field has emerged attack methods that generate more natural adversarial samples. In the research of generating adversarial samples, a series of available attack models have been proposed.

[0044] Alzantot et al. first use a synonym dictionary in the counter-fitting method to find synonyms in the word embedding space, then select synonyms that can maximize the probability of the target class to replace the original words, and finally apply genetic algorithms to find adversarial samples. However, the search space for searching synonyms for multiple word positions is very large. Textbugger and DeepwordBug use multiple strategies to modify important words or characters in sentences. Among them, the commonly used strategies are insertion, replacement, and deletion, which can effectively reduce the accuracy of text classifiers, but insertion and replacement will affect the semantic nature of adversarial samples. Textfooler uses a change strategy to replace synonyms. The replacement process needs to ensure that the POS of the original word and the replacement word is consistent, and USE is used to ensure that the semantic difference between the original sentence and the adversarial sentence is not large. In order to be able to use attack algorithms in the image field, Zhao et al. applied the Deepfool algorithm in the image field to the text field to find adversarial samples. First, the Deepfool algorithm is used to calculate the distance r i between the current sample point and the target class, then find the words that have a greater impact on the classification result, and finally select the changed sample after the word replacement that makes the distance r iThe PSO proposes two optimization strategies for finding word replacement adversarial samples. One is to use the root HowNet to find more synonyms. The root itself represents the meaning of the word, for example, "spect" in English means "to see", and "mini" means "small"; the other is to introduce the particle swarm optimization algorithm into the search for adversarial samples.

[0045] The above-mentioned synonym search strategy either obtains a synonym set through a synonym dictionary or calculates the cosine distance between words in the word embedding space. Then, some complex strategies such as language models and POS-check are needed to ensure the semantic nature and naturalness of the adversarial text. BertAttack proposes a strategy for finding synonyms using the BERT model. Assuming that the synonyms of the ith word are needed, the ith word is replaced with a blank (for example, [MASK]) and input into the BERT model to obtain a plurality of words suitable for the current context, and finally the antonyms are filtered out to obtain the synonym set of the ith word. However, BertAttack does not consider the synonym to be inserted when determining the importance of the word. Li et al. also use BERT to find synonyms, but the word modification strategy includes deletion, replacement and fusion, and Li et al. consider the synonym to be filled when selecting the replacement position.

[0046] The above research status of targeted adversarial sample generation shows that in the related art, the model for generating adversarial samples has the problem of poor semantic nature of the adversarial samples. Therefore, how to solve the problem of generating adversarial samples with good semantic nature has become a technical problem to be solved.

[0047] In combination with the above actual situation, the embodiments of the present application propose an adversarial sample generation scheme. The present application determines the synonyms of each word in the initial data and the semantic nature of the synonyms based on the transformer-based bidirectional encoding representation algorithm. Then, the semantic nature is combined to calculate the aggressiveness of each synonym, and the synonym with the highest aggressiveness is selected to replace the word of the initial data based on the semantic nature. The newly generated test data is used to attack the sample data, and whether the attack is successful is used to determine whether the test data can become an adversarial sample. Further, a selection replacement word index is proposed to generate an adversarial sample with good semantic nature, to strengthen the security training of the current neural network and improve the security of text recognition and text detection.

[0048] As shown in Figure 1 The method specifically includes:

[0049] At step 101, sample data is obtained, the sample data is taken as initial data, an importance score of each word in the initial data is calculated based on a semantic classifier, and the words are sorted based on the importance score.

[0050] In this step, the sample data is text data in various text formats, which can be a sentence, a paragraph, or a whole document, etc. Then, the sample data is taken as initial data to facilitate subsequent calculation. Then, the semantic classifier is a kind of classifier, and classification is a very important method of data mining. The concept of classification is to learn a classification function or construct a classification model (i.e. usually called classifier) based on existing data. The function or model can map the data records in the database to a certain one of the given categories, so as to be applied to data prediction. The classifier is a general term for the method of classifying samples in data mining, including decision tree, logistic regression, naive Bayes, neural network, etc. algorithm.

[0051] In this scheme, by using the semantic classifier, a classifier score can be output after inputting a piece of data, and the importance score of each word can be calculated by the semantic classifier. The importance score of a specific word is calculated by the difference between the score of the initial data containing all words and the score of the initial data after removing the specific word. The importance score is a score representing the importance of a word in the corresponding initial data. Then, since the initial data can be in Chinese form or in foreign language form, in initial data in English or French, etc. with a small size alphabet form, the word is the basic unit of text composition, for example: "apple"; and in initial data in Chinese form, the word can be a character, for example: "ah", or a combination of several characters, for example: "we", and the specific segmentation method can be realized according to the related segmentation algorithm. In this embodiment, the initial data is taken as English for illustration.

[0052] Finally, after calculating the importance scores of all words, all words can be sorted, which can be sorted in descending or ascending order according to the importance score, etc.

[0053] In some embodiments, the semantic classifier of the present application can be based on a basic classifier model and trained for IMDB, MR or MNLI data sets to obtain a final semantic classifier. Among them, for IMDB and MR data sets, a 1-layer bidirectional LSTM with 150 hidden units, BERT three different structure models can be used for training respectively. For MNLI data set, a model ESIM which comprehensively applies BiLSTM and attention mechanism is used. That is, in the present embodiment, the sample uses MR, IMBD sentiment classification data set, or MNLI natural language inference data set, three kinds of data sets can be directly obtained through the network. Among them, the MR data set is a 2-classification data set, the number of training sets is 9000, the number of test sets is 1000, and the average length is 20. Among them, the IMDB data set is a 2-classification data set, the number of training sets is 25000, the number of test sets is 25000, and the average length is 215. Among them, the MNLI data set is a reasoning 3-classification data set, the number of training sets is 433000, the number of test sets is 10000, and the average length is 11.

[0054] Step 102, calculating the synonym set of each word and the semantic score of each synonym by using the bidirectional encoding representation algorithm based on the converter.

[0055] In the present step, the bidirectional encoding representation algorithm based on the converter is the BERT algorithm, the full name of BERT is Bidirectional Encoder Representation from Transformer, the BERT algorithm is a bidirectional encoding representation algorithm based on the Transformer algorithm, the Transformer algorithm is based on the Multi-Head attention mechanism, and the BERT algorithm stacks multiple Transfromer models and adjusts the bidirectional Transformer in all layers to pre-train the bidirectional deep representation, and the pre-trained BERT model can be fine-tuned through an additional output layer, which has wider applicability and does not need to do more repetitive model training work.

[0056] In this embodiment, the pre-trained BERT algorithm model is used, the entire sample data without covering ([MASK]) is input into the BERT algorithm model, so that the model predicts the position of each word in the sample data, and generates the replacement word of each word. When generating the replacement word, some opposite words suitable for the current context may be generated, so the opposite word filtering operation can be performed first, and then the set of filtered replacement words is used as the synonym set of each word, wherein one word corresponds to one synonym set. Then, the BERT algorithm model outputs a score at the same time as outputting each replacement word, which is the semantic score of the replacement word. The semantic score represents the semantic closeness between the replacement word (or synonym) and the original word. Of course, in a specific application scenario, after the opposite word filtering operation is completed, the top preset number of replacement words can be selected as the synonym set of the corresponding word according to the semantic score of each word, and the replacement words with low semantic scores, which have little effect on the final result of the scheme, are removed. At the same time, this can reduce the subsequent calculation cost and improve the calculation efficiency.

[0057] In step 103, one of the words is selected as a replacement word based on the sorting order, and the attack score of each synonym of the replacement word is calculated based on the synonym set of the replacement word and the semantic score of each synonym.

[0058] In this step, a replacement word is selected, which can be the first word selected as the current replacement word according to the sorting order, for example, the words arranged in descending order of importance score, and the first word with the highest importance score is selected as the current replacement word. Of course, in a specific application scenario, the word can be selected according to the selection rule suitable for the specific application scenario. Then, the synonym set of the replacement word and the semantic score of each synonym in the synonym set are determined to calculate the attack score of each synonym of the replacement word. The physical meaning of the attack score is the ability of the synonym to attack the semantic classifier, that is, the ability or probability of causing the semantic classifier to misjudge the original sentence, and the higher the ability or probability represents the stronger the attack of the synonym on the sample data.

[0059] In some embodiments, the attack score of each synonym of the replacement word can be calculated as follows:

[0060]

[0061] wherein, represents the attack score of the i-th word w i i represents the i-th word w i ​​a synonym of the ith word w i the semantic score of w the initial score of w i a synonym of the ith word w i the initial score of w F(w1,...,w i-1 ,w i ,w i+1 ,...,w N ) represents the score of the initial data based on the output of the semantic classifier F, and F(w1,...,w i-1 ,w i ',w i+1 ,...,w N ) represents the score of the initial data after replacing the ith word w i with a synonym w i '.

[0062] In this embodiment, the semantic score is combined in the attack score, so that the semantic score becomes part of the attack score, so as to improve the semantic of the selected synonym while meeting the attack.

[0063] Step 104: selecting the synonym with the highest attack score, replacing the word to be replaced in the initial data to generate test data.

[0064] In this step, according to step 103, after calculating the attack score of each synonym of the word to be replaced, the synonym with the highest attack score is selected, and the word to be replaced in the initial data is replaced to generate test data. The test data is compared with the current initial data, and only the word to be replaced is replaced, and the rest is the same.

[0065] Step 105: judging whether the test data can complete the attack behavior on the sample data, if yes, the test data is used as the adversarial sample of the sample data.

[0066] In this step, it is judged whether the current data to be tested can complete the attack on the sample data. The adversarial sample is generally used for defense training of some judgment and recognition neural network. Taking the recognition neural network as an example, the attacker who wants to attack the recognition neural network will make the recognition neural network deviate from semantic understanding, for example, the recognition neural network recognizes the original text as "what is your name", which is a sentence asking the user's name attribute. After the recognition neural network recognizes the adversarial sample of the original text, it will be identified as "are you happy today" and other errors, which will eventually lead to recognition failure or other directions. The difference between the adversarial sample and the original text may only replace a few words. In this embodiment, the data to be tested obtained in step 104 and the original sample data are input into the corresponding recognition neural network and other neural networks that need to be trained. If the neural network has recognition error, it is considered that the data to be tested has completed the attack on the sample data, which can be used as the adversarial sample of the sample data.

[0067] Finally, after obtaining the adversarial sample, the adversarial sample can also be output. It can be used to store, display, use or reprocess the adversarial sample. According to different application scenarios and implementation needs, the output mode of the adversarial sample can be flexibly selected.

[0068] For example, for the application scenario of the method of the embodiment executed on a single device, the adversarial sample can be directly output on the display component (display, projector, etc.) of the current device in the form of display, so that the operator of the current device can directly see the content of the adversarial sample from the display component.

[0069] For example, for the application scenario of the method of the embodiment executed on a system composed of multiple devices, the adversarial sample can be sent to other preset devices in the system as receivers, i.e. synchronization terminals, through any data communication mode (wired connection, NFC, Bluetooth, wifi, cellular mobile network, etc.), so that the synchronization terminal can perform subsequent processing. Optionally, the synchronization terminal can be a preset server. The server is generally set in the cloud as a data processing and storage center, which can store and distribute the adversarial sample. The receiver of the distribution is a terminal device, and the holder or operator of the terminal device can be a sample data provider, an operator who performs neural network defense training, etc.

[0070] For example, for the application scenario of the method of the embodiment executed on a system composed of multiple devices, the adversarial sample can be directly sent to a preset terminal device through any data communication mode. The terminal device can be one or more of the devices listed in the foregoing paragraphs.

[0071] As can be seen from the above, the method for generating an adversarial sample according to the embodiments of the present application includes: obtaining sample data, taking the sample data as initial data, calculating the importance score of each word in the initial data based on a semantic classifier, and sorting the words based on the importance score; calculating the synonym set of each word and the semantic score of each synonym by using a bidirectional encoder representation from transformer-based algorithm; selecting a word as a word to be replaced based on the sorting order, calculating the attack score of each synonym of the word to be replaced based on the synonym set of the word to be replaced and the semantic score of each synonym, selecting the synonym with the highest attack score, replacing the word to be replaced in the initial data, and generating test data; determining whether the test data can complete the attack behavior on the sample data, and if so, taking the test data as the adversarial sample of the sample data. The bidirectional encoder representation from transformer-based algorithm is used to determine the synonym of each word in the initial data and the semantic property of the synonym, and then the attack property of each synonym is calculated based on the semantic property, the synonym with the highest attack property is selected to replace the word in the initial data, the newly generated test data is used to attack the sample data, and whether the attack is successful is used to determine whether the test data can become an adversarial sample. Further, a selection and replacement word index fusing semantic property is proposed to generate an adversarial sample with better semantic property, to strengthen the security training of the current neural network, and to improve the security of text recognition and text detection.

[0072] It should be noted that the method according to the embodiments of the present application can be executed by a single device, such as a computer or a server. The method according to the embodiments of the present application can also be applied to a distributed scenario, and be completed by multiple devices cooperating with each other. In this distributed scenario, one of the multiple devices can only execute one or more steps of the method according to the embodiments of the present application, and the multiple devices can interact with each other to complete the method.

[0073] It should be noted that the above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described above and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0074] In an optional example embodiment, the determining whether the to-be-tested data can complete the attack behavior on the sample data further includes: if not, selecting a next word of the current to-be-replaced word based on the sorting order to replace the to-be-replaced word, replacing the to-be-tested data with the initial data; re-generating the to-be-tested data, and based on the newly generated to-be-tested data, determining again whether the to-be-tested data can complete the attack behavior on the sample data.

[0075] In the embodiment, the to-be-tested data that replaces only one word is likely to be unsuccessful in the attack behavior on the sample data. Further, when the determination is unsuccessful, the next word of the current to-be-replaced word can be selected as a new to-be-replaced word according to the sorting of the previous words according to the importance scores. For example, the words are sorted in descending order according to the importance scores, the first word, i.e., the word with the highest importance score, is selected as the to-be-replaced word the first time, and when the determination is unsuccessful, the second word, i.e., the word with the second highest importance score, is selected as the to-be-replaced word. Meanwhile, the current to-be-tested data is taken as the initial data, i.e., the to-be-tested data that replaces one word is taken as the initial data. In this way, the attack scores of each synonym of the to-be-replaced word are calculated again based on the synonym set of the to-be-replaced word and the semantic scores of each synonym. The synonym with the highest attack score is selected to replace the to-be-replaced word in the initial data to generate to-be-tested data. Thus, the newly generated to-be-tested data replaces two words compared with the sample data, and the determination is performed again. If the determination is still unsuccessful, the operation of the embodiment can be repeated until the attack determination is successful, and the newly generated to-be-tested data is taken as the adversarial sample of the sample data.

[0076] In an optional example embodiment, the determining whether the to-be-tested data can complete the attack behavior on the sample data further includes: counting the number of times of re-generating the to-be-tested data, and if the number of times is greater than a preset threshold, it is considered that the to-be-tested data completes the attack behavior on the sample data.

[0077] In the embodiment, the number of words in the sample data is fixed. If the determination is performed repeatedly and cannot pass, the calculation may be wrong due to the reasons such as that all the words are selected and the to-be-replaced word cannot be found. Thus, a threshold can be set to prevent the number of loops from being too large or exceeding the expectation of an operator. In the embodiment, the number of times of re-generating the to-be-tested data can be understood as the number of loops or the number of times of re-generating the to-be-replaced word and the initial data.

[0078] In an optional example embodiment, the calculating the importance score of each word in the initial data based on the semantic classifier specifically includes:

[0079] Cw i =F(w1, ..., w) i-1 w i w i+1 , ..., w N )-F(w1,....,w i-1 w i+1 , ..., w N )

[0080] Among them, w i Let N represent the i-th word, N be the total number of words in the initial data, and Cw i F(w1,...,w) represents the importance score of the i-th word. i-1 ,w i ,w i+1 ,...,w N F(w1,....,w) represents the score of the initial data based on the output of the semantic classifier F. i-1 ,w i+1 ,...,w N ) represents the initial data based on the output of the semantic classifier F, after removing the i-th word w. i The score after that.

[0081] In an optional exemplary embodiment, calculating the aggression score for each synonym of the word to be replaced specifically involves:

[0082]

[0083] in, w represents the i-th word i A synonym w i ′'s offensive score, w represents the i-th word i A synonym w i The semantic score of ′, where β is a preset hyperparameter. w represents the i-th word i A synonym w i The initial score of ′ F(w1,...,w i-1 ,w i ,w i+1 ,...,w n F(w1,....,w) represents the score of the initial data based on the output of the semantic classifier F. i-1 ,w i ′,w i+1 ,...,w N) represents the initial data based on the semantic classifier F output of the i-th word w i replaced with synonyms w i score after.

[0084] In an optional example embodiment, the transformer-based bidirectional encoding representation algorithm is specifically a BERT algorithm; and the calculation of the synonym set of each word by using the transformer-based bidirectional encoding representation algorithm comprises: inputting the sample data into the BERT algorithm, so that the BERT algorithm performs synonym prediction on each word of the sample data, generates a plurality of word group sets corresponding to each word, performs antonym filtering on the word group sets, and takes the filtered word group sets as the synonym set.

[0085] In this embodiment, the pre-trained BERT algorithm model is used, the entire sample data without masking ([MASK]) is input into the BERT algorithm model, so that the model predicts the position of each word in the sample data, and generates replacement words for each word. When generating replacement words, some antonyms suitable for the current context may be generated, so the antonym filtering operation can be performed first, and then the set of replacement words filtered of antonyms is taken as the synonym set of each word, wherein one word corresponds to one synonym set. Then, the BERT algorithm model outputs a score at the same time as outputting each replacement word, which is the semantic score of the replacement word (or synonym). The semantic score represents the semantic closeness between the replacement word (or synonym) and the original word. Of course, in a specific application scenario, after the antonym filtering operation is completed, the top preset number of replacement words can be selected as the synonym set of the corresponding word according to the semantic score of each word, and the replacement words with lower semantic scores, which have less impact on the final result of the scheme, can be removed. At the same time, this can reduce the subsequent calculation cost and improve the calculation efficiency. The synonym dictionary can be used to filter out the antonyms to complete the antonym filtering operation.

[0086] In an optional example embodiment, the words are sorted based on the importance scores, specifically in descending order according to the importance scores; and one word is selected as the word to be replaced based on the sorting order, specifically the first word in the descending order is selected as the word to be replaced.

[0087] Based on the same concept, the present application also provides an adversarial sample generation device corresponding to the method of any of the above embodiments.

[0088] Reference Figure 2 , the adversarial sample generation device comprises:

[0089] The acquisition module 210 is configured to acquire sample data, take the sample data as initial data, calculate an importance score of each word in the initial data based on a semantic classifier, and sort the words based on the importance score.

[0090] The synonym module 220 is configured to calculate a synonym set of each word and a semantic score of each synonym by using a bidirectional encoder representations from transformer algorithm.

[0091] The calculation module 230 is configured to select one of the words as a word to be replaced based on the sorting order, and calculate an attack score of each synonym of the word to be replaced based on the synonym set of the word to be replaced and the semantic score of each synonym.

[0092] The generation module 240 is configured to select the synonym with the highest attack score, replace the word to be replaced in the initial data, and generate test data.

[0093] The judgment module 250 is configured to determine whether the test data can complete an attack behavior on the sample data, and if yes, take the test data as an adversarial sample of the sample data.

[0094] For the convenience of description, the above apparatus is described as various modules in terms of functions. Of course, the functions of the modules can be implemented in one or more software and / or hardware in the implementation of the embodiments of the present application.

[0095] The apparatus of the above embodiments is used to implement the corresponding adversarial sample generation method in the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described here.

[0096] In an optional example embodiment, the judgment module 250 is further configured to:

[0097] If no, replace the word to be replaced with the next word of the word to be replaced based on the sorting order, and replace the initial data with the test data;

[0098] Regenerate the test data, and determine again whether the test data can complete the attack behavior on the sample data based on the newly generated test data.

[0099] In an optional example embodiment, the judgment module 250 is further configured to:

[0100] Count the number of times of regenerating the test data, and if the number of times is greater than a preset threshold, consider that the test data completes the attack behavior on the sample data.

[0101] In an optional exemplary embodiment, the acquisition module 210 calculates the importance score of each word in the initial data based on a semantic classifier, specifically as follows:

[0102] Cw i =F(w1, ..., w) i-1 w i w i+1 , ..., w N )-F(w1,....,w i-1 w i+1 , ..., w N )

[0103] Among them, w i Let N represent the i-th word, N be the total number of words in the initial data, and Cw i F(w1,...,w) represents the importance score of the i-th word. i-1 ,w i ,w i+1 ,...,w N F(w1,....,w) represents the score of the initial data based on the output of the semantic classifier F. i-1 ,w i+1 ,...,w N ) represents the initial data based on the output of the semantic classifier F, after removing the i-th word w. i The score after that.

[0104] In an optional exemplary embodiment, the calculation module 230 calculates the aggression score of each synonym of the word to be replaced, specifically:

[0105]

[0106] in, w represents the i-th word i A synonym w i ′'s offensive score, w represents the i-th word i A synonym w i The semantic score of ′, where β is a preset hyperparameter. w represents the i-th word i A synonym w i The initial score of ′ F(w1,...,w i-1 ,w i ,w i+1 ,...,w N F(w1,....,w) represents the score of the initial data based on the output of the semantic classifier F.i-1 ,w i ′,w i+1 ,...,w N ) represents the score after the i-th word w i is replaced by a synonym w i ′ based on the initial data output by the semantic classifier F.

[0107] In an optional example embodiment, the transformer-based bidirectional encoding representation algorithm is BERT algorithm.

[0108] The synonym module 220 is further configured to:

[0109] input the sample data into the BERT algorithm, so that the BERT algorithm performs synonym prediction on each word of the sample data, generates a plurality of word group sets corresponding to each word one by one, performs antonym filtering on the word group sets, and takes the filtered word group sets as the synonym set.

[0110] In an optional example embodiment, the obtaining module 210 sorts the words based on the importance scores, specifically, arranges the words in descending order according to the importance scores.

[0111] The computing module 230 selects one word as a word to be replaced based on the sorting order, specifically:

[0112] According to the words arranged in descending order, the first word is selected as the word to be replaced.

[0113] Based on the same concept, the present application also provides an electronic device corresponding to the method of any of the above embodiments, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to realize the method of any of the above embodiments.

[0114] Figure 3 A more specific hardware structure of an electronic device is shown, which can include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040 and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030 and the communication interface 1040 are connected to each other through the bus 1050 for communication within the device.

[0115] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing relevant programs to implement the technical solutions provided by the embodiments of the present specification.

[0116] The memory 1020 can be implemented by a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and called and executed by the processor 1010.

[0117] The input / output interface 1030 is configured to connect input / output modules to implement information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input devices can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices can include a display, a speaker, a vibrator, an indicator light, etc.

[0118] The communication interface 1040 is configured to connect a communication module (not shown in the figure) to implement the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or through a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0119] The bus 1050 includes a channel for transmitting information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0120] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only include the components necessary to implement the solutions of the embodiments of the present specification, and does not have to include all the components shown in the figure.

[0121] The electronic device of the above embodiment is used to implement the corresponding adversarial sample generation method in any of the preceding embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here again.

[0122] Based on the same concept, corresponding to the method of any of the above embodiments, the present application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the adversarial sample generation method according to any of the above embodiments.

[0123] The computer-readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0124] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to perform the adversarial sample generation method according to any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which are not described here again.

[0125] Those skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the present application (including claims) is limited to these examples; under the idea of the present application, the above embodiments or technical features in different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the embodiments of the present application as described above. In order to be brief, they are not provided in detail.

[0126] Additionally, to simplify the description and discussion, and so as not to obscure the embodiments of the application being presented, the well-known functions or constructions of integrated circuit (IC) chips and other components can or can not be shown in the figures and will be omitted as not to unnecessarily obscure the embodiments of the application being presented. Moreover, the devices can be shown in block diagram form in order to avoid unnecessary obscurity of the present embodiments, and this also acknowledges the fact that the details in regard to the implementation of such block diagram devices are highly dependent on the platform within which the present embodiments are to be implemented (i.e., such details should be well within the purview of one of ordinary skill in the art). Where specific details are set forth in order to describe an illustrative embodiment of the application, it will be apparent to one of ordinary skill in the art that the embodiments of the application can be practiced without, or with variation of, these specific details. Thus, the description is to be considered as illustrative only and not restrictive in nature.

[0127] While the present application has been described with respect to a limited number of embodiments, it will be apparent to those skilled in the art that many alternate, modifications, and variations will be suggested by the foregoing description and that the embodiments of the present application should be considered in a descriptive sense only and not a restrictive sense. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0128] It is therefore intended that the present application cover all such alternatives, modifications, and variations as fall within the broad scope of the appended claims. Accordingly, any and all such modifications, variations, or equivalents that fall within the scope of the present application should be considered within its scope.

Claims

1. An adversarial sample generation method, characterized in that, The method comprises the following steps: obtaining sample data, taking the sample data as initial data, calculating the importance score of each word in the initial data based on a semantic classifier, and sorting the words based on the importance score; calculating the synonym set of each word and the semantic score of each synonym based on the transformer-based bidirectional encoding representation algorithm; selecting one of the words as a word to be replaced based on the sorting order, calculating the attack score of each synonym of the word to be replaced based on the synonym set of the word to be replaced and the semantic score of each synonym, and selecting the synonym with the highest attack score; selecting the synonym with the highest attack score, replacing the word to be replaced in the initial data to generate test data; determining whether the test data can complete the attack behavior on the sample data, and if so, taking the test data as the adversarial sample of the sample data; The method further comprises the following steps: wherein, denotes the i-th word, is the total number of words in the initial data, denotes the importance score of the i-th word, denotes the score of the initial data output by the semantic classifier F, denotes the score of the initial data output by the semantic classifier F with the i-th word removed, denotes the score of the initial data output by the semantic classifier F with the i-th word removed. calculating the attack score of each synonym of the word to be replaced, specifically: wherein, represents an i-th word one synonym of the i-th word, represents an i-th word one synonym of the i-th word, is a preset hyperparameter, represents an initial score of an i-th word one synonym of the i-th word, , represents a score based on the initial data output by the semantic classifier F, represents a score after replacing the i-th word with the synonym based on the initial data output by the semantic classifier F.

2. The method of claim 1, wherein, The method further comprises the following steps: if not, replacing the word to be replaced with the next word in the sorting order, replacing the initial data with the test data; regenerating the test data, and determining whether the test data can complete the attack behavior on the sample data based on the newly generated test data.

3. The method of claim 2, wherein, The method further comprises the following steps: If the number of times of regenerating the test data is greater than a preset threshold, it is determined that the test data completes the attack behavior on the sample data.

4. The method of claim 1, wherein, The transformer-based bidirectional encoding representation algorithm is specifically a BERT algorithm; The method further comprises the following steps: inputting the sample data into the BERT algorithm to enable the BERT algorithm to perform synonym prediction on each word of the sample data, generating a plurality of word group sets corresponding to each word, performing antonym filtering on the word group sets, and taking the filtered word group sets as the synonym set.

5. The method of claim 1, wherein, The method further comprises the following steps: sorting the words based on the importance score, specifically, arranging the words in descending order according to the importance score; The method further comprises the following steps:

6. An adversarial sample generation apparatus, comprising: selecting the first word as the word to be replaced according to the words arranged in descending order. The method comprises the following steps: an obtaining module, configured to obtain sample data, take the sample data as initial data, calculate the importance score of each word in the initial data based on a semantic classifier, and sort the words based on the importance score; a synonym module, configured to calculate the synonym set of each word and the semantic score of each synonym based on a transformer-based bidirectional encoding representation algorithm; The computing module is configured to select one of the words as a word to be replaced based on the sorting order, and calculate an attack score of each synonym of the word to be replaced based on the synonym set of the word to be replaced and the semantic score of each synonym; The generating module is configured to select the synonym with the highest attack score, replace the word to be replaced in the initial data, and generate test data; The judging module is configured to judge whether the test data can complete an attack behavior on the sample data, and if yes, take the test data as an adversarial sample of the sample data. The semantic classifier is configured to calculate an importance score of each word in the initial data, and the calculation includes: wherein, represents the i-th word, is the total number of words in the initial data, represents the importance score of the i-th word, represents the score of the initial data based on the output of the semantic classifier F, represents the score of the initial data based on the output of the semantic classifier F after removing the i-th word, represents the score of the initial data based on the output of the semantic classifier F after removing the i-th word. The computing module is configured to calculate an attack score of each synonym of the word to be replaced, and the calculation includes: wherein, represents an i-th word one synonym of the i-th word, represents an i-th word one synonym of the i-th word, is a preset hyperparameter, represents an initial score of an i-th word one synonym of the i-th word, , represents a score based on the initial data output by the semantic classifier F, represents a score after replacing the i-th word with a synonym based on the initial data output by the semantic classifier F.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium, comprising: The non-transitory computer readable storage medium stores computer instructions for causing the computer to implement the method in any one of claims 1 to 5.

Images