Adversarial sample defense method and device based on synonym embedding, equipment and storage medium

Abstract

The invention provides an adversarial sample defense method and device based on synonym embedding, equipment and a storage medium, and relates to the technical field of software security. The method adopts a mode of firstly detecting and then restoring, and comprises the following steps of: firstly, obtaining an antagonism score through probability distribution difference of a # imgabs0 # layer, setting a dynamic disturbance rate according to the antagonism score, and adopting different disturbance rates for a clean sample and a potential confrontation sample, so as to reduce interference on the clean sample as much as possible; then, embedding a synonym set average value to reduce the sensitivity of the model to a specific word in an input sample, which is expected to restore an original label; and finally, adopting a soft voting mode for a plurality of results to relieve the randomness brought by synonym embedding, and selecting a classification label finally output by the model. The invention designs an effective defense method which is used for detecting and defending countermeasure attacks and improving the robustness of the countermeasure attacks, and aims at improving the system stability and ensuring the user experience.

Description

Technical Field

[0001] The present invention relates to the field of software security technology, and in particular to an adversarial sample defense method, device, equipment and storage medium based on synonym embedding. Background Art

[0002] In recent years, the rapid development of deep neural networks has been widely used in the tourism industry. Through technologies such as intelligent recommendation systems, virtual tours, and automatic customer service, tourists can get more personalized and convenient services. For example, travel platforms can recommend suitable travel routes and hotels based on users' historical behaviors and preferences, and intelligent assistants can answer tourists' various questions in real time. This not only improves user experience, but also promotes the digital transformation of the tourism industry.

[0003] However, studies have shown that the local linearity and high-dimensional data processing characteristics of deep neural networks make them vulnerable to adversarial sample attacks in terms of security. These attacks may cause the system to recommend incorrect travel information, wrong prices or confusing itineraries, affecting user experience and even causing economic losses. For example, malicious adversarial samples may mislead the recommendation system to provide irrelevant travel products or services through slight changes in input.

[0004] Since the concept of adversarial examples was proposed, many studies have focused on the field of computer vision to explore how to generate and defend against these samples. However, in tourism-related natural language processing (NLP) applications, such as intelligent customer service and review analysis, there are still relatively few related studies. The robustness and security issues of NLP models are equally critical and require more attention and research to ensure that the intelligent systems in the tourism industry can remain stable and reliable under adversarial attacks. By strengthening the research on adversarial examples, the tourism industry can improve the security of the system and user trust, and promote the further development of digital services.

[0005] The emergence of adversarial examples has brought a series of serious problems to DNNs, posing severe challenges to their reliability and security. Based on the analysis of the current status, there are still some shortcomings in the current text-oriented adversarial example defense methods: how to strike a balance between the defense effect and the classification accuracy of clean samples, while taking into account the training cost of the defense method.

[0006] Current defense methods for text adversarial samples mainly focus on known attack methods. The limitation of this defense method is that it can only defend against specific data sets and attack methods, and its defense effect cannot be guaranteed for newly emerging adversarial samples. Although some detection-based methods can resist unknown attacks to a certain extent, some of them adopt the strategy of directly discarding adversarial samples, which will seriously reduce the model's classification accuracy for clean samples and user experience. At the same time, methods based on adversarial training may increase training costs, limiting the scalability and practicality of the method. Summary of the invention

[0007] Purpose of the invention: To propose an adversarial sample defense method, apparatus, device and storage medium based on synonym embedding, which adopts the method of adding additional components to the data without the need to retrain the model, and can show good performance in defending against different types of text adversarial sample generation methods, thereby effectively solving the above-mentioned problems existing in the prior art.

[0008] In a first aspect of the present invention, a method for defending against adversarial samples based on synonym embedding is proposed, comprising the following steps:

[0009] Get the text data set to be tested and its corresponding label information;

[0010] Perform data cleaning on the data in the text data set to obtain a cleaned text data set;

[0011] Generate a virtual sample, calculate an adversarial score based on the virtual sample, and set a dynamic perturbation rate;

[0012] Extracting a number of words from the cleaned text data set and performing an importance ranking operation on the words;

[0013] Extracting multiple synonyms according to the word importance ranking result and the dynamic disturbance rate, and embedding the multiple synonyms into samples to obtain multiple replacement samples;

[0014] For multiple replacement samples, soft voting is adopted and the voting result is used as the final output category.

[0015] In a further embodiment of the first aspect, the process of generating a virtual sample specifically includes:

[0016] Based on an NLP model, the embedding layer of the NLP model maps each word in the input to an n-dimensional vector. The input sequence consists of m n-dimensional vectors, and the matrix W with dimension n×m is pre As the original text x;

[0017] Direction Matrix W pre Add a random noise E of the same shape and size r , forming a new vector matrix W new , the vector matrix W new As a virtual sample x ′ , this process expression is as follows:

[0018] W new =W pre +μ·E r

[0019] Where μ is the length that limits the size of the noise disturbance.

[0020] In a further embodiment of the first aspect, the KL divergence is used to calculate the original text x and the virtual sample x ′ The distance D(f(x),f(x) between the two output probability distributions ′ )):

[0021] D(f(x),f(x ′ ))=max{KL(f(x),f(x ′ )),KL(f(x ′ ),f(x))}

[0022] Where f(x) represents the output probability distribution of the original text x; f(x ′ ) represents a virtual sample x ′ The output probability distribution of; KL(*) means calculating the KL divergence between two output probability distributions; max{*} means taking the maximum KL;

[0023] Adversarial score adv Calculated according to the following formula:

[0024]

[0025] In the formula, D(f(x),f(x ′ )) is the original text x and the virtual sample x ′ The distance between the two output probability distributions in ;

[0026] The calculation formula of dynamic disturbance rate P is as follows:

[0027] P=w+score adv

[0028] Where w is the basic disturbance rate.

[0029] In a further embodiment of the first aspect, before the importance ranking operation is performed on the words, the method further includes:

[0030] Calculate the importance of the word to the final prediction:

[0031] The text formed by deleting the word at the i-th position from the original text is input into the NLP model to be tested. The confidence output by the NLP model to be tested is compared with the confidence of the original text. The importance score Q of the word at the current position to the final prediction is obtained based on the deviation of the confidence. i ;

[0032] Calculate the degree to which the final prediction result is changed after being replaced by a synonym:

[0033] According to the synonym database, the word at the i-th position is replaced with a synonym of the word; at the same time, the replaced text is mapped using local sensitive hashing (LSH) to obtain a hash value. For the texts with the same hash value after mapping, one of them is selected and input into the NLP model to be tested. The synonym replacement score P of the word at the current position is calculated based on the confidence output by the NLP model to be tested and the confidence deviation of the original text. i ;

[0034] Calculate the correlation between the current position word and other position words:

[0035] The original text is input into the NLP model to be tested and mapped into multiple subspaces. Each subspace is transformed by different linear transformations, and then an independent self-attention calculation is performed to obtain a different attention weight matrix. The attention weight matrices obtained in all spaces are added together and averaged to obtain the final attention weight matrix M. The attention weight matrix M is regarded as the degree of association between each position and other positions.

[0036] The sum of each row of elements in the attention weight matrix M is recorded in W, and the correlation score between each position and other positions is obtained after normalization, and the correlation degree score W between the i-th position and other positions is obtained. i .

[0037] In a further embodiment of the first aspect, the word importance score of the i-th position is i The calculation formula is as follows:

[0038] score i =Q i ×P i ×W i

[0039] In the formula, Q i represents the importance score of the word at the ith position to the final prediction; P i represents the synonym replacement score of the word at the i-th position; W i Indicates the correlation score between the i-th position and other positions;

[0040] score i The larger the value, the more important the i-th position is to the input, and the greater the impact on the output of the NLP model;

[0041] After obtaining the word importance score for each position, sort them from largest to smallest.

[0042] In a further embodiment of the first aspect, a plurality of synonyms are extracted according to the word importance ranking result and the dynamic disturbance rate, and a plurality of replacement samples are obtained after the plurality of synonyms are embedded into the sample, specifically comprising:

[0043] For a text sequence X = {x 1 ,x 2 ,…,x n}, and use WordNet to build a synonym set for each of the K words in the top K word importance ranking.

[0044] X and X are transformed by a universal sentence encoder idx,pos Encoded into two n-dimensional vectors p and q:

[0045] p=Encoder(X),q=Encoder(X idx,pos )

[0046] Where X idx,pos It represents the sample obtained by replacing the word at position idx in the original text with the synonym at position pos in the synonym set S;

[0047] Calculate the cosine similarity between p and q as X and X idx,pos The approximate value of the semantic similarity between idx,pos ):

[0048]

[0049] The synonyms in the synonym set S are converted into vector representations, and then the cosine similarity between the synonyms is calculated. According to the set similarity threshold ε, synonyms with the same meaning are screened out;

[0050] The word embeddings in the synonym set are averaged to obtain an embedding vector that represents the overall semantics of the synonym set; this average embedding vector is applied to the embedding of the key word, and the expression is as follows:

[0051]

[0052] In the formula, syn i represents the synonym set associated with the i-th word; m is the number of synonyms that match the meaning of the original token in WordNet.

[0053] According to a second aspect of the present invention, an adversarial sample defense device is provided, which includes an acquisition unit, a data processing unit, a first execution unit, a second execution unit, a third execution unit, and an output unit.

[0054] The acquisition unit is used to obtain the text data set to be tested and its corresponding label information. The data processing unit is used to clean the data in the text data set to obtain a cleaned text data set. The first execution unit is used to generate a virtual sample, calculate the adversarial score based on the virtual sample, and set the dynamic perturbation rate. The second execution unit is used to extract a number of words from the cleaned text data set and perform an importance sorting operation on the words. The third execution unit is used to extract a plurality of synonyms based on the word importance sorting result and the dynamic perturbation rate, and obtain a plurality of replacement samples after embedding the plurality of synonyms into the sample. The output unit is used to adopt a soft voting method for the plurality of replacement samples, and use the voting result as the final output category.

[0055] According to a third aspect of the present invention, an electronic device is proposed, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the adversarial sample defense method based on synonym embedding as described in the first aspect is implemented.

[0056] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, which stores at least one executable instruction. When the executable instruction is executed on an electronic device, the electronic device executes the adversarial sample defense method based on synonym embedding as described in the first aspect.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] (1) The adversarial score is calculated by generating virtual samples, thereby setting a dynamic perturbation rate. The dynamic perturbation rate method can more flexibly adjust the amplitude of the perturbation in the case of adversarial samples. For clean samples, a smaller perturbation rate can be selected to minimize the negative impact on the accuracy of the model. For potential adversarial samples, their aggressiveness to the model can be judged based on the results of risk assessment, and the perturbation rate can be adjusted accordingly. If the risk of adversarial samples is high, a larger perturbation rate can be set to ensure that the model can correctly classify these samples.

[0059] (2) Generate a rich synonym library through synonym extraction for synonym embedding. In synonym extraction, this method will build a synonym set for each word at a key position. This is done to enrich the information at the key position in the input sequence to provide more semantic variants and expressions. In order to better extract the meaning of words and their synonyms, this method uses the SenseBERT model, which is a semantic representation model based on deep learning that can model and capture the different meanings of words. SenseBERT converts the synonyms in the synonym set S into vector representations, and then calculates the cosine similarity between synonyms to screen out synonyms with the same meaning. By combining WordNet and SenseBERT, it is possible to find and build a synonym set related to each word, while better retaining the semantic information of the word, enriching the expression of the text, and providing more accurate and rich semantic representations for subsequent processing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flowchart of an adversarial sample defense method based on synonym embedding in an embodiment of the present invention.

[0061] Figure 2 It is a diagram showing the effect before and after text data cleaning in an embodiment of the present invention.

[0062] Figure 3 It is a partial excerpt of text stop words in an embodiment of the present invention.

[0063] Figure 4 2 is an example diagram of word scoring in an embodiment of the present invention.

[0064] Figure 5 This is an abstract example diagram of synonym embedding and soft voting in an embodiment of the present invention.

[0065] Figure 6 2 is an example diagram of soft voting in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features known in the art are not described.

[0067] This embodiment discloses an adversarial sample defense method based on synonym embedding. The method adopts a detection-first-then-restore approach. First, the adversarial score is obtained by the difference in the probability distribution of the Softmax layer. The dynamic perturbation rate is set according to the adversarial score, and different perturbation rates are adopted for clean samples and potential adversarial samples to minimize the interference with clean samples. Then, the average value of the synonym set is embedded to reduce the sensitivity of the model to specific words in the input sample, hoping to restore the original label. Finally, a soft voting method is used for multiple results to alleviate the randomness brought by synonym embedding and select the classification label finally output by the model. The main purpose can be divided into two points:

[0068] 1. Design an effective defense method to detect and resist adversarial attacks and improve their robustness, aiming to improve system stability and ensure user experience.

[0069] 2. While ensuring the defensive effect, maintain the model's classification accuracy for clean samples, aiming to improve the security of the system and user trust, and promote the further development of digital services.

[0070] The process diagram is as follows Figure 1 As shown, the specific technical implementation steps are as follows:

[0071] Step 1: Get the text data set to be tested (including training set and test set) and its corresponding label information. The specific steps are as follows:

[0072] Step 11: Use the Yelp dataset download URL to download the dataset and its corresponding labels;

[0073] Step 12: Extract the required data from the corresponding compressed file and save it locally.

[0074] Step 2: Preprocess the data in the text dataset. The specific steps are as follows:

[0075] Step 21: First, perform data cleaning. Data cleaning mainly involves deleting invalid characters and extra spaces in text data, removing HTML tags, and converting to lowercase. Invalid characters refer to characters in text data that have no effect on the semantics of the sentence. Invalid characters are matched and replaced by regular expressions, retaining the text such as letters and numbers and punctuation marks that can express emotions, and replacing other characters and symbols with spaces. The specific matching rules are shown in Table 1.

[0076] Table 1 Text data cleaning rules table

[0077] Serial number Regular Expressions illustrate 1 "[^A-Za-z0-9(),!?\'\`]"→"" Remove characters other than A-Za-z0-9(),!?'` 2 "\'d"→"\'d" Add a space before 'd 3 "\'s"→"\'s" Add a space before 's 4 "n\'t"→"\n't" Add a space before n't 5 "\'ve"→"\'ve" Add a space before 've 6 "？"→"？" Add a space before the ? 7 "\s{2,}"→"" Merge consecutive spaces into one space 8 ","→"," Add a space before , 9 .strip().lower() Remove spaces before and after the text and convert all to lowercase 10 "！"→"！" Add a space before the exclamation mark

[0078] At the same time, redundant spaces will also be removed, because they will make the text data format inconsistent, which is not conducive to subsequent operations such as word segmentation and feature extraction. HTML tags are removed because there are escapes in the data crawling process, and these tags are meaningless for text analysis. Converting to lowercase is to avoid confusion caused by uppercase and lowercase letters and improve consistency. Figure 2 As shown in the figure, (a) is the original text data, and (b) is the effect diagram after data cleaning.

[0079] Step 22: After data cleaning, stop word removal is performed. Stop words refer to common words that appear frequently in text but usually contribute little to text analysis tasks, such as "the", "and", "is", etc. These words are ubiquitous in text, but they appear frequently and have little impact on the feature representation of the text, so they can be regarded as noise in some text analysis tasks. Removing stop words can reduce data dimensions, improve computational efficiency, and highlight important features of the text. Therefore, this method adds a stop word removal operation in the data preprocessing stage. Some stop word excerpts are shown in the figure below. Figure 3 shown.

[0080] Step 3: Generate virtual samples to calculate the adversarial score and set the dynamic perturbation rate. The specific steps are as follows:

[0081] Step 31: Calculate the adversarial score, which is mainly used to set the dynamic perturbation rate. The adversarial score in this paper is obtained by the inconsistency of the probability distribution change of the Softmax layer, and the distance between the two output probability distributions is calculated by using the KL divergence. Before calculation, it is necessary to obtain the virtual sample x ′ , the sample is obtained as follows: each word in the input is mapped to an n-dimensional vector in the embedding layer of the model. This input sequence (length m) will be composed of m n-dimensional vectors, that is, the matrix W pre The dimension is n×m, and then a random noise E of the same shape and size is added to this matrix r , a new vector matrix W will be formed new , that is, the virtual sample x ′ The process can be expressed as formula 1.1, where μ is the length that limits the size of the noise perturbation.

[0082] W new =W pre +μ·E r (1.1)

[0083] However, since the KL divergence is not symmetric, it is not a suitable distance. Therefore, the actual distance is calculated as the maximum KL of two possible combinations, as shown in formulas 1.2 and 1.3:

[0084]

[0085] D(f(x),f(x ′ ))=max{KL(f(x),f(x ′ )),KL(f(x ′ ),f(x))}(1.3)

[0086] Then we can calculate the distance D(f(x),f(x ′ )) performs threshold processing to classify text samples as adversarial or non-adversarial (this method does not do this, but only calculates the adversarial score). The detection algorithm of this method uses the maximum value instead of the minimum value to achieve better classification accuracy, because the maximum value is a more conservative estimate of the distance. The adversarial score of the above factors is recorded as score adv As shown in formula 1.4, this method can be used to calculate the score adv Controlled in the range of [0,1].

[0087]

[0088] Step 32: Dynamic perturbation rate setting. This method uses a dynamic perturbation rate to control the number of synonym embedding positions, which can more flexibly adjust the amplitude of the perturbation in the case of adversarial samples. For clean samples, a smaller perturbation rate can be selected to minimize the negative impact on the accuracy of the model. For potential adversarial samples, their aggressiveness to the model can be judged based on the results of risk assessment, and the perturbation rate can be adjusted accordingly. If the risk of adversarial samples is high, a larger perturbation rate can be set to ensure that the model can correctly classify these samples. The specific formula is shown in Formula 1.5, where w is the basic perturbation rate.

[0089] P=w+score adv (1.5)

[0090] Step 4: Sort the importance of words. The specific steps are as follows:

[0091] Step 41: The word importance score is calculated based on the following three factors:

[0092] (1) Importance of a word to the final prediction: The text formed by deleting the word at the i-th position from the original text is input into the deep neural network model to be tested. The confidence of the model output at this time is compared with the confidence of the original text model output. The importance score Q of the current word to the final prediction is obtained based on the deviation of the confidence. i .

[0093] (2) The degree of change in the final prediction result after being replaced by a synonym. According to the synonym database, the word at the i-th position is replaced with a synonym of the word. At the same time, in order to reduce the number of queries, the replaced text is mapped using Local Sensitive Hashing (LSH) to obtain a hash value. For the texts with the same hash value after mapping, one is selected and input into the deep neural network model to be tested. The synonym replacement score P of the word at the current position is calculated based on the confidence at this time and the confidence deviation of the original text. i ;

[0094] (3) The degree of association between the word at this position and the words at other positions. The original text is input into the deep neural network model and mapped into multiple subspaces. Each subspace is transformed by different linear transformations, and then an independent self-attention calculation is performed to obtain a different attention weight matrix. The attention weight matrices obtained in all spaces are added together and averaged to obtain the final attention weight matrix M, which is approximately regarded as the degree of association between each position and other positions. Then, the sum of each row element in the M matrix is ​​recorded in W, and the association score between each position and other positions is obtained after normalization. The degree of association score W between the i-th position and other positions can be obtained. i .

[0095] Finally, the score of the word at the i-th position is the product of the above three scores. The calculation formula 1.5 for the word score at the i-th position is as follows:

[0096] score i =Q i ×P i ×W i (1.5)

[0097] The score i The larger the value, the more important the i-th position is to the input, and the greater the impact on the output of the deep neural network model. After obtaining the word score for each position, they are sorted from large to small for subsequent selection of influential words.

[0098] Word scoring example Figure 4As shown. The LSH-based score corresponds to the degree of change in the final prediction result after being replaced by a synonym. Since the search space is huge and there will be multiple synonymous replacement words in the synonym vocabulary, if each replaced sample goes to the model to query the confidence change, this will greatly increase the number of queries, which is not in line with the application of deep neural network models in real scenarios. Therefore, LSH is used to reduce the number of queries by sampling queries on buckets as the query results of each bucket. At the same time, the attention-based score corresponds to the importance score of the word to the final prediction. In existing research, some researchers have proposed that deleting words to query the model confidence change is not reliable because it changes the structure of the original input. Therefore, this paper proposes a new score, namely a score based on association strength. This score does not require deleting or replacing words, but can be calculated by querying the model.

[0099] Step 5: Extract synonyms according to word importance ranking and perturbation rate, build corresponding synonym sets, and perform synonym embedding. The specific steps are as follows:

[0100] Step 51: Synonym extraction. This method will construct a synonym set for each word at a key position. This is done to enrich the information of the key position in the input sequence to provide more semantic variants and expressions.

[0101] First, for the text sequence X = {x 1 ,x 2 ,…,x n}, we can use WordNet to build a synonym set for each of the K words in the top K word importance ranking (K is obtained through the dynamic perturbation rate). Since words may have multiple parts of speech, in order to ensure that there are no grammatical errors after word replacement, we use NLTK to replace x idx Perform part-of-speech tagging (x idx is used for illustration purposes), select from WordNet idx A set S of synonyms with the same part of speech.

[0102] Secondly, in order to better determine the word w idx Related synonyms, this method is selected by semantic similarity. X and X are transformed into idx,pos Encoded into two n-dimensional vectors p and q, as shown in Formula 1.6, where X idx,pos It represents the sample obtained by replacing the word at position idx in the original text with the synonym at position pos in the synonym set S.

[0103] p=Encoder(X),q=Encoder(X idx,pos )(1.6)

[0104] Then calculate the cosine similarity between p and q as X and X idx,pos The approximate value of the semantic similarity between them. The specific expression is shown in formula 1.7:

[0105]

[0106] Finally, in order to better extract the meaning of words and their synonyms, this method uses the SenseBERT model, which is a semantic representation model based on deep learning, which can model and capture the different meanings of words. SenseBERT converts the synonyms in the synonym set S into vector representations, and then calculates the cosine similarity between synonyms. According to the set similarity threshold ε = 0.7, synonyms with the same meaning are screened out.

[0107] By combining WordNet and SenseBERT, we can find and build a synonym set related to each word, while better retaining the semantic information of the word, enriching the expression of the text, and providing more accurate and rich semantic representation for subsequent processing tasks. The specific expression is shown in formula 1.8, where m is the number of synonyms that match the original tag meaning in WordNet.

[0108] Syn(w idx )={syn 1 ,syn 2 ,…,syn m}(1.8)

[0109] Step 52: Average synonym embedding. This method uses the embeddings (i.e., word vectors) of words in the synonym set to improve the richness and accuracy of word representation. This method averages the embeddings of all words in the synonym set to obtain an average embedding vector, which is used to replace the embedding of each key token (key word).

[0110] The purpose of this method is to establish stronger semantic connections between words in the synonym set, comprehensively considering the semantic information of multiple words in the synonym set, rather than just being limited to the embedding of a single word. By averaging the word embeddings in the synonym set, an embedding vector that represents the overall semantics of the synonym set can be obtained. This average embedding vector is then applied to the embedding of the key words to obtain a richer and more accurate word representation, which helps improve the performance of subsequent processing tasks. The specific expression is shown in Formula 1.9:

[0111]

[0112] In order to better understand the operation of synonym embedding and classification label restoration, this paper provides Figure 5As an abstract example diagram, Figure 5 The observation can more intuitively understand the application of synonym embedding and classification label restoration. It should be noted that the dynamic perturbation rate setting is not in Figure 5 It is reflected in.

[0113] Step 6: Restore the classification labels and obtain the final output. The specific steps are as follows:

[0114] Step 61: In order to alleviate the randomness brought by synonym embedding, this method comprehensively considers the prediction information of multiple synonym embedded samples as the final prediction classification label of the model. For multiple samples after replacement, soft voting is adopted, and the voting results are used as the final output category. Among them, soft voting is a collective decision-making method that considers the consensus among multiple samples after replacement. By comprehensively considering these samples, a more reliable and accurate final result can be obtained. In soft voting, the prediction result of each sample is regarded as a vote, and the weights of different samples can be adjusted according to their confidence or other relevant metrics. By combining the voting results of multiple samples, a more stable final output with stronger predictive ability can be obtained.

[0115] An example of soft voting idea is shown below: Figure 6 As shown in the figure, the left side of the arrow represents the confidence distribution of a binary classification model for five inputs, which are (0.99, 0.01), (0.49, 0.51), etc. According to the soft voting idea, as shown on the right side of the arrow in the figure, we can see the process of adding the category confidences of these five input samples. The final confidence sum of category A is 0.616, which is greater than the confidence sum of category B, 0.384, so the final prediction result is determined to be category A.

[0116] In addition, the present embodiment also discloses an adversarial sample defense device, which includes an acquisition unit, a data processing unit, a first execution unit, a second execution unit, a third execution unit, and an output unit. The acquisition unit is used to acquire a text data set to be tested and its corresponding label information. The data processing unit is used to perform data cleaning on the data in the text data set to obtain a cleaned text data set. The first execution unit is used to generate a virtual sample, calculate the adversarial score based on the virtual sample, and set the dynamic perturbation rate. The second execution unit is used to extract a number of words from the cleaned text data set and perform an importance sorting operation on the words. The third execution unit is used to extract a plurality of synonyms based on the word importance sorting result and the dynamic perturbation rate, and obtain a plurality of replacement samples after embedding the plurality of synonyms into the sample. The output unit is used to adopt a soft voting method for the plurality of replacement samples, and use the voting result as the final output category. The adversarial sample defense device can automatically execute all the process steps of the adversarial sample defense method based on synonym embedding disclosed in the above embodiment, which will not be repeated here.

[0117] In addition, this embodiment also discloses an electronic device, which includes a processor, a memory, a communication interface and a communication bus. The processor, the memory and the communication interface communicate with each other through the communication bus. The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute all the process steps of the adversarial sample defense method based on synonym embedding disclosed in the above embodiment, which will not be repeated here.

[0118] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication may be performed through an input / output (I / O) interface. In addition, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0119] It should be noted that the above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0120] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0121] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A method for defending against adversarial samples based on synonym embedding, characterized in that: The steps include: Get the text data set to be tested and its corresponding label information; Perform data cleaning on the data in the text data set to obtain a cleaned text data set; Generate a virtual sample, calculate an adversarial score based on the virtual sample, and set a dynamic perturbation rate; Extracting a number of words from the cleaned text data set and performing an importance ranking operation on the words; Extracting multiple synonyms according to the word importance ranking result and the dynamic disturbance rate, and embedding the multiple synonyms into samples to obtain multiple replacement samples; For multiple replacement samples, soft voting is adopted and the voting result is used as the final output category.

2. The adversarial sample defense method based on synonym embedding according to claim 1 is characterized in that: The generating of the virtual sample specifically comprises: Based on an NLP model, the embedding layer of the NLP model maps each word in the input to an n-dimensional vector. The input sequence consists of m n-dimensional vectors, and the matrix W with dimension n×m is pre As the original text x; Direction Matrix W pre Add a random noise E of the same shape and size r , forming a new vector matrix W new , the vector matrix W new As a virtual sample x ′ , this process expression is as follows: W new =W pre +μ·E r Where μ is the length that limits the size of the noise disturbance.

3. The adversarial sample defense method based on synonym embedding according to claim 2 is characterized in that: Use KL divergence to calculate the original text x and virtual sample x ′ The distance D(f(x),f(x) between the two output probability distributions ′ )): D(f(x),f(x ′ ))=max{KL(f(x),f(x ′ )),KL(f(x ′ ),f(x))} Where f(x) represents the output probability distribution of the original text x; f(x ′ ) represents a virtual sample x ′ The output probability distribution of; KL(*) means calculating the KL divergence between two output probability distributions; max{*} means taking the maximum KL; Adversarial score adv Calculated according to the following formula: In the formula, D(f(x),f(x ′ )) is the original text x and the virtual sample x ′ The distance between the two output probability distributions in ; The calculation formula of dynamic disturbance rate P is as follows: P=w+score adv Where w is the basic disturbance rate.

4. The adversarial sample defense method based on synonym embedding according to claim 2 is characterized in that: Before sorting the words by importance, it also includes: Calculate the importance of the word to the final prediction: The text formed by deleting the word at the i-th position from the original text is input into the NLP model to be tested. The confidence output by the NLP model to be tested is compared with the confidence of the original text. The importance score Q of the word at the current position to the final prediction is obtained based on the deviation of the confidence. i ; Calculate the degree to which the final prediction result is changed after being replaced by a synonym: According to the synonym database, the word at the i-th position is replaced with a synonym of the word; at the same time, the replaced text is mapped using local sensitive hashing (LSH) to obtain a hash value. For the texts with the same hash value after mapping, one of them is selected and input into the NLP model to be tested. The synonym replacement score P of the word at the current position is calculated based on the confidence output by the NLP model to be tested and the confidence deviation of the original text. i .

5. The adversarial sample defense method based on synonym embedding according to claim 4 is characterized in that: Before sorting the importance of words, it also includes calculating the correlation between the current position word and the words in other positions: The original text is input into the NLP model to be tested and mapped into multiple subspaces. Each subspace is transformed by different linear transformations, and then an independent self-attention calculation is performed to obtain a different attention weight matrix. The attention weight matrices obtained in all spaces are added together and averaged to obtain the final attention weight matrix M. The attention weight matrix M is regarded as the degree of association between each position and other positions. The sum of each row of elements in the attention weight matrix M is recorded in W, and the correlation score between each position and other positions is obtained after normalization, and the correlation degree score W between the i-th position and other positions is obtained. i .

6. The adversarial sample defense method based on synonym embedding according to claim 5, characterized in that: The importance score of the word at the i-th position i The calculation formula is as follows: score i =Q i ×P i ×W i In the formula, Q i Represents the importance score of the word at the i-th position to the final prediction; P i Represents the synonym replacement score of the word at the i-th position; W i Indicates the correlation score between the i-th position and other positions; score i The larger the value, the more important the i-th position is to the input, and the greater the impact on the output of the NLP model; After obtaining the word importance score for each position, sort them from largest to smallest.

7. The adversarial sample defense method based on synonym embedding according to claim 6, characterized in that: The extracting of multiple synonyms according to the word importance ranking result and the dynamic disturbance rate, and embedding the multiple synonyms into samples to obtain multiple replacement samples specifically includes: For a text sequence X = {x1, x2, ..., x n }, and use WordNet to build a synonym set for each of the K words in the top K word importance ranking. X and X are transformed by a universal sentence encoder idx,pos Encoded into two n-dimensional vectors p and q: p=Encoder(X),q=Encoder(X idx,pox ) Where X idx,pos It represents the sample obtained by replacing the word at position idx in the original text with the synonym at position pos in the synonym set S; Calculate the cosine similarity between p and q as X and X idx,pos The approximate value of the semantic similarity between idx,pos ): The synonyms in the synonym set S are converted into vector representations, and then the cosine similarity between the synonyms is calculated. According to the set similarity threshold ε, synonyms with the same meaning are screened out; The word embeddings in the synonym set are averaged to obtain an embedding vector that represents the overall semantics of the synonym set; this average embedding vector is applied to the embedding of the key word, and the expression is as follows: In the formula, syn i represents the synonym set associated with the i-th word; m is the number of synonyms that match the meaning of the original token in WordNet.

8. An adversarial sample defense device, characterized in that: include: An acquisition unit, used to acquire a text data set to be tested and its corresponding label information; A data processing unit, used for performing data cleaning on the data in the text data set to obtain a cleaned text data set; A first execution unit, configured to generate a virtual sample, calculate an adversarial score according to the virtual sample, and set a dynamic perturbation rate; A second execution unit is used to extract a number of words from the cleaned text data set and perform an importance ranking operation on the words; A third execution unit is used to extract a plurality of synonyms according to the word importance ranking result and the dynamic disturbance rate, and to obtain a plurality of replacement samples after embedding the plurality of synonyms into the samples; The output unit is used to adopt a soft voting method for multiple replacement samples and use the voting result as the final output category.

9. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the adversarial sample defense method based on synonym embedding as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction, and when the executable instruction is executed on the electronic device, the electronic device executes the adversarial sample defense method based on synonym embedding as described in any one of claims 1 to 7.

Images