Adversarial text sample generation method, computer device and medium

Abstract

The present disclosure provides an adversarial text sample generation method, a computer device and a medium. The method comprises: constructing an atomic operation sequence for a to-be-operated text sample; operating the to-be-operated text sample using the atomic operation sequence, wherein in the operation process, the insertion operation and the deletion operation in the atomic operation sequence are unified into a replacement operation by using a blank word; selecting a candidate word of a to-be-operated word in the to-be-operated text sample from a plurality of candidate words according to a loss function gradient of the to-be-operated word in the to-be-operated text sample and word vector representations of the plurality of candidate words; and constructing an adversarial text sample of the to-be-operated text sample according to the selected candidate word. The present disclosure further improves the adversarial success rate of the machine learning model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and more specifically, to a method for generating adversarial text samples using a machine learning model, a computer device, and a medium. Background Technology

[0002] Currently, machine learning models are widely used in the field of artificial intelligence, especially in Natural Language Processing (NLC). Adversarial testing is frequently used in testing machine learning models. This involves inputting a text sample to be manipulated into the machine learning model, which is expected to output a target value. If the text sample is then slightly perturbed, such as replacing one or two synonyms, the original meaning is minimally affected and difficult for humans to discern, but the machine learning model produces an incorrect output—that is, an output that is not the target value—then the adversarial test is considered successful. Adversarial testing can be used to measure the training effectiveness of machine learning models and can generate adversarial text samples to supplement the training model, improving its robustness, security, and resolvability.

[0003] Most existing adversarial techniques employ atomic substitution operations, replacing a word in the original text sample with another synonym. This atomic substitution is repeated until the output of a machine learning model becomes a non-target value when the sample is input, at which point the adversarial effort is considered successful. For example, Ebrahimi et al., 2017; Wallace103 et al., 2019 utilized a gradient-based atomic substitution operation to replace words in the text sample with the word that maximizes the first-order approximation of the prediction loss function. Since each word substitution does not affect the number of words (length) in the text sample, it limits the exploration in the decision space. In other words, this word-only adversarial approach may not find the best-performing adversarial text sample, because the best-performing adversarial text sample may not have the same number of words as the original text sample. Summary of the Invention

[0004] In view of this, this disclosure aims to further improve the success rate of adversarial attacks against machine learning models.

[0005] To achieve this objective, according to one aspect of this disclosure, a method for generating adversarial text samples using a machine learning model is provided, comprising:

[0006] Construct an atomic operation sequence for the text sample to be operated on. The atomic operation sequence includes multiple atomic operations to be executed sequentially. The atomic operation includes one of the word replacement, insertion, and deletion operations.

[0007] The atomic operation sequence is used to operate on the text sample to be operated on. During the operation, the insertion, deletion and replacement operations in the atomic operation sequence are unified into a replacement operation by using whitespace words.

[0008] Based on the loss function gradient of the word to be operated on in the text sample to be operated on and the word vector representation of multiple candidate words, candidate words of the word to be operated on are selected from the multiple candidate words;

[0009] Construct adversarial text samples of the text sample to be manipulated based on the selected candidate words.

[0010] Optionally, constructing the adversarial text sample of the text sample to be operated on based on the selected candidate words includes:

[0011] The candidate words are used to replace the words to be operated on, thus forming a text sample after the operation.

[0012] Determine the similarity between the text sample to be operated on and the text sample after operation;

[0013] If the similarity is lower than a predetermined similarity threshold, the text sample to be operated on will continue to be used to construct adversarial text samples, while the text sample after operation will be abandoned to construct dialogue text samples.

[0014] If the similarity is higher than a predetermined similarity threshold, then an adversarial text sample is constructed using the post-operation text sample.

[0015] Optionally, the step of selecting candidate words for the word to be operated on from the multiple candidate words based on the loss function gradient of the word to be operated on in the text sample to be operated on and the word vector representations of multiple candidate words includes: selecting the word with the smallest difference from the loss function gradient among the word vector representations of multiple candidate words as the candidate word for the word to be operated on.

[0016] Optionally, before selecting the word with the smallest difference from the gradient of the loss function among the word vector representations of multiple candidate words as the candidate word of the word to be operated on, the method further includes: using a context filtering model to filter out a first subset among the multiple candidate words; correspondingly, selecting the word with the smallest difference from the gradient of the loss function among the word vector representations of the multiple candidate words as the candidate word of the word to be operated on includes: selecting the word with the smallest difference from the gradient of the loss function among the word vector representations in the first subset as the candidate word of the word to be operated on.

[0017] Optionally, the first subset includes candidate words that have a higher-than-preset matching degree with the context of the word to be operated on in the text sample to be operated on.

[0018] Optionally, the step of using whitespace to unify the insertion, deletion, and replacement operations in the atomic operation sequence into a replacement operation includes:

[0019] For the insertion operation in the atomic operation sequence, a blank word is added at a selected position in the text sample to be operated on, so that the number of words in the text sample to be operated on after adding the blank word is the same as the number of words in the text sample after operation, wherein the blank word is the word to be operated on;

[0020] For the deletion operation in the atomic operation sequence, a blank word is added to the text sample after the operation so that the number of words in the text sample to be operated on and the text sample after the operation with the added blank word are the same. The words in the above are used as words to be operated on in turn, and all of the multiple candidate words are blank words.

[0021] Optionally, for the insertion operation in the atomic operation sequence, before adding a blank word at a selected position in the text sample to be operated on, the method further includes: taking turns using the positions of the beginning, end, and adjacent words as the selected positions; correspondingly, the method further includes: for each text sample formed after replacing the blank word at the selected position, determining the loss function value of the text sample input to the machine learning model, and taking the text sample after replacement with the largest loss function value as the text sample after operation.

[0022] Optionally, for the deletion operation in the atomic operation sequence, the step of selecting candidate words for the word to be operated from the plurality of candidate words based on the loss function gradient of the word to be operated in the text sample to be operated and the word vector representations of the plurality of candidate words includes:

[0023] Determine the gradient of the loss function for each word in the text sample to be processed;

[0024] The word whose gradient of the loss function differs most from the word vector of the blank word is replaced with the blank word.

[0025] Optionally, the number of atomic operations in the atomic operation sequence is the number of words in the text sample to be operated on multiplied by a predetermined coefficient or a predetermined number; after executing one atomic operation in the atomic operation sequence, the method further includes:

[0026] The processed text sample is input into the machine learning model to determine whether the machine learning model outputs the target value.

[0027] If no target value is output, the adversarial process is considered successful, and the atomic operations following the target atomic operation in the atomic operation sequence are discarded.

[0028] Optionally, the machine learning model includes an encoder and a decoder, and the loss function is a joint loss function of the encoder input and the decoder input.

[0029] According to one aspect of this disclosure, a computer device is provided, comprising: a memory for storing computer-executable code; and a processor for executing the computer-executable code to implement the method described above.

[0030] According to one aspect of this disclosure, a computer-readable medium is provided, including computer-executable code that, when executed by a processor, implements the method described above.

[0031] In this embodiment, the text sample to be manipulated undergoes a series of atomic operations, each of which can be one of a word substitution, insertion, or deletion operation. There are mature existing techniques for obtaining adversarial text samples through substitution operations. Existing techniques cannot obtain adversarial text samples through insertion or deletion operations. This embodiment cleverly utilizes blank words to transform word insertion and deletion operations into word substitution operations. For word insertion, it is equivalent to first inserting a blank word into the text sample to be manipulated, and then performing a substitution operation on that blank word. For word deletion, it is equivalent to replacing a word with a blank word. Therefore, they can all be regarded as substitution operations. Based on the loss function gradient of the word to be manipulated in the text sample and the word vector representations of multiple candidate words, a candidate word for the word to be manipulated is selected from the multiple candidate words, and used to replace the word to be manipulated, thus obtaining the adversarial text sample. Since the adversarial text sample is generated not only through word substitution, but also because the generated adversarial text sample is of variable length, exploration in the decision space is unrestricted. By not being limited by the number of words in the text sample, it is possible to find the adversarial text sample with the best adversarial effect, thereby improving the success rate of adversarial attacks against machine learning models. The adversarial text samples of this disclosure can be further used to optimize machine learning models, improving the robustness and security of the machine learning models. Attached Figure Description

[0032] The above and other objects, features, and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0033] Figure 1 The diagram illustrates the architecture of the adversarial text sample generation method of the machine learning model according to an embodiment of the present disclosure.

[0034] Figure 2 A schematic diagram illustrating the generation of adversarial text samples using a sequence of atomic operations according to an embodiment of the present disclosure is shown.

[0035] Figure 3A flowchart illustrating an adversarial text sample generation method using a machine learning model according to an embodiment of the present disclosure is shown.

[0036] Figure 4 The effectiveness of the adversarial method according to embodiments of the present disclosure and existing adversarial methods in adversarial machine learning models for natural language understanding (NLU) tasks is shown in comparison.

[0037] Figure 5 A comparison of the effectiveness of an adversarial text sample generation method utilizing whitespace words according to embodiments of the present disclosure with that of a simple insertion or deletion adversarial text sample generation method is shown.

[0038] Figure 6 The presentation shows a comparison of the performance of a machine learning model for non-automatic regression neural machine translation (NAT) obtained by adversarial tuning training using the method according to embodiments of the present disclosure, and a machine learning model for NAT obtained by adversarial tuning training using existing techniques.

[0039] Figure 7 The paper illustrates a comparison of the performance of a machine learning model for NAT obtained by adversarial tuning training according to the method of embodiments of the present disclosure with that of a machine learning model for NAT without adversarial tuning when facing various adversarial challenges.

[0040] Figure 8 A hardware structure diagram of a computer device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0041] The present disclosure is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present disclosure below, certain specific details are described in detail. Those skilled in the art will fully understand the present disclosure even without these details. To avoid obscuring the substance of the present disclosure, well-known methods, processes, and procedures are not described in detail. Furthermore, the accompanying drawings are not necessarily drawn to scale.

[0042] While machine learning models have achieved great success in various fields, they have been found to be vulnerable to adversarial attacks. An adversarial attack involves adding small, imperceptible perturbations to the original input, yet causing the machine learning model to produce an incorrect output. An adversarial example is a sample with this perturbation. This type of attack is particularly challenging for machine learning models used in Natural Language Processing (NLC) because even small perturbations to text data can alter its original semantics and thus be detected. When a text sample is input into a machine learning model, the model should output the desired target value. If a slight perturbation is then added to the text sample, such as replacing one or two synonyms, the original meaning is not significantly affected, but the machine learning model produces an incorrect output—that is, an output that is not the target value—the adversarial attack is considered successful. Adversarial attacks can be used to measure the training effectiveness of machine learning models and can be used to supplement training, i.e., retraining, to improve the robustness of the machine learning model.

[0043] Figure 1 This invention illustrates an architecture used in the adversarial text sample generation method of the machine learning model according to an embodiment of the present disclosure. The architecture includes a training machine 101, an adversarial text sample generation device 102, an optimization machine 103, and a testing machine 104.

[0044] The training machine 101 is the device used to train the machine learning model. The adversarial text sample generation device 102 is the core of this embodiment. It constructs adversarial text samples for the machine learning model trained by the training machine 101. The adversarial text samples are constructed based on the text samples to be operated on. The text samples to be operated on are the original text samples used to train or fine-tune the machine learning model. The fine-tuning machine 103 is the device used to fine-tune the performance of the machine learning model in downstream tasks after the training machine 101 has trained it. The samples used can be adversarial text samples generated by the adversarial text sample generation device 102. The testing machine 104 is the device used to test the machine learning model with test samples after the fine-tuning machine 103 has fine-tuned it. If the test passes, the machine learning model can be put into use; otherwise, it continues to be fine-tuned by the fine-tuning machine 103, and so on.

[0045] The training machine 101, the adversarial text sample generation device 102, the tuning machine 103, and the testing machine 104 can be a single computer, a cluster of multiple computers, or a part of a single computer (e.g., a virtual machine), or a collection of parts of multiple computers (e.g., the sum of virtual machines on multiple computers), or they can be in the form of a cloud.

[0046] Figure 1The diagram shown is merely one architecture used in an embodiment of this disclosure; other architectures are also possible. For example, the adversarial text samples generated in this embodiment may not be used to expand the tuning samples of the tuning machine 103, but rather simply to evaluate the machine learning model trained by the training machine 101.

[0047] According to one embodiment of this disclosure, a method for generating adversarial text samples using a machine learning model is provided. Figure 1 In the system architecture, it can be executed by the adversarial text sample generation device 102. For example... Figure 3 As shown, the method includes:

[0048] Step 310: Construct an atomic operation sequence for the text sample to be operated on. The atomic operation sequence includes multiple atomic operations to be executed sequentially. The atomic operation includes one of the word replacement operation, insertion operation, and deletion operation.

[0049] Step 320: Use the atomic operation sequence to operate on the text sample to be operated on. During the operation, use whitespace words to unify the insertion operation, deletion operation and replacement operation in the atomic operation sequence into a replacement operation.

[0050] Step 330: Based on the loss function gradient of the word to be operated on in the text sample to be operated on and the word vector representation of multiple candidate words, select the candidate word of the word to be operated on from the multiple candidate words;

[0051] Step 340: Construct adversarial text samples of the text sample to be operated on based on the selected candidate words.

[0052] These steps are described in detail below.

[0053] The text sample to be operated on in step 310 is the original text sample used to train or fine-tune a machine learning model, such as the text of a sentence. An atomic operation is the smallest unit of perturbation to a text sample, such as replacing one word with another, inserting a word, or deleting a word. Atomic operations can include one of the following: word replacement, insertion, or deletion.

[0054] After an atomic operation, the processed text sample is input into a machine learning model. If the machine learning model still outputs the target value, then the adversarial process has failed. At this point, another atomic operation is performed until the machine learning model outputs a non-target value, indicating successful adversarial execution. In this embodiment, an atomic operation sequence is pre-constructed. The atomic operation sequence is a set of sequentially executed atomic operations. The first atomic operation in the sequence is executed on the original text sample, followed by the second atomic operation on the processed text sample, and so on until the last atomic operation is executed. However, in practice, after each atomic operation, it is necessary to verify whether the processed text sample has successfully adversarially executed. That is, the processed text sample is input into the machine learning model, and it is determined whether the machine learning model outputs the target value. The target value is the value that the machine learning model should output for the text sample to be manipulated. If the target value is output, it indicates that the adversarial process has failed, and the next atomic operation in the atomic operation sequence is executed. If the target value is not output, it indicates that the adversarial process has succeeded, and the atomic operations following that atomic operation in the atomic operation sequence are discarded. Therefore, not all atomic operations in the atomic operation sequence are necessarily executed; depending on the situation, only a portion of the first few operations may be executed.

[0055] In one embodiment, the number of atomic operations in the atomic operation sequence is set to the number of words in the original text sample x, |x|, multiplied by a predetermined coefficient λ. The number of atomic operations in the atomic operation sequence represents the upper limit of adversarial operations. The predetermined coefficient λ is a hyperparameter used to control the trade-off between the overall adversarial success rate and the modification rate. It can be preset as needed. When a higher adversarial success rate is required, more atomic operations are allowed. The more atomic operations are applied, the greater the likelihood of a change in the output target value, i.e., the adversarial success rate increases, but frequent application of atomic operations reduces the modification rate. When a higher modification rate is required, fewer atomic operations are applied. These are all controlled by the magnitude of λ. This embodiment achieves a good trade-off between the overall adversarial success rate and the modification rate. Alternatively, in another embodiment, the number of atomic operations in the atomic operation sequence can also be set to a predetermined number. The advantage of this embodiment is its simplicity of implementation.

[0056] In one embodiment, each atomic operation in the atomic operation sequence is randomly selected from word substitution, insertion, and deletion operations. For example, if the first atomic operation in the atomic operation sequence is randomly selected as deletion, the second atomic operation is randomly selected as deletion, the third atomic operation is randomly selected as substitution, and so on, then the atomic operation sequence is {delete, delete, substitute, ...}.

[0057] In another embodiment, the atomic operations in the atomic operation sequence can be selected from word replacement, insertion, and deletion operations according to predetermined rules. For example, if the predetermined rule is to repeatedly select in the order of replacement, insertion, and deletion, then the atomic operation sequence is {replacement, insertion, deletion, replacement, insertion, deletion...}. The advantage of this embodiment is that the adversarial bias can be flexibly set by changing the rules to adapt to different application requirements. For example, some scenarios may dictate a higher proportion of insertion-based adversarial operations; therefore, the proportion of insertion operations in the atomic operation sequence can be increased through predetermined rules to flexibly adapt to different requirements.

[0058] Next, in step 320, the atomic operation sequence is used to operate on the text sample to be operated on. During the operation, the insertion, deletion and replacement operations in the atomic operation sequence are unified into a replacement operation by using blank words.

[0059] A substitution operation replaces one word in a text sample with another. An insertion operation inserts a word into a text sample. A deletion operation removes a word from a text sample. Although there may be cases where multiple words are replaced, multiple words are inserted, or multiple words are deleted, these can be viewed as combinations of multiple substitution operations, multiple insertion operations, and multiple deletion operations, respectively.

[0060] Since insertion and deletion operations are ultimately converted into replacement operations through whitespace (described in detail below), replacement is the most basic operation.

[0061] For the insertion operation in the atomic operation sequence, a blank word is added at a selected position in the text sample to be operated on, so that the number of words in the text sample after adding the blank word is the same as the number of words in the text sample after the operation. In this case, the blank word is the word to be operated on. Let the text sample to be operated on be x = (x1, x2, x3...x...). n ), where x1, x2, x3……x n This refers to the n words sequentially included in the sample. Assuming the selected position is between the (i-1)th word and the ith word, after inserting a blank word [BLK], the text sample x' = (x1, ..., x...). i-1

BLK

[0062] For the replacement and insertion operations described above, there is a question of the replacement position: which word in the text sample should be replaced, and where should the blank word be inserted? Although the atomic operation sequence specifies the operation performed by each atomic operation, the effect of performing a replacement or insertion operation at different positions is completely different. In one embodiment, a default replacement or insertion position can be assigned to the replacement or insertion operation. When performing a replacement or insertion operation, it is performed at that replacement position or the insertion operation at that insertion position. However, in another embodiment, all possible positions can be traversed, and replacement or insertion can be attempted at all positions. Finally, the results of the attempts at replacement or insertion at all positions are evaluated, and the best result is selected. This embodiment will be described in detail below.

[0063] For the deletion operation in the atomic operation sequence, a blank word is added to the text sample after the operation to ensure that the number of words in the text sample to be operated on and the text sample after the operation with the added blank word are the same. In this case, the added word in the text sample after the operation is always a blank word, unlike the replacement and insertion operations where there is a problem of selecting different candidate words. Therefore, the key to the deletion operation is to select the word to be operated on from the text sample to be operated on. Words in the text sample to be operated on can be used in turn, and finally, all the results are evaluated to select the best result. This embodiment will be described in detail below.

[0064] Next, in step 330, based on the loss function gradient of the word to be operated on in the text sample and the word vector representations of multiple candidate words, a candidate word for the word to be operated on is selected from the multiple candidate words. Specifically, in one embodiment, the word with the smallest difference from the loss function gradient can be selected as the candidate word for the word to be operated on from the word vector representations of multiple candidate words.

[0065] Suppose the text sample to be manipulated is x, and the result obtained after inputting x into the machine learning model, i.e., the target value, is y. Thus, y = f(x). f() is equivalent to the function of the machine learning model applying operations to the text sample to be manipulated. In adversarial scenarios, the goal is to generate a manipulated text sample x based on x. adv This makes x and the text sample x after the operation... adv The similarity sim(x,x) adv If the similarity is greater than the predetermined similarity threshold θ (i.e., x and x...), then... adv The difference in meaning is not significant and is difficult for humans to perceive. The manipulated text sample x adv The result f(x) obtained after inputting into the machine learning model advIf the target value y is not the one that causes the machine learning model to produce a different output when the meaning of the input text has not changed significantly, then the adversarial process is successful. This can be expressed by the formula:

[0066] When sim(x,x) adv When )>θ, f(x) adv )≠y (Formula 1)

[0067] x includes n words connected in sequence, namely x1, x2, x3...x n Here, x1 represents the first word, x2 represents the second word, and so on. Since in step 320, both insertion and deletion operations are converted into replacement operations using whitespace words, therefore x1, x2, x3…x n This may include blank words. In this case, all operations—replacement, insertion, and deletion—are transformed into assigning x1, x2, x3…x… n The problem involves replacing one word in a text sample with one of a plurality of candidate words. These candidate words can be words from a candidate word list. For both replacement and insertion operations, the candidate word list pre-stores all conceivable words that might exist in the text sample. j In this way, the words to be replaced (insertion and deletion operations are converted into replacement operations) are... Then it can be selected from this candidate word list. For the deletion operation, the candidate word list is a list of completely blank words, because the deletion operation can only replace one word in the text sample to be operated on with a blank word.

[0068] In 2014, Goodfellow et al. proposed a gradient-based atomic substitution operation that replaces words in a text sample with the word that maximizes the first-order approximation of the prediction loss function. This method is expressed as the following formula:

[0069]

[0070] e(x i () refers to selecting the word to be replaced from x. i Word vector representation. A word vector representation is a vector representation of a word. Machine learning models cannot process pure natural language. In order to process natural language using machine learning models, natural language must be represented as vectors. Different words are represented as different vectors. Converting words into word vector representations can be achieved through various existing methods, such as looking up word vector dictionaries. e(x j ) is any word x in the candidate word list. j The word vector representation. In this embodiment, the entire candidate word list can be represented, i.e., the multiple candidate words mentioned above (in other embodiments below, it can represent a further subset thereof, which will be detailed below). T represents the transpose of a vector. L(y, f(x)) represents the loss function of the machine learning model in the process of inputting x into the machine learning model to obtain the output y. The loss function is a known function in the training of the machine learning model, so it will not be described in detail. This indicates that the loss function applies to the word x to be replaced. i The gradient on. The word to be replaced is x i The final replacement word. For all words x in the candidate word list. j Calculate Which x j corresponding To maximize, we will use this x. j As That is, the word x in the text sample to be operated on i Replace with the word that maximizes the first-order approximation of the prediction loss function.

[0071] The essence of Formula 2 lies in replacing the word x with... i Replace with the word vector representation in the candidate word list and the word to be replaced x i Loss function gradient in the text sample to be processed The closest word. In other words, which word in the candidate word list has a word vector representation that most closely matches the word to be replaced, x. i Loss function gradient in the text sample to be processed The closest, its corresponding This will maximize the result. In practice, specifically, the word to be replaced, x, can be determined first. i Loss function gradient in the text sample to be processed Then, the word vector representation e(x) of each word in the candidate word list is... j )and In comparison, the word vector representation e(x) in the candidate word list is determined. j ) and the gradient of the loss function The word with the smallest difference is chosen as the word to select. Replace the word x to be replaced i This embodiment can use a simplified method to select words from the candidate word list to complete the replacement operation.

[0072] In the above embodiments, This represents the entire candidate word list, i.e., the multiple candidate words mentioned above. To further improve the selection of replacement words from the candidate word list... The efficiency can be improved by pre-selecting a first subset from the candidate word list and then choosing replacement words from the first subset. Specifically, the words in the candidate word list can be filtered out into a first subset using a context filtering model. The context filtering model is a known machine learning model that measures the degree of matching between a candidate word and the context of the word to be replaced in a text sample. During training, a text sample set can be constructed. Each text sample in the text sample set has a word to be replaced, which has context within the text sample. Additionally, each text sample has a pre-configured corresponding replacement word, with a pre-defined label indicating whether the replacement word is suitable to replace the word to be replaced. The text samples (including the word to be replaced and its context) in the text sample set are input into the machine learning model, which provides a judgment result on whether the corresponding replacement word is suitable for replacement, and compares this result with the pre-defined label. If the ratio of the judgment result in the text sample set matching the label exceeds a predetermined ratio, the machine learning model is considered successfully trained and can be used as a filtering model. By inputting the text sample to be operated on, containing the word to be replaced, and each word in the candidate word list into the trained model, the probability of matching between each word in the candidate word list and the context of the word to be replaced in the text sample to be operated on is obtained, i.e., the matching degree. Words with a matching degree higher than a preset threshold in the candidate word list can be extracted to form a first subset. Replacement words are then further determined from this first subset, significantly narrowing the scope of replacement word selection and improving efficiency. In another embodiment, words with the highest matching degree ranking in the candidate word list (e.g., within the top 10) can be grouped together to form a first subset. Then, the word whose word vector representation has the smallest difference from the gradient of the loss function is selected from the first subset and used to replace the word to be replaced. Searching for replacement words in the first subset, rather than the entire candidate word list, greatly reduces the search overhead of the replacement operation.

[0073] In the above embodiments, the position of the word to be replaced in the text sample is defaulted. Thus, the word to be replaced is fixed, and only a suitable word needs to be selected from the candidate word list. However, in practice, the word to be replaced x... i The word could be any word in the text sample to be operated on. Therefore, in one embodiment, each word in the text sample to be operated on can be used in turn as the word to be replaced, and step 330 can be executed. In this way, for each word in x, a corresponding replacement word is obtained. This corresponds to a replacement scheme. The choice of which replacement scheme to use can be determined as follows: For each text sample to be replaced by taking turns using each word in the text sample to be operated on, determine the loss function value of each text sample input to the machine learning model, and use the text sample with the largest loss function value as the replaced text sample. The replacement scheme that results in the largest loss function value indicates its strongest attack effect; using this replacement scheme globally optimizes the execution effect of the replacement operation. Since insertion and deletion operations have been converted into replacement operations, this optimizes the execution effect of these atomic operations.

[0074] Furthermore, for the insertion operation, the default position for adding a blank word in the text sample to be operated on is used. However, in practice, this position can be any position in the text sample to be operated on. Therefore, in one embodiment, before adding a blank word at a selected position in the selected position of the text sample to be operated on during the insertion operation in the atomic operation sequence, the beginning, end, and adjacent word positions of the text sample to be operated on are used in turn as the selected positions. In step 330, after inserting a blank word at each selected position, a replacement word for the blank word at that position is obtained. A replacement scheme for whitespace words at different positions is developed. The choice of which whitespace replacement scheme to use is determined as follows: For each text sample resulting from replacing whitespace words at selected positions, the loss function value of the input machine learning model for that text sample is determined, and the text sample with the highest loss function value is used as the processed text sample. In other words, the scheme that results in the largest loss function value among these whitespace replacement schemes is adopted, thereby optimizing the overall performance of the insertion operation.

[0075] For the deletion operation, since the replacement word must be a blank word, but the word to be replaced can be a word in a different position in the text sample to be operated on, the loss function gradient of each word in the text sample to be operated on is determined, and then the word with the largest difference between the loss function gradient of the text sample to be operated on and the word vector of the blank word is taken as the final determined replacement word, thereby achieving a good adversarial effect.

[0076] Next, in step 340, an adversarial text sample is constructed based on the selected candidate words. Specifically, in one embodiment, step 340 includes: replacing the word to be operated on with the candidate words to form a post-operated text sample; determining the similarity between the text sample to be operated on and the post-operated text sample; if the similarity is lower than a predetermined similarity threshold, then continuing to construct an adversarial text sample using the text sample to be operated on and abandoning the construction of a dialogue text sample using the post-operated text sample; if the similarity is higher than the predetermined similarity threshold, then using the post-operated text sample to construct an adversarial text sample.

[0077] If the similarity is lower than a predetermined similarity threshold, the reason for abandoning the use of the post-operation text sample to construct the dialogue text sample is: the definition of successful adversarial response requires that sim(x, x) must be equal to the threshold value. adv When )>θ, f(x) adv For a given similarity score to be considered a successful adversarial sample, the score must be ≠ y. Therefore, if the similarity score is below a predetermined similarity threshold θ, it cannot be used as an adversarial sample. If the similarity score is above the predetermined similarity threshold θ, it can be used as an adversarial sample.

[0078] Determine the x and the text sample x after the operation. adv Semantic similarity can be performed using a general sentence encoder (e.g., proposed by Cer et al. in 2018). It compares two samples x with x... adv Encode them into a pair of fixed-length vectors and calculate the cosine similarity between them.

[0079] Two important branches of Natural Language Processing (NLC) are Natural Language Understanding (NLU) and Non-Autoregressive Neural Machine Translation (NAT). There are some differences between machine learning model adversarial approaches used for NAT and those used for NLC.

[0080] First, in the NAT domain, there is no concept of successful adversarial processing. Therefore, unlike adversarial processing for NLU machine learning models, there is no way to input the text sample after each atomic operation into the machine learning model to determine whether adversarial processing was successful. If successful, no further atomic operations are performed. Therefore, when adversarial processing for NAT machine learning models, the number of atomic operations in the atomic operation sequence can be directly set to a predetermined number. After the predetermined number of atomic operations, adversarial processing is considered successful, and the successfully adversarial samples are added to the tuning training sample set of the NAT machine learning model for further training.

[0081] Second, as pointed out in the 2020 paper by Ghazvininejad et al., there is an inconsistency between training and inference in mask prediction decoding models commonly used in the NAT field. Some observed decoder inputs are correct golden targets during training, but are not at all during inference. Therefore, in one embodiment of this disclosure, when the machine learning model includes an encoder and a decoder, the loss function used in the above atomic operations is made to be a joint loss function of the encoder input and the decoder input, and not just the loss function of the encoder input. In this way, the model observes a reliable decoder input, rather than a so-called golden target, thus compensating for the aforementioned inconsistency between training and inference. Furthermore, by maintaining the target and providing the model with encoder and decoder inputs of variable length, the robustness of the model regarding length prediction is improved.

[0082] Figure 2 A schematic diagram is shown illustrating how an adversarial text sample is obtained using a sequence of atomic operations according to an embodiment of the present disclosure.

[0083] The sacrifice model 200 is the adversarial machine learning model. The original text sample 201 is “...But I got a headache.” Assume there are 7 atomic operations in the sequence. The first atomic operation is a substitution operation. Replacing “headache” with “migraine” in the original text sample 201 “...But I got a headache.” results in “...But I got a migraine.” Inputting this into the machine learning model does not change the model's target output value, so the second atomic operation continues.

[0084] The second atomic operation is the insertion operation. The insertion operation first inserts a blank word [BLK] at the selected position, i.e., "...But I got a [BLK] migraine." Then, it replaces the blank word [BLK] with the word "episodic" selected from the candidate word list, resulting in "...But I got a episodic migraine." Inputting this into the machine learning model does not change the model's target output value, so the third atomic operation continues.

[0085] The third atomic operation is deletion. The loss function gradient of each word in the sentence “...But I got a episodic migraine.” is compared with the word vector representation of the blank word, with the largest difference being “But.” This word is deleted, resulting in “...I got a episodic migraine.” Inputting this result into the machine learning model does not change the model's target output value, so the fourth atomic operation continues.

[0086] The fourth atomic operation is an insertion operation. A blank word [BLK] is inserted at the selected position in "...I got a episodic migraine.", becoming "...I got a episodic migraine [BLK]." Then, the blank word [BLK] is replaced with the word "instead" selected from the candidate word list, resulting in "...I got a episodic migraine instead." Inputting this into a machine learning model changes the model's target value; "...I got a episodic migraine instead." is the adversarial text sample 202. Since the fourth atomic operation has already changed the target value, subsequent atomic operations in the sequence are ignored.

[0087] Experimental and commercial value

[0088] The following mainly uses NLU and NAT applications as examples to illustrate the experimental effects and commercial value of the embodiments disclosed in this disclosure.

[0089] In the NLU task, the adversarial performance of the embodiments of this disclosure was primarily verified. In the NAT task, the potential of the embodiments of this disclosure in adversarial training was primarily verified.

[0090] In the NLU task, after training a bidirectional transformer encoder representation (BERT) machine learning model for NLU, the pre-trained BERT model was optimized on a downstream classification dataset as a sacrifice model. The experiments primarily followed the experimental settings of Jin et al. (2019) and Li et al. (2020) to ensure a fair basis for comparison.

[0091] Regarding datasets, experiments were conducted on two sentiment classification datasets and two natural language inference datasets. The two sentiment classification datasets included the Yelp dataset proposed by Zhang et al. in 2015 and IMDB, a known sentiment classification dataset based on movie reviews. The two natural language inference datasets included the MNLI dataset proposed by Williams et al. in 2017 and the SNLI dataset proposed by Bowman et al. in 2015.

[0092] In the experiments, 1000 test samples were selected from the datasets for each task and preprocessed accordingly. The method of this embodiment was compared with two existing black-box adversarial methods (TextFooler proposed by Jin et al. in 2019 and BERT-Attack proposed by Li et al. in 2020), such as... Figure 4 As shown. In Figure 4 The experimental results from the papers TextFooler and BERT-Attack are directly copied. In the experiments, the method of this disclosure is also compared with a white-box adversarial method, HotFlip, proposed by Ebrahimi et al. in 2017. HotFlip is primarily based on replacing atomic operations for adversarial purposes. In the experiments, word-level variations of the model are considered, and the results when attacking BERT are obtained by reimplementing the model based on its code.

[0093] In the experiment, the performance of the adversarial model was evaluated from the following aspects:

[0094] -OriAcc: The original model prediction accuracy without adversarial feedback.

[0095] - Post-adversarial accuracy (AttAcc): The model performance of the adversarial examples generated by the adversarial method. Using the same sacrifice model for the adversarial methods of this disclosure or prior art, a lower AttAcc indicates a more significant adversarial effect.

[0096] - Perturbation Ratio (Perturb%): The average percentage of words with perturbations in the total number of words in the text sample when adversarial attacks are performed. Intuitively, the less perturbation, the lower the accuracy after adversarial attacks, indicating that the adversarial attacks are more effective.

[0097] - Semantic Similarity (Sim): The semantic similarity between the text sample to be manipulated and the adversarial text sample. It is measured by the Average Universal Sentence Encoder (USE) proposed by Cer et al. in 2018.

[0098] like Figure 4 As shown, the adversarial method of this disclosure surpasses the effectiveness of all prior art adversarial methods. In particular, compared to HotFlip, which is based solely on replacing atomic operations, this disclosure achieves a more successful adversarial effect with less perturbation while maintaining higher semantic similarity. This demonstrates that more natural adversarial examples can be obtained by introducing insertion and deletion operations.

[0099] This disclosure transforms insertion and deletion operations into replacement operations by introducing blank words. To demonstrate that blank words are more effective than some simple low-frequency words and repeated words, a comparative experiment was conducted between blank words and simple insertion and simple deletion. Simple insertion refers to randomly inserting a low-frequency word into the text sample. Simple deletion refers to randomly deleting a word from the text sample.

[0100] Experimental results are as follows Figure 5As shown in the figure. In the experiment, the method of this disclosure embodiment and the methods of simple insertion and simple deletion were applied to the IMDB dataset. It can be seen that the simple insertion scheme leads to low semantic similarity, indicating that even a random low-frequency word can change the context of the text sample when inserted. The simple deletion scheme reduces the accuracy after the attack and perturbs more words, demonstrating the effectiveness and efficiency of the method of this disclosure embodiment in determining the importance of different words and in adversarial processing.

[0101] As described above, the effectiveness of the adversarial method of the present disclosure embodiments on NLU tasks has been verified. The following experiments verify the effectiveness of using the method of the present disclosure embodiments for adversarial training of machine learning models for NAT tasks.

[0102] For the experimental datasets, several known machine translation datasets were used, including the IWSLT 14 German-English dataset, the WMT 14 English-German dataset, and the WMT 16 Roman-English dataset. Some preprocessing was performed on these datasets.

[0103] Regarding the machine learning model, the mask prediction model proposed by Ghazvininejad et al. in 2019 was adopted as the sacrificed NAT model. First, a mask prediction model was pre-trained. For this mask prediction model, adversarial training according to embodiments of this disclosure was performed on the training set. Then, the model was fine-tuned on the original training set and the set of adversarial examples obtained from the adversarial training. In experiments, the model obtained from the adversarial sequence of embodiments of this disclosure was compared with a regular mask prediction model and a HotFlip model that only performs substitution atomic operations. The experimental BLEU scores are as follows: Figure 6 As shown, the BLEU score is a known metric for measuring the translation quality of a translation model. It can be seen that the HotFlip model's BLEU score is not significantly improved compared to the original mask prediction model, indicating that fixed-length adversarial exercises involving only replacement operations do not greatly help in training the NAT model. However, the adversarial exercises in this embodiment, which involve insertion, replacement, and deletion atomic operations, greatly aid in training the NAT model. Figure 6 The transformer model is a known model that outperforms the mask prediction model. It can be seen that the mask prediction model, after adversarial training with the three atomic operations of insertion, replacement, and deletion according to the embodiments of this disclosure, performs close to the transformer model.

[0104] Figure 7This paper illustrates a comparison of the performance of a machine learning model for NAT trained using the adversarial tuning method according to embodiments of the present disclosure, and a machine learning model for NAT without adversarial tuning, when facing various adversarial challenges. After training, the machine learning model is first validated on a large number of validation samples on a validation set. After successful validation, it is then tested on a large number of test samples on a test set. Figure 7 The diagram illustrates the BLEU scores of the original mask prediction model on the validation and test sets of the IWSLT14 German-English model, as well as its BLEU scores after being challenged by the HotFlip method and then by the method of this embodiment. It also shows the BLEU scores of the mask prediction model trained with the adversarial method of this embodiment on the validation and test sets of the IWSLT14 German-English model, again after being challenged by the HotFlip method and then by the method of this embodiment. It can be seen that the machine learning model for NAT, after adversarial tuning according to this embodiment, exhibits significantly enhanced performance against various adversarial attacks.

[0105] Therefore, the embodiments disclosed herein have great commercial potential.

[0106] Hardware implementation of embodiments of this disclosure

[0107] The adversarial text sample generation method of the machine learning model according to an embodiment of the present disclosure can be derived from... Figure 8 The computer equipment 800 is implemented. In Figure 1 In the system architecture, it can be an adversarial text sample generation device 102. Figure 8 The computer device 800 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0108] like Figure 8 As shown, the computer device 800 is presented in the form of a general-purpose computing device. The components of the computer device 800 may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including storage unit 820 and processing unit 810).

[0109] The storage unit stores program code that can be executed by the processing unit 810, causing the processing unit 810 to perform the steps of the various exemplary embodiments of this disclosure described in the description section of the exemplary methods above. For example, the processing unit 810 can perform actions such as... Figure 3 The steps shown are as follows.

[0110] Storage unit 820 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 8201 and / or cache memory 8202, and may further include a read-only memory (ROM) 8203.

[0111] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0112] Bus 830 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0113] Computer device 800 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with the computer device 800, and / or with any device that enables the computer device 800 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 850. Furthermore, computer device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 860. As shown, network adapter 860 communicates with other modules of computer device 800 via bus 830. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with computer device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0114] It should be understood that the above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. For those skilled in the art, there are many variations of the embodiments in this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

[0115] It should be understood that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0116] It should be understood that the foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0117] It should be understood that the use of a singular form to describe an element or to show only one element in the accompanying drawings does not imply that the number of such element is limited to one. Furthermore, modules or elements described or shown as separate herein may be combined into a single module or element, and modules or elements described or shown as single herein may be broken down into multiple modules or elements.

[0118] It should also be understood that the terminology and expressions used herein are for descriptive purposes only, and one or more embodiments described herein should not be limited to these terms and expressions. The use of these terms and expressions does not exclude any illustrative and descriptive equivalent features (or parts thereof), and it should be recognized that various modifications that may exist should also be included within the scope of the claims. Other modifications, variations, and substitutions may also exist. Accordingly, the claims should be considered to cover all such equivalents.

Claims

1. A method for generating adversarial text samples using a machine learning model, comprising: Construct an atomic operation sequence for the text sample to be operated on. The atomic operation sequence includes multiple atomic operations to be executed sequentially. The atomic operation includes one of the word replacement, insertion, and deletion operations. The atomic operation sequence is used to operate on the text sample to be operated on. During the operation, the insertion, deletion and replacement operations in the atomic operation sequence are unified into a replacement operation by using whitespace words. Based on the matching degree, a preliminary screening is performed on multiple candidate words in the candidate word list to obtain a first subset. According to the loss function gradient of the word to be operated on in the text sample to be operated on, and the word vector representation of multiple candidate words in the first subset, candidate words of the word to be operated on are selected from multiple candidate words in the first subset. The matching degree is used to represent the context matching degree between multiple candidate words in the candidate word list and the word to be operated on in the text sample to be operated on. Construct adversarial text samples of the text sample to be manipulated based on the selected candidate words.

2. The method according to claim 1, wherein, The process of constructing the adversarial text sample of the text sample to be operated on based on the selected candidate words includes: The candidate words are used to replace the words to be operated on, thus forming a text sample after the operation. Determine the similarity between the text sample to be operated on and the text sample after operation; If the similarity is lower than a predetermined similarity threshold, then continue to use the text sample to be operated on to construct adversarial text samples, and abandon the use of the text sample after operation to construct adversarial text samples; If the similarity is higher than a predetermined similarity threshold, then an adversarial text sample is constructed using the post-operation text sample.

3. The method according to claim 1 or 2, wherein, The step of selecting candidate words for the word to be operated on from multiple candidate words in the first subset based on the loss function gradient of the word to be operated on in the text sample to be operated on and the word vector representation of multiple candidate words in the first subset includes: Among the word vector representations of multiple candidate words in the first subset, the word with the smallest difference from the gradient of the loss function is selected as the candidate word of the word to be operated on.

4. The method according to claim 3, wherein, Before selecting the word with the smallest difference from the gradient of the loss function among the word vector representations of multiple candidate words in the first subset as the candidate word of the word to be operated on, the method further includes: using a context filtering model to filter out the first subset from multiple candidate words in the candidate word list.

5. The method according to claim 4, wherein, The first subset includes candidate words that have a higher-than-preset threshold matching degree with the context of the word to be operated on in the text sample to be operated on.

6. The method according to claim 3, wherein, The method of unifying the insertion, deletion, and replacement operations in the atomic operation sequence into a single replacement operation using whitespace includes: For the insertion operation in the atomic operation sequence, a blank word is added at a selected position in the text sample to be operated on, so that the number of words in the text sample to be operated on after adding the blank word is the same as the number of words in the text sample after operation, wherein the blank word is the word to be operated on; For the deletion operation in the atomic operation sequence, a blank word is added to the text sample after the operation so that the number of words in the text sample to be operated and the text sample after the operation with the added blank word are the same. In this case, the words in the text sample to be operated are taken in turn as the words to be operated, and multiple candidate words in the candidate word list are blank words.

7. The method according to claim 6, wherein, For the insertion operation in the atomic operation sequence, before adding a blank word at the selected position of the text sample to be operated on, the method further includes: taking turns using the beginning, end, and positions between adjacent words of the text sample to be operated on as the selected position. The method further includes: for each text sample formed after replacing the blank word at a selected position, determining the loss function value of the text sample input to the machine learning model, and taking the text sample with the largest loss function value as the text sample after operation.

8. The method according to claim 6, wherein, For the deletion operation in the atomic operation sequence The step of selecting candidate words for the word to be operated on from multiple candidate words in the first subset based on the loss function gradient of the word to be operated on in the text sample to be operated on and the word vector representation of multiple candidate words in the first subset includes: Determine the gradient of the loss function for each word in the text sample to be operated on; The word whose gradient of the loss function in the text sample to be operated on has the largest difference from the word vector of the blank word is replaced with the blank word.

9. The method according to claim 1, wherein, The number of atomic operations in the atomic operation sequence is the number of words in the text sample to be operated on multiplied by a predetermined coefficient or a predetermined number. After performing one atomic operation in the sequence of atomic operations, the method further includes: The processed text sample is input into the machine learning model to determine whether the machine learning model outputs the target value. If no target value is output, the adversarial process is considered successful, and the atomic operations following the target atomic operation in the atomic operation sequence are discarded.

10. The method according to claim 1, wherein, The machine learning model includes an encoder and a decoder, and the loss function is a joint loss function of the encoder input and the decoder input.

11. A computer device, comprising: Memory, used to store executable code in a computer; A processor for executing the computer-executable code to implement the method of any one of claims 1-10.

12. A computer-readable medium, characterized in that, It includes computer-executable code, which, when executed by a processor, implements the method of any one of claims 1-10.

Images