Context adversarial trigger generation method, system, device, product and medium

Abstract

The present invention relates to the field of artificial intelligence security technology, and provides a context adversarial trigger generation method, system, device, product and medium, including obtaining a source text and a first context adversarial trigger, obtaining a first candidate vocabulary through the first context adversarial trigger and the source text, and obtaining a second context adversarial trigger through the first candidate vocabulary; obtaining and updating a first mask language model to obtain a second mask language model; selecting a label training sample to obtain a second candidate vocabulary, generating a third context adversarial trigger according to the second candidate vocabulary and iterating to obtain a fourth context adversarial trigger and a maximum second mask language model parameter, updating the second mask language model through the maximum second mask language model parameter to obtain a third mask language model, and obtaining a target context adversarial trigger by iterating the third mask language model. The present invention can effectively simulate more diverse trigger attacks.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security technology, and in particular to a context adversarial trigger generation method, system, device, product and medium. Background Art

[0002] Deep neural network models have become the mainstream of natural language processing models. With their powerful learning ability, deep neural network models can capture complex semantic relationships from large amounts of text data and are widely used in a variety of natural language processing tasks. However, deep neural network models are also threatened by adversarial attacks.

[0003] Adversarial attacks against natural language processing deep neural network models aim to generate adversarial samples that can deceive deep neural network models by perturbing the original input text. Although adversarial attacks pose a threat to the security of deep neural network models, they also play a positive role in revealing the vulnerabilities of deep neural network models. Since deep neural network models are usually black box models, their internal working mechanisms are often opaque and difficult to understand intuitively, adversarial attacks can be used to evaluate and explain these black box models. In addition, adversarial samples have also been shown to have a significant effect in enhancing the robustness of deep neural network models in adversarial training.

[0004] According to the different semantic granularity, existing adversarial attack methods can be divided into three categories: character-level, word-level, and sentence-level attacks. Among them, word-level attacks are the most common form of attack, which mainly involves replacing, inserting, or deleting words to deceive deep neural network models. Among them, adversarial trigger attacks are a special type of word-level attack. Unlike typical word-level attack methods based on word replacement, adversarial trigger attacks perturb the original input text by appending a word sequence (i.e., trigger) at the beginning or end of the input without changing the content or semantics of the original text. However, existing adversarial trigger attack methods often use a single trigger to attack all texts without considering contextual information, so they are limited in generating diverse and effective triggers. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a contextual adversarial trigger generation method, system, device, product and medium to generate more diverse adversarial triggers with contextual awareness to reveal the weaknesses and vulnerabilities of deep neural network models and improve the robustness of deep neural network models.

[0006] The present invention provides a context adversarial trigger generation method, comprising:

[0007] S1: Obtain a public training set, and obtain source text and source text labels from the public training set;

[0008] S2: Obtain a first context adversarial trigger, obtain a first candidate vocabulary list through the first context adversarial trigger and the source text, and generate a second context adversarial trigger through the first candidate vocabulary list;

[0009] S3: Acquire a first mask language model, obtain a maximum first mask language model parameter through the second context adversarial trigger, and update the first mask language model through the maximum first mask language model parameter to obtain a second mask language model;

[0010] S4: selecting a label training sample through the source text label, predicting through the second mask language model and the label training sample to obtain a second candidate vocabulary, generating a third context adversarial trigger according to the second candidate vocabulary; iterating the third context adversarial trigger through a dynamic threshold filtering strategy to obtain a fourth context adversarial trigger;

[0011] S5: obtaining a maximum second mask language model parameter through the fourth context adversarial trigger, obtaining a third mask language model through the maximum second mask language model parameter, obtaining a target mask language model through iteration of the third mask language model, and predicting a target context adversarial trigger through the target mask language model.

[0012] According to the context adversarial trigger generation method provided by the present invention, step S2 further comprises:

[0013] S21: Acquire the first context adversarial trigger and the vocabulary library, and fuse the plurality of source texts with the first context adversarial trigger to obtain a plurality of fused texts;

[0014] S22: replacing the first context adversarial triggers in the plurality of fused texts word by word using the vocabulary library, calculating label loss functions of the plurality of fused texts, and calculating a first-order Taylor approximation of the context of the first context adversarial trigger in the plurality of fused texts using the label loss function;

[0015] S23: obtaining a single sample gradient and an average gradient of the first context adversarial trigger through the first-order Taylor approximation of the plurality of fused texts, determining a single sample gradient vocabulary and an average gradient vocabulary in turn according to the single sample gradient and the average gradient, merging the single sample gradient vocabulary with the average gradient vocabulary to obtain the first candidate vocabulary;

[0016] S24: performing a greedy search on the first candidate vocabulary to obtain a target first candidate vocabulary, and updating the first context adversarial trigger using the target first candidate vocabulary to obtain a second context adversarial trigger.

[0017] According to the context adversarial trigger generation method provided by the present invention, step S3 further comprises:

[0018] S31: acquiring the first mask language model, calculating the maximum log-likelihood value of the second context adversarial trigger in the first mask language model, and obtaining the maximum first mask language model parameter;

[0019] S32: Substitute the maximum first mask language model parameter into the first mask language model to obtain the second mask language model.

[0020] According to the context adversarial trigger generation method provided by the present invention, step S4 further comprises:

[0021] S41: Select a type of source text label and use the corresponding source text as the label training sample;

[0022] S42: predicting the labeled training sample by using the second masked language model to obtain an initial second candidate vocabulary, performing a greedy search on the initial second candidate vocabulary to obtain the second candidate vocabulary, and generating a third contextual adversarial trigger according to the second candidate vocabulary;

[0023] S43: Obtain a probability threshold, and obtain a text label retention probability by calculating the probability that the text label of the label training sample does not change due to the third context adversarial trigger;

[0024] S44: The third context adversarial trigger is screened and iterated by using the probability threshold and the text label retention probability; meanwhile, the probability threshold is iterated until the probability threshold converges, and the third context adversarial trigger when the probability threshold converges is used as the fourth context adversarial trigger.

[0025] According to the context adversarial trigger generation method provided by the present invention, step S5 further comprises:

[0026] S51: Calculate the maximum log-likelihood value of the fourth context adversarial trigger in the second mask language model to obtain a maximum second mask language model parameter;

[0027] S52: updating the second masked language model by using the maximum second masked language model parameter to obtain a third masked language model;

[0028] S53: Determine an expected number of iterations, iterate the third mask language model until the number of iterations reaches the expected number of iterations, obtain the target mask language model, and perform prediction through the target mask language model to obtain the target context adversarial trigger.

[0029] According to the context adversarial trigger generation method provided by the present invention, the target mask language model obtains the target context adversarial trigger through autoregressive decoding prediction.

[0030] The present invention also provides a contextual confrontation trigger generation system, comprising:

[0031] Source text module: used to obtain a public training set, and obtain source text and source text labels from the public training set;

[0032] A second context adversarial trigger module: used to obtain a first context adversarial trigger, obtain a first candidate vocabulary list through the first context adversarial trigger and the source text, and generate a second context adversarial trigger through the first candidate vocabulary list;

[0033] A second mask language model module: used to obtain the first mask language model, obtain a maximum first mask language model parameter through the second context adversarial trigger, and update the first mask language model through the maximum first mask language model parameter to obtain a second mask language model;

[0034] The fourth context adversarial trigger module is used to select a label training sample through the source text label, predict through the second mask language model and the label training sample to obtain a second candidate vocabulary, and generate a third context adversarial trigger according to the second candidate vocabulary; iterate the third context adversarial trigger through a dynamic threshold filtering strategy to obtain a fourth context adversarial trigger;

[0035] Target context adversarial trigger module: used to obtain the maximum second mask language model parameter through the fourth context adversarial trigger, obtain the third mask language model through the maximum second mask language model parameter, obtain the target mask language model through the iteration of the third mask language model, and predict the target context adversarial trigger through the target mask language model.

[0036] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of any of the above-described context adversarial trigger generation methods are implemented.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described context adversarial trigger generation methods.

[0038] The present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the steps of any of the context adversarial trigger generation methods described above.

[0039] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0040] The contextual adversarial trigger generation method, system, device, product and medium provided by the present invention can generate more diverse and high-quality adversarial triggers with contextual awareness, thereby more effectively exposing model vulnerabilities, and can also be used for adversarial training to improve the robustness and effectiveness of deep neural network models. Adversarial training requires adversarial samples. After adversarial training is performed by generating adversarial samples using the contextual adversarial triggers generated by the present invention, the trained deep neural network model is no longer sensitive to adversarial samples generated by various existing adversarial triggers, significantly enhancing the defense capability against general adversarial trigger attacks.

[0041] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a flowchart of the context adversarial trigger generation method provided by the present invention.

[0044] Figure 2 It is a schematic diagram of the effect of the context adversarial trigger of the context adversarial trigger generation method provided by the present invention interfering with the text label judgment result of the deep neural network model.

[0045] Figure 3 It is a structural schematic diagram of the context adversarial trigger generation system provided by the present invention.

[0046] Figure 4It is a structural schematic diagram of a context adversarial trigger generating device provided by the present invention.

[0047] Reference numerals:

[0048] 100. Source text module; 200. Second context adversarial trigger module; 300. Second mask language model module; 400. Fourth context adversarial trigger module; 500. Target context adversarial trigger module; 810. Processor; 820. Communication interface; 830. Memory; 840. Communication bus. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme in the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0050] In the description of the embodiments of the present invention, it should be noted that the terms “first”, “second” and “third” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0051] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0052] Combine the following Figures 1 to 4 Describing embodiments of the present invention:

[0053] Figure 1 A flow chart of the context adversarial trigger generation method provided in the present invention, wherein source text and source text labels can be obtained through S1, a second context adversarial trigger can be generated through S2, a second masked language model can be generated through S3, a fourth context adversarial trigger can be generated through S4, and a target context adversarial trigger can be generated through S5.

[0054] The present invention provides a context adversarial trigger generation method, comprising:

[0055] S1: Obtain a public training set, and obtain source text and source text labels from the public training set;

[0056] Furthermore, the purpose of this stage is to obtain a public training set from the outside world, and obtain the source text and source text label from the public training set for subsequent use. In this embodiment, the source text is a text that has practical meaning and can be read and understood normally; the source text label is a text label that marks the emotional color and content of the source text. For example, when the source text is an evaluation, the source text label can be positive or negative, and when the source text is news, the source text label can be true news or false news. The public training set is a data set containing N source texts and source text labels. , where the source text label is the i-th source text The corresponding text label.

[0057] S2: Obtain a first context adversarial trigger, obtain a first candidate vocabulary list through the first context adversarial trigger and the source text, and generate a second context adversarial trigger through the first candidate vocabulary list;

[0058] Furthermore, the purpose of this stage is to obtain a fused text and calculate a first-order Taylor approximation of the context of the first context adversarial trigger in the fused text, thereby obtaining a first candidate vocabulary, and obtaining a second context adversarial trigger based on the first candidate vocabulary. Step S2 further includes:

[0059] S21: Acquire the first context adversarial trigger and the vocabulary library, and fuse the plurality of source texts with the first context adversarial trigger to obtain a plurality of fused texts;

[0060] S22: replacing the first context adversarial triggers in the plurality of fused texts word by word using the vocabulary library, calculating label loss functions of the plurality of fused texts, and calculating a first-order Taylor approximation of the context of the first context adversarial trigger in the plurality of fused texts using the label loss function;

[0061] S23: obtaining a single sample gradient and an average gradient of the first context adversarial trigger through the first-order Taylor approximation of the plurality of fused texts, determining a single sample gradient vocabulary and an average gradient vocabulary in turn according to the single sample gradient and the average gradient, merging the single sample gradient vocabulary with the average gradient vocabulary to obtain the first candidate vocabulary;

[0062] S24: performing a greedy search on the first candidate vocabulary to obtain a target first candidate vocabulary, and updating the first context adversarial trigger using the target first candidate vocabulary to obtain a second context adversarial trigger.

[0063] With respect to the above steps, the specific implementation methods in this embodiment are as follows:

[0064] A contextual adversarial trigger is a sequence of words in the context of a normal text. This sequence can be appended before or after the text to make the deep neural network model make mistakes when judging the text label of the text. This sequence of words is called a contextual adversarial trigger. Figure 2 This is a schematic diagram of the effect of contextual adversarial triggers interfering with the text label judgment results of the deep neural network model. It can be seen that the original input text is "we loved the place and the bloody marry is awsome". For this original input text, the text label given by the deep neural network model is "positive"; when the contextual adversarial trigger "this sterilized decor" containing three words is added to the beginning of the text and the input with the trigger is generated, the prediction of the deep neural network model changes. At this time, the text label given is "negative", but the words and sentences of the contextual adversarial trigger itself have no emotional tendency. Each original input text has a corresponding contextual adversarial trigger.

[0065] First, obtain a context adversarial trigger including a series of meaningless word sequences as the first context adversarial trigger, for example, "the the the" is a first context adversarial trigger with a length of 3; obtain a vocabulary library including multiple words. Then, splice and fuse the first context adversarial trigger to the beginning or end of multiple source texts to obtain multiple fused texts. Select words from the vocabulary library to replace the words in the first context adversarial trigger from left to right, and then calculate the label loss function of each fused text in the multiple fused texts in turn, that is:

[0066]

[0067] Among them, L() is the label loss function, is the i-th first context adversarial trigger, represents the length of the first context adversarial trigger, Prediction through deep neural network model express and Splicing and Fusion represents the 1st, 2nd, ..., 1st in the i-th first context adversarial trigger words, which are the independent variables of the multivariate loss function. Then, according to the first-order expansion of Taylor's formula, we can infer that ,in represents the i-th first context adversarial trigger No. words, the first-order expansion of Taylor's formula can be rewritten as follows:

[0068]

[0069] in, word, represents the gradient of the loss function with respect to the word in the first context adversarial trigger before replacement, )express The multivariate loss function for the words in , and ) is a constant, so we only need to optimize the item on the right that contains the gradient to obtain the first-order Taylor approximation of the context of the first context adversarial trigger in multiple fused texts:

[0070]

[0071] Among them, V represents the vocabulary library, () indicates finding the maximum value of the independent variable, G indicates the gradient difference value of the words in the first context adversarial trigger, and T indicates transposition. In this way, after multiple replacements of multiple fused texts, the single sample gradients and average gradients of multiple first context adversarial triggers can be obtained through first-order Taylor approximation. Here, the single sample gradient refers to the gradient difference value of the words in the first context adversarial trigger, and the average gradient refers to the average value of the single sample gradients of the words at the same position of the first context adversarial trigger in multiple fused texts; for example, if there are two fused texts, including two first context adversarial triggers, then the average gradient of the first word in the two fused texts is the average value of the single sample gradients of the first word in the two first context adversarial triggers. Then, by calculating the single sample gradient and the average gradient, the top ranking of each word in the first context adversarial trigger in each fused text can be obtained. The replacement word list corresponding to the single sample gradient is used as the single sample gradient vocabulary, and the top ranking words in multiple fusion texts can also be obtained. The replacement word list corresponding to the average gradient of is used as the average gradient vocabulary list, and the single sample gradient vocabulary list is merged with the average gradient vocabulary list to obtain the first candidate vocabulary list.

[0072] Finally, a greedy search is performed on the first candidate vocabulary to obtain a target first candidate vocabulary, and then words are selected from the target first candidate vocabulary to update the first context adversarial trigger to obtain a second context adversarial trigger.

[0073] S3: Acquire a first mask language model, obtain a maximum first mask language model parameter through the second context adversarial trigger, and update the first mask language model through the maximum first mask language model parameter to obtain a second mask language model;

[0074] Furthermore, the purpose of this stage is to obtain the maximum first mask language model parameter, and obtain the second mask language model through the maximum first mask language model parameter, wherein step S3 further includes:

[0075] S31: acquiring the first mask language model, calculating the maximum log-likelihood value of the second context adversarial trigger in the first mask language model, and obtaining the maximum first mask language model parameter;

[0076] S32: Substitute the maximum first mask language model parameter into the first mask language model to obtain the second mask language model.

[0077] Here, first obtain the first mask language model. In this embodiment, obtain a BERT series mask language model as the first mask language model. The first mask language model can generate a context adversarial trigger of a specified length by prediction, wherein the length of the context adversarial trigger of the specified length is , which is generated as follows:

[0078]

[0079] here, Indicates Contextual adversarial triggers of specified length The words, Represents a predicted contextual adversarial trigger of a specified length From the first to the words; Represents a contextual adversarial trigger of a specified length that has not yet been predicted From the to words, Represents the contextual adversarial trigger of the specified length currently being predicted No. words, MLM() stands for the BERT series masked language model. For The conditions required for prediction. Then substitute the second context adversarial trigger into the first mask language model, and calculate the maximum log-likelihood value of the second context adversarial trigger in the first mask language model, so as to obtain the maximum first mask language model parameter:

[0080]

[0081] in, represents the parameters of the first mask language model, Indicates Second Context Adversarial Trigger The words, () indicates that the masked language model predicts the word as The probability function Represents the predicted second context adversarial trigger From the first to the words, Represents the second contextual adversarial trigger that has not yet been predicted From the to words, represents the first mask language model parameter when the log-likelihood value takes the maximum value, is the maximum first mask language model parameter, is the number of the second context adversarial triggers. Substitute the maximum first mask language model parameter into the first mask language model to obtain the second mask language model.

[0082] S4: selecting a label training sample through the source text label, predicting through the second mask language model and the label training sample to obtain a second candidate vocabulary, generating a third context adversarial trigger according to the second candidate vocabulary; iterating the third context adversarial trigger through a dynamic threshold filtering strategy to obtain a fourth context adversarial trigger;

[0083] Furthermore, the purpose of this stage is to predict through the label training sample and the second mask language model to obtain a third context adversarial trigger, and iterate the third context adversarial trigger through the dynamic threshold filtering strategy to obtain a fourth context adversarial trigger. Among them, step S4 further includes:

[0084] S41: Select a type of source text label and use the corresponding source text as the label training sample;

[0085] S42: predicting the labeled training sample by using the second masked language model to obtain an initial second candidate vocabulary, performing a greedy search on the initial second candidate vocabulary to obtain the second candidate vocabulary, and generating a third contextual adversarial trigger according to the second candidate vocabulary;

[0086] S43: Obtain a probability threshold, and obtain a text label retention probability by calculating the probability that the text label of the label training sample does not change due to the third context adversarial trigger;

[0087] S44: The third context adversarial trigger is screened and iterated by using the probability threshold and the text label retention probability; meanwhile, the probability threshold is iterated until the probability threshold converges, and the third context adversarial trigger when the probability threshold converges is used as the fourth context adversarial trigger.

[0088] First, a class of source text labels is selected from the source text labels as the designated labels, and the source text corresponding to the designated labels is used as the label training sample to ensure that the target context adversarial trigger generated at the end can attack the source text with the designated text label; for example, the source text label "positive" is selected from the source text labels as the designated label, and all source texts with the text label "positive" can be used as label training samples. Then, the context adversarial trigger of the label training sample is predicted by the second masked language model, and the top-ranked words in the prediction process are used as the initial second candidate vocabulary, and then the initial second candidate vocabulary is greedily searched to obtain the second candidate vocabulary. The third context adversarial trigger can be generated by selecting words from the second candidate vocabulary according to the specified context adversarial trigger length.

[0089] Then, get the probability threshold Calculate the Third context adversarial trigger The probability that the text label of the labeled training sample does not change , get the probability of text label preservation; here, For the labeled training samples, For the Labeled training samples The corresponding specified label, () indicates that the deep neural network model predicts the text label as Then, the third context adversarial trigger is screened by the probability threshold and the text label retention probability, iteratively, that is, when When the third context adversarial trigger is considered to have high quality, it is retained; otherwise, it is discarded and a new third context adversarial trigger is generated for judgment, thereby iterating the third context adversarial trigger. Indicates the number of iterations. Probability threshold Depend on Each time the third context adversarial trigger is screened and iterated, the probability threshold Also iterate, that is:

[0090]

[0091] in, represents a given initial threshold, Indicates the specified target threshold, Represents the maximum number of iterations. When the third context adversarial trigger iterates once, the probability threshold will also iterate once. When the probability threshold reaches the specified target threshold, it is considered that the probability threshold converges, and then the iteration of the third context adversarial trigger and the probability threshold is stopped, and the third context adversarial trigger at this time is used as the fourth context adversarial trigger.

[0092] S5: obtaining a maximum second mask language model parameter through the fourth context adversarial trigger, obtaining a third mask language model through the maximum second mask language model parameter, obtaining a target mask language model through iteration of the third mask language model, and predicting a target context adversarial trigger through the target mask language model.

[0093] Furthermore, the purpose of this stage is to obtain the maximum second mask language model parameter through the fourth context adversarial trigger, and then obtain the third mask language model through the maximum second mask language model parameter, and obtain the target context adversarial trigger by iterating the third mask language model. Among them, step S5 further includes:

[0094] S51: Calculate the maximum log-likelihood value of the fourth context adversarial trigger in the second mask language model to obtain a maximum second mask language model parameter;

[0095] S52: updating the second masked language model by using the maximum second masked language model parameter to obtain a third masked language model;

[0096] S53: Determine an expected number of iterations, iterate the third mask language model until the number of iterations reaches the expected number of iterations, obtain the target mask language model, and perform prediction through the target mask language model to obtain the target context adversarial trigger.

[0097] First, the maximum log-likelihood value of the fourth context adversarial trigger in the second mask language model is calculated to obtain the maximum second mask language model parameter, and then the second mask language model is updated by the maximum second mask language model parameter to obtain the third mask language model, and then the expected number of iterations is determined, and the third mask language model is used to replace the second mask language model in step S42 to predict the label training sample, and then steps S42 to S52 are repeated to iterate the third mask language model until the number of iterations reaches the expected number of iterations to obtain the target mask language model; the target context adversarial trigger is predicted by the target mask language model. Among them, since the length of the target context adversarial trigger is short, the target mask language model can predict the target context adversarial trigger through autoregressive decoding.

[0098] The present invention also verifies the effectiveness of the context adversarial trigger generation method. Here, two English datasets FAKE and YELP are used. The FAKE dataset is a fake news detection dataset, in which news is divided into two categories: true news and fake news. The YELP dataset is a sentiment classification dataset, in which text sentiment is divided into two categories: positive and negative. The two datasets are divided into training sets, validation sets, and test sets for experiments and their average lengths are counted. Table 1 shows the statistical information of the FAKE and YELP datasets:

[0099] Table 1 Statistics of FAKE and YELP datasets

[0100]

[0101] Next, the target context adversarial trigger is concatenated with the contents of the two datasets, and the processed dataset contents are sent to the deep neural network model for detection, thereby attacking the deep neural network model. The results are shown in Table 2, where CAT represents this method, and RAT and UAT represent existing methods:

[0102] Table 2 Comparison of attack effects between this method and existing methods

[0103]

[0104] As can be seen from Table 2, the method of the present application achieved the best results in terms of the success rate of attacks using adversarial triggers. In both fake news detection and sentiment analysis tasks, the present method significantly reduced the classification accuracy of the deep neural network model, which shows that the present method successfully identified weaknesses in the model's decision-making. The superiority of the present method is particularly evident on the FAKE dataset, which may be because the FAKE dataset contains more specific features compared to the YELP dataset. Specifically, the FAKE dataset involves the classification of news content, in which certain elements (such as places, names, and dates) are closely related to the authenticity of the news, which often leads to deep neural network models making high-confidence judgments based on some news elements. Due to the contextual characteristics of the present method, it is able to more effectively identify and utilize these specific features, resulting in a more significant performance gap.

[0105] Then, we conducted an adversarial training experiment with contextual adversarial triggers. The main evaluation indicators used here are the classification accuracy (expressed as PAA) of the adversarially trained deep neural network model when attacked, the attack success rate (expressed as ASR), and the classification accuracy (expressed as CA) of the deep neural network model on clean data, that is, the data set without adding contextual adversarial triggers. Among them, the upward arrow indicates that the higher the value of the indicator, the better, and the downward arrow indicates that the lower the value of the indicator, the better. For the FAKE data set, "+" indicates true news and "-" indicates fake news; for the YELP data set, "+" indicates positive categories and "-" indicates negative categories. The experimental results show that the present invention is ahead of other methods in improving the robustness of the model.

[0106] As shown in Table 3, the deep neural network model (expressed as CAT-based) trained adversarially using the method proposed in the present invention is compared with the initial deep neural network model (expressed as Original) without adversarial training and the deep neural network model (expressed as UAT-based) trained adversarially using other existing methods. The present method achieves the best effect in improving the defense capability of the deep neural network model against such trigger attacks, and the classification accuracy on clean data has not decreased significantly, and in some cases has even improved.

[0107] Table 3. Experimental results of adversarial training of contextual adversarial triggers

[0108]

[0109] In summary, the present invention can effectively perform adversarial training on the deep neural network model by generating target context adversarial triggers that adapt to the context, thereby improving the robustness of the deep neural network model and helping to discover vulnerabilities in the deep neural network model.

[0110] The context confrontation trigger generation system provided by the present invention is described below. The context confrontation trigger generation system described below and the context confrontation trigger generation method described above can be referenced to each other.

[0111] Figure 3 The structural diagram of the context adversarial trigger generation system is shown as an example. Figure 3 As shown, the method for generating a context adversarial trigger as described above includes:

[0112] Source text module 100: used to obtain a public training set, and obtain source text and source text labels from the public training set;

[0113] The second context adversarial trigger module 200 is used to obtain a first context adversarial trigger, obtain a first candidate vocabulary list through the first context adversarial trigger and the source text, and generate a second context adversarial trigger through the first candidate vocabulary list;

[0114] The second mask language model module 300 is used to obtain the first mask language model, obtain the maximum first mask language model parameter through the second context adversarial trigger, and update the first mask language model through the maximum first mask language model parameter to obtain the second mask language model;

[0115] Fourth context adversarial trigger module 400: used to select label training samples through source text labels, predict through the second mask language model and the label training samples to obtain a second candidate vocabulary, generate a third context adversarial trigger according to the second candidate vocabulary; iterate the third context adversarial trigger through a dynamic threshold filtering strategy to obtain a fourth context adversarial trigger;

[0116] Target context adversarial trigger module 500: used to obtain the maximum second mask language model parameter through the fourth context adversarial trigger, obtain the third mask language model through the maximum second mask language model parameter, obtain the target mask language model through iteration of the third mask language model, and predict the target context adversarial trigger through the target mask language model.

[0117] on the other hand, Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the context adversarial trigger generation method, which includes:

[0118] S1: Obtain a public training set, and obtain source text and source text labels from the public training set;

[0119] S2: Obtain a first context adversarial trigger, obtain a first candidate vocabulary list through the first context adversarial trigger and the source text, and generate a second context adversarial trigger through the first candidate vocabulary list;

[0120] S3: Acquire a first mask language model, obtain a maximum first mask language model parameter through the second context adversarial trigger, and update the first mask language model through the maximum first mask language model parameter to obtain a second mask language model;

[0121] S4: selecting a label training sample through the source text label, predicting through the second mask language model and the label training sample to obtain a second candidate vocabulary, generating a third context adversarial trigger according to the second candidate vocabulary; iterating the third context adversarial trigger through a dynamic threshold filtering strategy to obtain a fourth context adversarial trigger;

[0122] S5: obtaining a maximum second mask language model parameter through the fourth context adversarial trigger, obtaining a third mask language model through the maximum second mask language model parameter, obtaining a target mask language model through iteration of the third mask language model, and predicting a target context adversarial trigger through the target mask language model.

[0123] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0124] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, the computer can execute the context adversarial trigger generation method provided by the above methods, the method comprising:

[0125] S1: Obtain a public training set, and obtain source text and source text labels from the public training set;

[0126] S2: Obtain a first context adversarial trigger, obtain a first candidate vocabulary list through the first context adversarial trigger and the source text, and generate a second context adversarial trigger through the first candidate vocabulary list;

[0127] S3: Acquire a first mask language model, obtain a maximum first mask language model parameter through the second context adversarial trigger, and update the first mask language model through the maximum first mask language model parameter to obtain a second mask language model;

[0128] S4: selecting a label training sample through the source text label, predicting through the second mask language model and the label training sample to obtain a second candidate vocabulary, generating a third context adversarial trigger according to the second candidate vocabulary; iterating the third context adversarial trigger through a dynamic threshold filtering strategy to obtain a fourth context adversarial trigger;

[0129] S5: obtaining a maximum second mask language model parameter through the fourth context adversarial trigger, obtaining a third mask language model through the maximum second mask language model parameter, obtaining a target mask language model through iteration of the third mask language model, and predicting a target context adversarial trigger through the target mask language model.

[0130] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the context adversarial trigger generation method provided by the above methods, the method comprising:

[0131] S1: Obtain a public training set, and obtain source text and source text labels from the public training set;

[0132] S2: Obtain a first context adversarial trigger, obtain a first candidate vocabulary list through the first context adversarial trigger and the source text, and generate a second context adversarial trigger through the first candidate vocabulary list;

[0133] S3: Acquire a first mask language model, obtain a maximum first mask language model parameter through the second context adversarial trigger, and update the first mask language model through the maximum first mask language model parameter to obtain a second mask language model;

[0134] S4: selecting a label training sample through the source text label, predicting through the second mask language model and the label training sample to obtain a second candidate vocabulary, generating a third context adversarial trigger according to the second candidate vocabulary; iterating the third context adversarial trigger through a dynamic threshold filtering strategy to obtain a fourth context adversarial trigger;

[0135] S5: obtaining a maximum second mask language model parameter through the fourth context adversarial trigger, obtaining a third mask language model through the maximum second mask language model parameter, obtaining a target mask language model through iteration of the third mask language model, and predicting a target context adversarial trigger through the target mask language model.

[0136] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0137] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A contextual adversarial trigger generation method, characterized in that: include: S1: Obtain a public training set, and obtain source text and source text labels from the public training set; S2: Obtain a first context adversarial trigger, obtain a first candidate vocabulary list through the first context adversarial trigger and the source text, and generate a second context adversarial trigger through the first candidate vocabulary list; S3: Acquire a first mask language model, obtain a maximum first mask language model parameter through the second context adversarial trigger, and update the first mask language model through the maximum first mask language model parameter to obtain a second mask language model; S4: selecting a label training sample according to the source text label, performing prediction by using the second masked language model and the label training sample to obtain a second candidate vocabulary, and generating a third context adversarial trigger according to the second candidate vocabulary; Iterating the third context adversarial trigger through a dynamic threshold filtering strategy to obtain a fourth context adversarial trigger; S5: obtaining a maximum second mask language model parameter through the fourth context adversarial trigger, obtaining a third mask language model through the maximum second mask language model parameter, obtaining a target mask language model through iteration of the third mask language model, and predicting a target context adversarial trigger through the target mask language model.

2. The context adversarial trigger generation method according to claim 1, characterized in that: Step S2 further comprises: S21: Acquire the first context adversarial trigger and the vocabulary library, and fuse the plurality of source texts with the first context adversarial trigger to obtain a plurality of fused texts; S22: replacing the first context adversarial triggers in the plurality of fused texts word by word using the vocabulary library, calculating label loss functions of the plurality of fused texts, and calculating a first-order Taylor approximation of the context of the first context adversarial trigger in the plurality of fused texts using the label loss function; S23: obtaining a single sample gradient and an average gradient of the first context adversarial trigger through the first-order Taylor approximation of the plurality of fused texts, determining a single sample gradient vocabulary and an average gradient vocabulary in turn according to the single sample gradient and the average gradient, merging the single sample gradient vocabulary with the average gradient vocabulary to obtain the first candidate vocabulary; S24: performing a greedy search on the first candidate vocabulary to obtain a target first candidate vocabulary, and updating the first context adversarial trigger using the target first candidate vocabulary to obtain a second context adversarial trigger.

3. The context adversarial trigger generation method according to claim 1, characterized in that: Step S3 further comprises: S31: acquiring the first mask language model, calculating the maximum log-likelihood value of the second context adversarial trigger in the first mask language model, and obtaining the maximum first mask language model parameter; S32: Substitute the maximum first mask language model parameter into the first mask language model to obtain the second mask language model.

4. The context adversarial trigger generation method according to claim 1, characterized in that: Step S4 further comprises: S41: Select a type of source text label and use the corresponding source text as the label training sample; S42: predicting the labeled training sample by using the second masked language model to obtain an initial second candidate vocabulary, performing a greedy search on the initial second candidate vocabulary to obtain the second candidate vocabulary, and generating a third contextual adversarial trigger according to the second candidate vocabulary; S43: Obtain a probability threshold, and obtain a text label retention probability by calculating the probability that the text label of the label training sample does not change due to the third context adversarial trigger; S44: The third context adversarial trigger is screened and iterated by using the probability threshold and the text label retention probability; meanwhile, the probability threshold is iterated until the probability threshold converges, and the third context adversarial trigger when the probability threshold converges is used as the fourth context adversarial trigger.

5. The context adversarial trigger generation method according to claim 1, characterized in that: Step S5 further comprises: S51: Calculate the maximum log-likelihood value of the fourth context adversarial trigger in the second mask language model to obtain a maximum second mask language model parameter; S52: updating the second masked language model by using the maximum second masked language model parameter to obtain a third masked language model; S53: Determine an expected number of iterations, iterate the third mask language model until the number of iterations reaches the expected number of iterations, obtain the target mask language model, and perform prediction through the target mask language model to obtain the target context adversarial trigger.

6. The context adversarial trigger generation method according to claim 5, characterized in that: The target mask language model predicts the target context adversarial trigger through autoregressive decoding.

7. A contextual adversarial trigger generation system, configured to execute the contextual adversarial trigger generation method according to any one of claims 1 to 6, characterized in that: include: Source text module: used to obtain a public training set, and obtain source text and source text labels from the public training set; A second context adversarial trigger module: used to obtain a first context adversarial trigger, obtain a first candidate vocabulary list through the first context adversarial trigger and the source text, and generate a second context adversarial trigger through the first candidate vocabulary list; A second mask language model module: used to obtain the first mask language model, obtain a maximum first mask language model parameter through the second context adversarial trigger, and update the first mask language model through the maximum first mask language model parameter to obtain a second mask language model; A fourth context adversarial trigger module: configured to select a label training sample through a source text label, perform prediction through the second masked language model and the label training sample to obtain a second candidate vocabulary, and generate a third context adversarial trigger according to the second candidate vocabulary; Iterating the third context adversarial trigger through a dynamic threshold filtering strategy to obtain a fourth context adversarial trigger; Target context adversarial trigger module: used to obtain the maximum second mask language model parameter through the fourth context adversarial trigger, obtain the third mask language model through the maximum second mask language model parameter, obtain the target mask language model through the iteration of the third mask language model, and predict the target context adversarial trigger through the target mask language model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the context adversarial trigger generation method as described in any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the context adversarial trigger generation method as described in any one of claims 1 to 6 are implemented.

10. A computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, characterized in that: When the program instructions are executed by a computer, the computer can perform the steps of the contextual adversarial trigger generation method as described in any one of claims 1 to 6.

Images