Iterative adversarial attack method for image-guided story ending generation model

Abstract

This invention discloses an iterative adversarial attack method for an image-guided story ending generation model. This method effectively and imperceptibly attacks text-image sample pairs by fusing image modality attacks into text modality attacks. When every small perturbation in the original text becomes a continuous space, the image modality is iteratively perturbed. This allows for attack search on adversarial text and images of the image-guided story ending generation model, rather than independent searches (single-modal attack methods). Experimental results show that the iterative adversarial attack method outperforms existing single-modal adversarial attack methods (kNN and WordSwap) and multi-modal adversarial attack methods (co-attack) in terms of attack success rate and semantic similarity.

Description

Technical Field

[0001] This invention relates to the field of image modality attack technology, and in particular to an iterative adversarial attack method for an image-guided story ending generation model, used to understand the adversarial robustness of multimodal text generation models and to study their reliability and security. Background Technology

[0002] Existing adversarial attack methods primarily focus on unimodal data, such as image-based attacks like FGSM and PGD, and text-based attacks like BERT-Attack and TextBugger. These unimodal attack methods neglect the complementary information in multimodal data (e.g., ...). Figure 1 As shown, this approach cannot maximize the attack on multimodal models. The latest multimodal attack method, Co-attack, simply uses a step-by-step attack mechanism: first perturbing the discrete input (text), then perturbing the continuous input (image) based on the adversarial text output. This type of method struggles to find the most vulnerable multimodal information pairs, and Co-attack is primarily geared towards multimodal matching and classification tasks. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an iterative adversarial attack method for image-guided story ending generation models. This invention attacks image-guided story ending generation models (IgSEG) by iteratively finding the most vulnerable multimodal information pairs. This invention has been tested on three image-guided story ending generation models on two real datasets. The results show that our method outperforms existing baseline methods (Co-attack, kNN, CharSwap).

[0004] This invention is achieved through the following technical solution:

[0005] An iterative adversarial attack method for an image-guided story ending generation model includes the following steps:

[0006] (1) Input the original text and original image, select vulnerable words from the story context, i.e. target words; generate multiple alternative words for each target word, replace the target word with the alternative words one by one, and generate potential adversarial text;

[0007] (2) The generated potential adversarial text is used as supervision information and the original image is input into the image attack algorithm to find the image-text information pair that is most likely to affect the output of the target model. If all the replacement words corresponding to the vulnerable word cannot achieve the attack target, the replacement word that makes the target model lose the most is selected to replace the target word. Then the next target word is replaced with the corresponding replacement word. The image-text information pair that is most vulnerable to attack is searched iteratively until the attack is successful and multimodal adversarial samples are generated.

[0008] (3) Attack target: The quality of the story ending generated by inputting the generated multimodal adversarial sample into the target model is lower than the quality of the story ending generated by inputting the original text and the original image.

[0009] Step (1) involves inputting the original text and image, selecting vulnerable words (target words) from the story context, generating multiple alternative words for each target word, and replacing the target word with the alternative words one by one to generate potential adversarial text, as follows:

[0010] Use x t x i Represents the original input text and image, where x t =[w0, ...w h ,...],w h Represents text x t The h-th word in F((x) t x i )) represents the normal output of the image-guided story ending generation model, where the vulnerable word w is in the original text xt. h Importance score Q wh for:

[0011] Q wh =F((x) t x i ))-F((x t@h ,x_i))

[0012] In the above formula, x t@h = [w0, ..., w h-1 MASK, w h+1 ...] is replaced by [MASK] h The story context after; in the original context x t Importance score of all words in Q x The definition is as follows:

[0013] Q x =[Q w1 Q wh Q wn ]

[0014] The set of importance scores for all words Q x Sort the words in descending order and select the top K words with the highest scores to put them into the important word set L;

[0015] To generate possible perturbations for each word in the important word set L, and to ensure that the generated adversarial text is semantically and visually similar to the original text, a character-level perturbation generation method (S) is used. c ) and word-level perturbation generation methods (S w As a text perturbation mechanism, therefore, for the h-th important word w h The set of substitute words C wh for:

[0016]

[0017] For text adversarial attacks, the vulnerable word is w. h Generate a set of replacement words C wh For the set of substitute words C wh Each replacement word s in j Use the replacement word s j Replace the original text x t The word w in h Generate potential adversarial text x′ t .

[0018] The specific method for step (2) is as follows:

[0019] For image attacks, potential adversarial text is used as supervisory information to attack image information that is complementary to the text perturbation, outputting a potential adversarial image:

[0020]

[0021] In the above formula, 'a' represents the number of iterations. Let x′ represent a set of allowed perturbations. i The generated adversarial image is represented by ∈, which represents the step size during image perturbation, and sign represents the sign function in mathematics. Represents the image x i Find the partial derivative, L adv Let θ represent the adversarial loss function, θ represent the parameters of the target model, and c h,j Indicates using C wh The text after the j-th replacement word replaces the target word, and y is the ending of the real story corresponding to the original text and image;

[0022] When using adversarial sample (x′) t ,x′ iThe BLEU score of the story ending generated using the original sample (x) as input is compared to... t ,x i If the BLEU score of the story ending generated as input is lower than a preset threshold λ, it indicates that the multimodal adversarial example has successfully attacked the target model. The judgment formula is as follows:

[0023]

[0024] Otherwise, save the original ending Y and the potentially erroneous ending Y generated this time. f The loss value between, and the potential adversarial text x′ t Then use the set of replacement words C wh The next replacement word s in j+1 Replace the word w in the original text h ;

[0025] If the set of replacement words C wh None of the replacement words in the equation satisfy the above formula, and the algorithm will continue to use word w by maximizing the adversarial loss. h Based on the potential adversarial text generated by the substitution word, iteratively attacks the next vulnerable word w. h+1 .

[0026] The advantages of this invention are: it can effectively perform imperceptible attacks on text-image sample pairs, integrating image modality attacks into text modality attacks. It iteratively uses each small perturbation in the original text as supervisory information to guide the perturbation of the image modality, maximizing the search for the most vulnerable multimodal information pairs. This allows for attack search on adversarial text and images in image-guided story ending generation models, rather than independent searches (single-modality attack methods). Experimental results show that the iterative adversarial attack method outperforms existing single-modality adversarial attack methods (kNN and WordSwap) and multimodal adversarial attack methods (Co-attack) in terms of attack success rate and semantic similarity. Attached Figure Description

[0027] Figure 1 This is a diagram illustrating the effect of existing single-modal attack methods.

[0028] Figure 2 This is a diagram illustrating the effect of the iterative multimodal attack method of this invention.

[0029] Figure 3 To show the average runtime of Iterative-attack and Co-attack for generating adversarial examples when attacking MGCL in the top 100 stories of the VIST-E dataset.

[0030] Figure 4 This is a diagram of the algorithm of the present invention. Detailed Implementation

[0031] An iterative adversarial attack method for an image-guided story ending generation model includes the following steps:

[0032] (1) Input the original text and original image, select vulnerable words from the story context, i.e. target words; generate multiple alternative words for each target word, replace the target word with the alternative words one by one, and generate potential adversarial text;

[0033] Potential adversarial text can be used as input for iterative adversarial attacks to find complementary adversarial images. Generally, the greater the influence of a word on the input of the target model, the more likely it is to be attacked. We select vulnerable words from the story context that can significantly influence the output of the target model.

[0034] Use x t x i Represents the original input text and image, where x t =[w o , ...w h ,...],w h Represents text x t The h-th word in F((x) t x i )) represents the normal output of the image-guided story ending generation model, where the original text x t Vulnerable words in Chinese h Importance score Q wh for:

[0035] Q wh =F((x) t x i ))-F((x t@h ,x_i))

[0036] In the above formula, x t@h = [w0, ..., w h-1 MASK, w h+1 ...] is replaced by [MASK] h The story context after; in the original context x t Importance score of all words in Q x The definition is as follows (e.g.) Figure 4 As shown in Algorithm 1, lines 2-4):

[0037] Q x =[Q w1 Q whQ wn ]

[0038] The set of importance scores for all words Q x Sort the words in descending order and select the top K words with the highest scores to put them into the important word set L (Algorithm 1, line 5);

[0039] To generate possible perturbations for each word in the important word set L, and to ensure that the generated adversarial text is semantically and visually similar to the original text, a character-level perturbation generation method (S) is used. c ) and word-level perturbation generation methods (S w As a text perturbation mechanism, therefore, for the h-th important word w h The set of substitute words C wh for:

[0040]

[0041] For text adversarial attacks, the vulnerable word is w. h Generate a set of replacement words C wh (Algorithm 1, line 7), for the set of replacement words C wh Each replacement word s in j Use the replacement word s j Replace the original text x t The word w in h Generate potential adversarial text x′ t (Algorithm 1, lines 10-11).

[0042] (2) The generated potential adversarial text is used as supervision information and the original image is input into the image attack algorithm to find the image-text information pair that is most likely to affect the output of the target model. If all the replacement words corresponding to the vulnerable word cannot achieve the attack target, the replacement word that makes the target model lose the most is selected to replace the target word. Then the next target word is replaced with the corresponding replacement word. The image-text information pair that is most vulnerable to attack is searched iteratively until the attack is successful and multimodal adversarial samples are generated.

[0043] (3) Attack target: The quality of the story ending generated by inputting the generated multimodal adversarial sample into the target model is lower than the quality of the story ending generated by inputting the original text and the original image.

[0044] To iteratively attack image-guided story ending generation models, we propose a novel iterative multimodal attack method that integrates image attacks with text attacks, such as... Figure 2As shown. Our proposed method iteratively attacks the input image and text to find the image-text information pairs that are most likely to influence the output of the target model, until a multimodal adversarial sample (x′) is generated. t ,x′ i This ensures that the quality of the story ending generated by the target model is lower than the threshold λ relative to the quality of the story ending generated by the target model from the original sample input.

[0045] For image attacks, we target image information that is complementary to the text perturbation, and output a potential adversarial image (Algorithm 1, line 12):

[0046]

[0047] In the above formula, 'a' represents the number of iterations. Let x′ represent a set of allowed perturbations. i The generated adversarial image is represented by ∈, which represents the step size during image perturbation, and sign represents the sign function in mathematics. L represents taking the partial derivative with respect to the image x_i. adv Let θ represent the adversarial loss function, θ represent the parameters of the target model, and c h,j Indicates using C wh The text after the j-th replacement word replaces the target word, and y is the ending of the real story corresponding to the original text and image;

[0048] When using adversarial sample (x′) t ,x′ i The BLEU score of the story ending generated using the original sample (x) as input is compared to... t ,x i If the BLEU score of the story ending generated as input is lower than a preset threshold λ, it indicates that the multimodal adversarial example has successfully attacked the target model. The judgment formula is as follows:

[0049]

[0050] Otherwise, save the original ending Y and the potentially erroneous ending Y generated this time. f The loss value between, and the potential adversarial text x′ t Then use the set of replacement words C wh The next replacement word s in j+1 Replace the word w in the original text h (Algorithm 1, lines 10-19);

[0051] If the set of replacement words C wh None of the replacement words in the equation satisfy the above formula, and the algorithm will continue to use word w by maximizing the adversarial loss. h Based on the potential adversarial text generated by the substitution word, iteratively attacks the next vulnerable word w.h+1 (Algorithm 1, lines 20-23).

[0052] Experimental setup

[0053] Datasets. In this invention, VIST-E and LSMDC-E were used to validate the image-guided story ending generation task. VIST-E consists of 49,913 training samples, 4,963 validation samples, and 5,030 test samples, and is a modified version of the VIS dataset. Since each sample in VIST contains a five-sentence story, each sample in VIST-E includes the story's ending, an image associated with the ending, and the first four sentences of the story as background. Furthermore, the maximum length of each sentence is set to 40 words. LSMDC-E contains 20,151 training samples, 1,477 validation samples, and 2,005 test samples, and is a modified version of LSMDC 2021. Similarly, we use the first four sentences of each five-sentence story as background and the last sentence as the story ending. Since each sentence is associated with a set of movie frames, we use the last set of frames as the image set associated with the ending. The maximum sentence length is set to 20 words.

[0054] Hyperparameter settings. For image modal perturbation, we use a PGD attack. The maximum perturbation is set to 2 / 255. The step size is set to 1.25. The number of PGD iterations is set to 20. For fair comparison, the maximum number of characters for text perturbation is set to 2 among all adversarial attack methods. The threshold λ is set to 0.5.

[0055] Image-guided story ending generation model:

[0056] Seq2Seq is an attention-based model with stacked RNNs.

[0057] Transformer is a parallel model based entirely on an attention mechanism. To adapt to the task of image-guided story ending generation, SeqSeq and Transformer use concatenated text and visual features as input.

[0058] MGCL first introduces ending-related images to guide the generation of story endings. It proposes a GCN-based text encoder and an LSTM-based decoder to construct logically consistent and semantically rich story endings.

[0059] Adversarial attack methods. We compare Iterative-attack with Co-attack, the latter simply combining text and image attacks to perform multimodal adversarial attacks on visual language pre-trained models; kNN and CharSwap are white-box non-target attacks against neural machine translation models, where kNN replaces some words with their neighbors in the embedding space; CharSwap swaps characters in the target word according to certain rules.

[0060] II. Evaluation Matrix

[0061] To fully evaluate our proposed method, we report different performance metrics: (1) Attack Success Rate (ASR), which measures the proportion of adversarial examples generated by the iterative-attack that successfully mislead the target model. That is, if the BLEU score at the end of the story is less than half that of the original story, we consider the adversarial example successful. (2) Relative Degradation of Story Ending Generation Quality. We use RDBLEU and RDchrF to represent these two metrics. We calculate the relative degradation of story ending generation quality to compare scores across different models and datasets. (3) Semantic Similarity (Sim.), which calculates the similarity between the original and adversarial text, typically approximated by Universal Sentence Encoding. (4) Complexity Score (Perp.), a metric measuring the fluency of the generated adversarial text, calculated using the complexity score of GPT-2. The entire method is implemented in PyTorch, and all experiments were performed on a single GeForce RTX 1080 GPU.

[0062] III. Discussion of Experimental Results

[0063] Tables 1 and 2 show the attacks on different IgSEGs using different adversarial attack methods on the VIST-E and LSMDC-E datasets. 1 Experimental results of the model. From Table 1 and Table 2, we can draw the following conclusions:

[0064] (1) In summary, for MGCL and Seq2Se, our proposed Iterative-attack reduces the BLEU score of over 60% of the story endings to less than half of the BLEU score of the unattacked story endings (except for the Transformer model, where Iterative-attack achieves an attack success rate of over 30% on both datasets). Furthermore, in all test cases, the semantic similarity of the adversarial text generated by Iterative-attack exceeds 0.96, indicating that Iterative-attack can maintain a high semantic similarity to the original context.

[0065] Table 1 shows the performance data of different IgSEG models under white-box attacks on the VIST-E dataset.

[0066]

[0067] Table 2 shows the performance data of different IgSEG models under white-box attacks on the LSMDC-E dataset.

[0068]

[0069]

[0070] (2) Compared with other baseline methods, Iterative-attack achieved higher success rates against different IgSEG models, and Iterative-attack could significantly reduce the quality of generated story endings. kNN and CharSwap performed the worst in ASR, RDBLEU, and RDchrF among all adversarial attacks against IgSEG models (IgSEG is an abbreviation for image-guided story ending generation), indicating that in multimodal text generation tasks, adversarial attacks on only the text modality data cannot maximize the text generation probability between multimodal inputs and real story endings.

[0071] (3) As a multimodal adversarial attack method for classification tasks, Co-attack outperforms kNN and CharSwap in multimodal text generation tasks, but is significantly inferior to Iterative-attack. The reason is that although Co-attack gradually utilizes text adversarial attacks and image adversarial attacks to attack the input of multimodal neural networks, Co-attack cannot find vulnerable and complementary multimodal information pairs.

[0072] IV. Ablation Experiment

[0073] We compared various variants of the Iterative-attack on the VIST-E and LSMDC-E datasets, as shown in Tables 3 and 4, respectively. The proposed variants of the Iterative-attack are as follows:

[0074] • Text-attack: A variant of Iterative-attack that removes image adversarial attacks when attacking IgSEG models.

[0075] • Image-attack: A variant of Iterative-attack that removes text adversarial attacks when attacking IgSEG models.

[0076] Character-attack attack: A variant of Iterative-attack, which removes the word-level alternative word generation strategy when attacking the IgSEG model.

[0077] • Word-attack: A variant of Iterative-attack that removes the character-level replacement word generation strategy when attacking the IgSEG model.

[0078] Table 3 shows the performance data of Iterative-attack against different modalities of MGCL on the VIST-E and LSMDC-E datasets.

[0079]

[0080] Table 4 shows the performance data of MGCL for different levels of text perturbation attacks in Iterative-attack on the VIST-E and LSMDC-E datasets.

[0081]

[0082] From Tables 3 and 4, we can draw the following conclusions:

[0083] (1) Iteratively perturbing the multimodal input is always stronger than perturbing any unimodal input (Text-attack and Image-attack), which suggests that the adversarial samples generated by multimodal adversarial attacks are more dangerous than those generated by unimodal adversarial attacks.

[0084] (2) On both datasets, text attacks outperform image attacks. However, without image adversarial attacks to disrupt key visual information, text attacks outperform multimodal adversarial attacks. This suggests that due to the complementarity between multimodal data, the information shift caused by a single-modal adversarial attack can be corrected by another modal data, thus leading to adversarial failure.

[0085] (3) Iterative-attack outperforms Character-attack and Word-attack, which suggests that the alternative words generated by combining character-level perturbation and word-level perturbation have a greater impact on the output of the target model than the perturbation at a single level.

[0086] V. Comparison of running times

[0087] Since kNN and CharSwap are text-oriented adversarial attack methods, for a fair comparison, we... Figure 3 The image shows the average runtime of Iterative-attack and Co-attack for generating adversarial examples when attacking MGCL in the first 100 stories of the VIST-E dataset, at different maximum iteration numbers. We can observe that Iterative-attack runs slightly longer than Co-attack, but they are on the same order of magnitude. More importantly, when the runtimes of Iterative-attack and Co-attack are similar, the success rate of Iterative-attack is not worse than that of Co-attack.

[0088] VI. The impact of the number of the most disturbed words in the text

[0089] To verify the effectiveness of using different numbers of perturbation words P in iterative attacks, we set the number of perturbation words P from 1 to 5 in the first 100 stories, as shown in Table 5. The success rate of the attack on the VIST-E dataset is increasing, but the similarity and fluency of the adversarial text are decreasing. Furthermore, the attack runtime is gradually increasing, indicating that we need to strike a trade-off between attack success rate and the number of perturbation words.

[0090] Table 5 shows the relevant data when using different numbers of word perturbations during the attack.

[0091]

Claims

1. An iterative adversarial attack method for an image-guided story ending generation model, characterized in that: Specifically, the steps include the following: (1) Input the original text and original image, select vulnerable words from the story context, i.e. target words; generate multiple alternative words for each target word, replace the target word with the alternative words one by one, and generate potential adversarial text; (2) The generated potential adversarial text is used as supervision information and the original image is input into the image attack algorithm to find the image-text information pair that is most likely to affect the output of the target model. If all the replacement words corresponding to the vulnerable word cannot achieve the attack target, the replacement word that makes the target model lose the most is selected to replace the target word. Then the next target word is replaced with the corresponding replacement word. The image-text information pair that is most vulnerable to attack is searched iteratively until the attack is successful and multimodal adversarial samples are generated. (3) Attack target: The quality of the story ending generated by inputting the generated multimodal adversarial sample into the target model is lower than the quality of the story ending generated by inputting the original text and the original image.

2. The iterative adversarial attack method for an image-guided story ending generation model according to claim 1, characterized in that: Step (1) involves inputting the original text and image, selecting vulnerable words (target words) from the story context, generating multiple alternative words for each target word, and replacing the target word with the alternative words one by one to generate potential adversarial text, as follows: Use x t ,x i Represents the original input text and the original image, where x t =[w0,...w h ,…],w h Represents text x t The h-th word in F((x) t ,x i )) represents the normal output of the image-guided story ending generation model, where the original text x t Vulnerable words in Chinese h Importance score Q wh for: Q wh =F((x t ,x i ))-F((x t@h ,x_i)) In the above formula, x t@h =[w0,…,w h-1 MASK,w h+1 …] is replaced by [MASK] h The story context after; in the original context x t Importance score of all words in Q x The definition is as follows: Q x =[Q w1 ,…,Q wh ,…,Q wn ] The set of importance scores for all words Q x Sort the words in descending order and select the top K words with the highest scores to put them into the important word set L; To generate possible perturbations for each word in the important word set L, and to ensure that the generated adversarial text is semantically and visually similar to the original text, a character-level perturbation generation method (S) is used. c ) and word-level perturbation generation methods (S w As a text perturbation mechanism, therefore, for the h-th important word w h The set of substitute words C wh for: C wh =S c ⊕S w For text adversarial attacks, the vulnerable word is w. h Generate a set of replacement words C wh For the set of substitute words C wh Each replacement word s in j Use the replacement word s j Replace the original text x t The word w_h in the text generates potential adversarial text x′. t .

3. The iterative adversarial attack method for an image-guided story ending generation model according to claim 2, characterized in that: The specific method for step (2) is as follows: For image attacks, the goal is to attack image information that complements text perturbations, outputting potentially adversarial images: In the above formula, 'a' represents the number of iterations. Let x′ represent a set of allowed perturbations. i The generated adversarial image is represented by ∈, where ∈ represents the step size during image perturbation, and sign represents the sign function in mathematics. Represents the image x i Find the partial derivative, L adv Let θ represent the adversarial loss function, θ represent the parameters of the target model, and c h,j Indicates using C wh The text after the j-th replacement word replaces the target word, and y represents the ending of the real story corresponding to the original text and the image; When using adversarial sample (x′) t ,x′ i The BLEU score of the story ending generated using the original sample (x) as input is compared to... t ,x i If the BLEU score of the story ending generated as input is lower than a preset threshold λ, it indicates that the multimodal adversarial example has successfully attacked the target model. The judgment formula is as follows: Otherwise, save the original ending Y and the potentially erroneous ending Y generated this time. f The loss value between, and the potential adversarial text x′ t Then use the set of replacement words C wh The next replacement word s in j+1 Replace the word w in the original text h ; If the set of replacement words C wh None of the replacement words in the equation satisfy the above formula, and the algorithm will continue to use word w by maximizing the adversarial loss. h Based on the potential adversarial text generated by the substitution word, iteratively attack the next vulnerable word w. h+1 .

Images