Multimodal jailbreaking attack method and device for visual language model

Abstract

The application provides a multi-modal jailbreaking attack method and device for a visual language model, which has the following characteristics and comprises the following steps: step S1, collecting random noise as an initial adversarial image; step S2, optimizing the initial adversarial image according to a first optimization target to obtain an optimized adversarial image; step S3, constructing an initial adversarial text; step S4, jointly optimizing the optimized adversarial image and the initial adversarial text according to a second optimization target to obtain an adversarial image prefix and an adversarial text suffix; and step S5, inputting the adversarial image prefix, a malicious query and the adversarial text suffix as a whole into the visual language model to obtain a high-toxicity positive reply. In summary, the method can realize a high-success-rate jailbreaking attack on the visual language model.

Description

Multimodal jailbreak attack methods and devices targeting visual language models Technical Field

[0001] This invention relates to the field of artificial intelligence security, specifically to a method and apparatus for multimodal jailbreak attacks targeting visual language models. Background Technology

[0002] Recent advancements in visual language models (VLMs), such as OpenAI's GPT-4 and Google's Flamingo, have garnered widespread attention from academia, industry, and the public. These developments have also attracted the attention of AI security researchers, who are increasingly focused on evaluating the adversarial robustness of multimodal models. While VLMs are expected to exhibit the same alignment properties as large language models (LLMs)—providing helpful responses to user queries while rejecting potentially harmful requests—the additional visual modality introduces new vulnerabilities.

[0003] Greshake et al. explored the effectiveness of manually injecting deceptive text into input images. In contrast, other studies have proposed more sophisticated image-domain adversarial attack methods. Carlini et al. fixed the initial portion of the harmful target output and optimized the input image to increase the probability of generating it. Bagdasaryan et al. and Bailey et al. employed similar strategies, generating attacker-selected text using teacher coercion techniques, although this text may not be directly related to harmful content. Another white-box attack proposed by Qi et al. employs a similar principle to Bagdasaryan et al., aiming to find a general adversarial visual input. Specifically, this attack no longer focuses on specific output sentences but attempts to maximize the probability of generating derogatory outputs from a corpus containing 66 harmful sample sentences. This strategy was inspired by Wallace et al., who also utilized a discrete search-based optimization algorithm to find a general adversarial trigger, increasing the probability of generating a small set of harmful sentences. Shayegani et al. proposed attacking publicly available visual encoders, such as CLIP used in multimodal models, thus eliminating the need for fully white-box access.

[0004] While the methods described above have shown impressive results, they primarily focus on exploring adversarial robustness against image-based unimodal attacks, a strategy based on the consensus that attacks in a continuous image space are more effective than attacks in a discrete text space.

[0005] However, the above methods result in the full attack surface available in the multimodal model not being fully utilized. Therefore, there is still room for improvement in jailbreak attacks on multimodal models, i.e., visual-language models. Summary of the Invention

[0006] This invention is made to solve the above-mentioned problems, and aims to provide a multimodal jailbreak attack method and apparatus targeting visual language models.

[0007] This invention provides a multimodal jailbreak attack method targeting visual language models, used to perform jailbreak attacks on visual language models through malicious queries to obtain corresponding highly toxic positive responses. It includes the following steps: Step S1, collecting random noise as an initial adversarial image; Step S2, using the initial adversarial image as input to the visual language model, and optimizing the initial adversarial image according to a first optimization objective to obtain an optimized initial adversarial image as an optimized adversarial image; Step S3, constructing initial adversarial text; Step S4, using the optimized adversarial image, the initial adversarial text, and existing malicious user queries as input to the visual language model, and jointly optimizing the optimized adversarial image and the initial adversarial text according to a second optimization objective to obtain an optimized adversarial image as an adversarial image prefix and an optimized initial adversarial text as an adversarial text suffix; Step S5, using the adversarial image prefix, malicious query, and adversarial text suffix as a whole input to the visual language model to obtain a highly toxic positive response, wherein the first optimization objective is to increase the probability of the visual language model generating a highly toxic response based on the initial adversarial image, and the second optimization objective is to increase the probability of the visual language model generating a positive response based on the optimized adversarial image and the initial adversarial text.

[0008] The multimodal jailbreak attack method for visual language models provided by this invention may also have the following feature: wherein the expression for the first optimization objective is: In the formula For the initial adversarial image, For the visual language model, the text input is empty, s i 'm' represents the sample size of the corpus of highly toxic responses, where 'm' is a text response containing harmful content.

[0009] The multimodal jailbreak attack method for visual language models provided by the present invention may also have the following feature: in step S2, the first optimization objective is solved by the projection gradient descent method to obtain the optimized adversarial image.

[0010] The multimodal jailbreak attack method for visual language models provided by this invention may also have the following feature: the initial adversarial text consists of multiple exclamation marks.

[0011] The multimodal jailbreak attack method for visual language models provided by this invention may also have the following feature: wherein the expression for the second optimization objective is: In the formula To optimize adversarial images, For the initial adversarial text, t i For an affirmative response, g i For malicious user queries, m represents the sample size of the corpus.

[0012] The multimodal jailbreak attack method for visual language models provided by this invention may also have the following features: In step S4, the image domain optimization problem in the second optimization objective is solved by the projection gradient descent method, and the text domain optimization problem in the second optimization objective is solved by the greedy coordinate gradient method, so as to obtain the adversarial image prefix and adversarial text suffix.

[0013] This invention also provides a multimodal jailbreak attack device targeting visual language models, used to perform jailbreak attacks on visual language models through malicious queries to obtain corresponding highly toxic positive responses. It features the following characteristics: a model storage module for storing visual language models; a universal key generation module for generating adversarial image prefixes and adversarial text suffixes based on the visual language model; and a malicious response generation module for inputting the adversarial image prefixes, malicious queries, and adversarial text suffixes as a whole into the visual language model to obtain highly toxic positive responses. The universal key generation module includes: an initial adversarial image generation unit for collecting random noise as an initial adversarial image; and an optimized adversarial image generation unit for using the initial adversarial image as input to the visual language model and optimizing it according to a first optimization objective. The initial adversarial image is optimized to obtain an optimized initial adversarial image as an optimized adversarial image. An initial adversarial text generation unit is used to construct the initial adversarial text. A universal key generation unit stores a training dataset containing multiple malicious user queries. It is used to take the optimized adversarial image, the initial adversarial text, and the malicious user queries as input to the visual language model. According to the second optimization objective, the optimized adversarial image and the initial adversarial text are jointly optimized to obtain an optimized adversarial image as an adversarial image prefix and an optimized initial adversarial text as an adversarial text suffix. The first optimization objective is to increase the probability that the visual language model generates a highly toxic response based on the initial adversarial image. The second optimization objective is to increase the probability that the visual language model generates a positive response based on the optimized adversarial image and the initial adversarial text.

[0014] The role and effect of invention

[0015] According to the multimodal jailbreak attack method and apparatus targeting visual language models of the present invention, a highly toxic optimized adversarial image is obtained by optimizing the initial adversarial image through a first optimization objective. Then, the optimized adversarial image and the initial adversarial text are jointly optimized through a second optimization objective to further generate a universal key that can obtain a positive response. Finally, the combination of the universal key and a malicious query enables the visual language model to generate a corresponding highly toxic positive response. Therefore, the multimodal jailbreak attack method and apparatus targeting visual language models of the present invention can fully utilize both text and image attack surfaces to achieve a high success rate jailbreak attack against visual language models. Attached Figure Description

[0016] Figure 1 is a schematic diagram illustrating the working principle of the multimodal jailbreak attack device in an embodiment of the present invention;

[0017] Figure 2 is a block diagram of a multimodal jailbreak attack device in an embodiment of the present invention;

[0018] Figure 3 is a flowchart illustrating the multimodal jailbreak attack method for visual language models in an embodiment of the present invention. Detailed Implementation

[0019] To make the technical means, creative features, objectives and effects of the present invention easy to understand, the following embodiments, in conjunction with the accompanying drawings, specifically illustrate the multimodal jailbreak attack method and apparatus of the present invention targeting visual language models.

[0020] This embodiment provides a multimodal jailbreak attack device targeting visual language models, which is used to perform jailbreak attacks on visual language models through malicious queries to obtain corresponding highly toxic positive responses.

[0021] In this embodiment, a malicious query is an instruction containing harmful information, and the corresponding highly toxic affirmative response is a text response generated by the visual language model that is related to the malicious query and contains harmful content. For example, if a user sends a malicious query to the visual language model, instructing it to provide a specific plan for doing something bad, the corresponding highly toxic affirmative response is the specific plan for doing that bad thing generated by the visual language model.

[0022] Figure 1 is a schematic diagram illustrating the working principle of the multimodal jailbreak attack device in an embodiment of the present invention.

[0023] As shown in Figure 1, the multimodal jailbreak attack device generates adversarial image prefixes and adversarial text suffixes targeting the visual language model as a master key through joint optimization. When a user queries, the master key and the malicious query are input into the visual language model as a whole. The visual language model first processes the adversarial image prefix in the input through a visual encoder and mapping layer to obtain the corresponding image features. Then, the image features, the malicious query, and the adversarial text suffix are combined into a whole vector and used by the subsequent language model to generate text, thus obtaining a highly toxic positive response, namely, "Okay, here are the detailed steps on how to do... (a bad thing):...". The phrase "Okay, here are the detailed steps on how to do... (a bad thing):" indicates that the visual language model has generated corresponding content for the malicious query, meaning that the content is a targeted positive response, rather than other content unrelated to the malicious query.

[0024] Figure 2 is a block diagram of a multimodal jailbreak attack device in an embodiment of the present invention.

[0025] As shown in Figure 2, the multimodal jailbreak attack device 100 includes a model storage module 10, a universal key generation module 20, a malicious response generation module 30, and a control module 40 that controls the operation of the above modules.

[0026] The model storage module 10 is used to store the visual language model.

[0027] The universal key generation module 20 is used to generate adversarial image prefixes and adversarial text suffixes based on the visual language model stored in the model storage module 10. It includes an initial adversarial image generation unit 201, an optimized adversarial image generation unit 202, an initial adversarial text generation unit 203, and a universal key generation unit 204.

[0028] The initial adversarial image generation unit 201 is used to collect random noise as the initial adversarial image.

[0029] The optimized adversarial image generation unit 202 is used to take the initial adversarial image as input to the visual language model and optimize the initial adversarial image according to the first optimization objective to obtain the optimized initial adversarial image as the optimized adversarial image.

[0030] The first optimization objective is to increase the probability that the visual language model generates a highly toxic response based on the initial adversarial image, and its expression is:

[0031]

[0032] In the formula For the initial adversarial image, The text input for the visual language model is empty, meaning that the optimized adversarial image generation unit 202 only uses the initial adversarial image as input to the visual language model. i , where m represents the sample size of the corpus of highly toxic responses.

[0033] In this embodiment, the first optimization objective is solved by the projection gradient descent method to obtain the optimized adversarial image.

[0034] The initial adversarial text generation unit 203 is used to construct the initial adversarial text. In this embodiment, the initial adversarial text is 20 consecutive exclamation marks "!".

[0035] The universal key generation unit 204 stores a training dataset containing multiple malicious user queries. It is used to take the optimized adversarial image, the initial adversarial text, and the malicious user query as input to the visual language model, and to jointly optimize the optimized adversarial image and the initial adversarial text according to the second optimization objective, so as to obtain the optimized adversarial image as the adversarial image prefix and the optimized initial adversarial text as the adversarial text suffix.

[0036] The second optimization objective is to increase the probability that the visual language model generates a positive response based on the optimized adversarial image and the initial adversarial text, and its expression is:

[0037]

[0038] In the formula To optimize adversarial images, For the initial adversarial text, t i For an affirmative response, g i For malicious user queries, m represents the sample size of the corpus.

[0039] In this embodiment, the image domain optimization problem in the second optimization objective is solved by the projection gradient descent method, and the text domain optimization problem in the second optimization objective is solved by the greedy coordinate gradient method, so as to obtain the adversarial image prefix and adversarial text suffix.

[0040] The malicious response generation module 30 is used to input adversarial image prefixes, malicious queries, and adversarial text suffixes as a whole into the visual language model to obtain highly toxic positive responses.

[0041] The control module 40 stores the control program that controls the operation of each module.

[0042] The process of using the multimodal jailbreak attack device 100 to perform a multimodal jailbreak attack on the visual language model is described below with reference to the accompanying drawings.

[0043] Figure 3 is a flowchart illustrating the multimodal jailbreak attack method for visual language models in an embodiment of the present invention.

[0044] As shown in Figure 3, the multimodal jailbreak attack method targeting the visual language model includes the following steps:

[0045] Step S1: Random noise is collected by the initial adversarial image generation unit 201 as the initial adversarial image.

[0046] Step S2: The initial adversarial image is used as the input of the visual language model by the optimized adversarial image generation unit 202, and the initial adversarial image is optimized according to the first optimization objective to obtain the optimized initial adversarial image.

[0047] Step S3: Initial adversarial text is constructed using the initial adversarial text generation unit 203.

[0048] Step S4: The universal key generation unit 204 takes the optimized adversarial image, the initial adversarial text, and the existing malicious user query as input to the visual language model, and performs joint optimization on the optimized adversarial image and the initial adversarial text according to the second optimization objective to obtain the optimized adversarial image as the adversarial image prefix and the optimized initial adversarial text as the adversarial text suffix.

[0049] Step S5: The malicious response generation module 30 takes the adversarial image prefix, malicious query and adversarial text suffix as the whole input to the visual language model to obtain a highly toxic positive response.

[0050] In this embodiment, firstly, steps S1 to S2 are performed to optimize the initial adversarial image, resulting in an optimized adversarial image infused with toxic semantics, which enables the visual language model to generate a highly toxic response. Next, steps S3 to S4 are performed to jointly optimize the optimized adversarial image and the initial adversarial text, obtaining an adversarial image prefix and an adversarial text suffix as a universal key, enabling the visual language model to further generate a positive response. Finally, the universal key and the malicious query are input as a whole into the visual language model, causing it to generate a highly toxic and positive response, i.e., a highly toxic positive response.

[0051] In this embodiment, based on the existing visual language model, the multimodal jailbreak attack method for visual language models in this embodiment (i.e., the method of this invention) and the jailbreak attack method mentioned in the existing literature "Qi X, Huang K, Panda A, et al. Visual adversarial examples jailbreak large language models[J]. arXiv preprint arXiv:2306.13213,2023." (i.e., the general jailbreak method) were used for jailbreak attacks. The success rate of the method of this invention was 96%, while the success rate of the general jailbreak method was 64%. It can be seen that the method of this invention has a better attack success rate than the existing methods.

[0052] Furthermore, existing jailbreak attack methods targeting visual language models only optimize the adversarial target when there is no text input. This results in the jailbreak attack failing to strictly follow user instructions, making it easier to generate harmful statements unrelated to the user's query. The method of this invention, however, solves this problem by setting a first optimization objective and a second optimization objective, achieving the effect of jailbreak attacks strictly adhering to user instructions.

[0053] The multimodal jailbreak attack method for visual language models in this embodiment can be further used for security detection of visual language models. By using this method, the jailbreak success rate of the visual language model to be detected can be obtained, thereby evaluating and improving the security performance of the visual language model to be detected.

[0054] The role and effect of the embodiments

[0055] According to the multimodal jailbreak attack method and apparatus targeting visual language models involved in this embodiment, the initial adversarial image is optimized to obtain a highly toxic optimized adversarial image through a first optimization objective. Then, the optimized adversarial image and the initial adversarial text are jointly optimized through a second optimization objective to further generate a universal key that can obtain a positive response. Finally, the combination of the universal key and malicious queries enables the visual language model to generate a corresponding highly toxic positive response. In summary, this method can fully utilize both text and image attack surfaces to achieve a high success rate jailbreak attack against visual language models.

[0056] Those skilled in the art should understand that this invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to this invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A multimodal jailbreak attack method targeting visual language models, used to perform jailbreak attacks on visual language models through malicious queries to obtain corresponding highly toxic positive responses, characterized in that... Includes the following steps: Step S1: Collect random noise as the initial adversarial image; Step S2: The initial adversarial image is used as input to the visual language model, and the initial adversarial image is optimized according to the first optimization objective to obtain the optimized initial adversarial image as the optimized adversarial image. The expression of the first optimization objective is: In the formula To optimize adversarial images infused with toxic semantics, The text input for the visual language model is empty. The response is highly toxic. The sample size of the high-toxicity response corpus is defined as text responses containing harmful content; Step S3: Construct initial adversarial text; Step S4: Use the optimized adversarial image, the initial adversarial text, and existing malicious user queries as input to the visual language model, and perform joint optimization on the optimized adversarial image and the initial adversarial text according to the second optimization objective to obtain the optimized adversarial image as the adversarial image prefix and the optimized initial adversarial text as the adversarial text suffix. The expression for the second optimization objective is: In the formula For the initial adversarial text, For an affirmative response, For the malicious user query; Step S5, input the adversarial image prefix, the malicious query, and the adversarial text suffix as a whole into the visual language model to obtain the highly toxic positive response, wherein the first optimization objective is to increase the probability of the visual language model generating a highly toxic response based on the initial adversarial image, and the second optimization objective is to increase the probability of the visual language model generating a positive response based on the optimized adversarial image and the initial adversarial text.

2. The multimodal jailbreak attack method targeting visual language models according to claim 1, characterized in that: in, In step S2, the first optimization objective is solved using the projection gradient descent method to obtain the optimized adversarial image.

3. The multimodal jailbreak attack method targeting visual language models according to claim 1, characterized in that: in, The initial adversarial text consists of multiple exclamation marks.

4. The multimodal jailbreak attack method targeting visual language models according to claim 1, characterized in that: in, In step S4, the image domain optimization problem in the second optimization objective is solved by the projection gradient descent method, and the text domain optimization problem in the second optimization objective is solved by the greedy coordinate gradient method, so as to obtain the adversarial image prefix and the adversarial text suffix.

5. A multimodal jailbreak attack device targeting a visual language model, used to perform jailbreak attacks on the visual language model through malicious queries to obtain corresponding highly toxic positive responses, characterized in that... A multimodal jailbreak attack method targeting a visual language model, as described in any one of claims 1 to 4, comprises: a model storage module for storing the visual language model; a universal key generation module for generating adversarial image prefixes and adversarial text suffixes based on the visual language model; and a malicious response generation module for inputting the adversarial image prefixes, the malicious query, and the adversarial text suffixes as a whole into the visual language model to obtain the highly toxic positive response. The universal key generation module comprises: an initial adversarial image generation unit for collecting random noise as an initial adversarial image; and an optimized adversarial image generation unit for using the initial adversarial image as input to the visual language model and optimizing the initial adversarial image according to a first optimization objective to obtain an optimized initial adversarial image. An adversarial image is used as an optimized adversarial image; an initial adversarial text generation unit is used to construct the initial adversarial text; a universal key generation unit stores a training dataset containing multiple malicious user queries, used to take the optimized adversarial image, the initial adversarial text, and the malicious user queries as inputs to the visual language model, and jointly optimize the optimized adversarial image and the initial adversarial text according to a second optimization objective, to obtain an optimized adversarial image as a prefix of the adversarial image and an optimized initial adversarial text as a suffix of the adversarial text. The first optimization objective is to increase the probability that the visual language model generates a highly toxic response based on the initial adversarial image, and the second optimization objective is to increase the probability that the visual language model generates a positive response based on the optimized adversarial image and the initial adversarial text.

Images