Multimodal black-box attack method and device for large-scale visual language models
By constructing iteratively optimized attack graph-text pairs, the problems of unrealistic white-box attacks and semantic distortion in large-scale visual language models are solved, efficient multimodal black-box attacks are achieved, and the security performance of the model is improved.
Patent Information
- Application Number
- CN202411503191.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-25
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2044-10-25
AI Technical Summary
In existing technologies, white-box attack methods in large-scale visual language models are unrealistic in real scenarios and the semantics of adversarial images are easily distorted, making detection or defense difficult, and there is a lack of effective multimodal black-box attack methods.
A multimodal black-box attack method and device are provided. By constructing system prompt words and dialogue templates, iteratively optimized attack image-text pairs are generated. The historical data of the visual language model is analyzed and optimized to generate targeted attack text and images, reducing the risk of semantic distortion.
It achieves a higher jailbreak attack success rate in large-scale visual language models, improves the security performance of the model, detects and optimizes defense capabilities, and reduces the possibility of being detected or defended.
Smart Images
Figure CN119415728B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of AI security, and specifically relates to a multimodal black-box attack method and device for large-scale visual language models. Background Art
[0002] With the widespread adoption of OpenAI's GPT-4 and Google's Gemini, increasing attention has focused on large-scale visual-language models (VLMs) and their trustworthy deployment in real-world applications. While VLMs build on highly aligned large language models, the combination of text and visual modalities introduces new security vulnerabilities. Recent research demonstrates that VLMs are extremely vulnerable to jailbreak attacks. Through malicious jailbreak prompts, models can be tricked into outputting harmful content that they are otherwise restricted from. This makes these risks a critical concern for the secure deployment of VLMs.
[0003] Greshake et al. explored the effectiveness of manually injecting deceptive text into input images. In contrast, other studies have proposed more sophisticated adversarial attack methods in the image domain. Carlini et al., Bagdasaryan et al., and Bailey et al. fixed a target output chosen by the attacker and optimized the input image to increase the probability of generating it. Another white-box attack proposed by Qi et al. aims to find a universal adversarial visual input. Specifically, this attack no longer focuses on a specific output sentence, but instead attempts to maximize the probability of generating derogatory output from a corpus containing multiple harmful sample sentences. This strategy was inspired by Wallace et al., who also used a discrete search-based optimization algorithm to find a universal adversarial trigger that increases the probability of generating a small set of harmful sentences. Shayegani et al. proposed attacking publicly available visual encoders, such as CLIP used in multimodal models, thereby eliminating the need for full white-box access.
[0004] The above existing methods mainly focus on exploring the robustness under white-box attacks based on adversarial images. This strategy is based on full or partial white-box access to the model, which is often unrealistic in real scenarios. In addition, the semantics of adversarial images are often distorted, making them easier to detect or defend against. Summary of the Invention
[0005] The present invention is made to solve the above-mentioned problems, and its purpose is to provide a multimodal black-box attack method and device for large-scale visual language models.
[0006] The present invention provides a multimodal black-box attack method for a large visual language model, which is used to generate a jailbreak image-text pair for the target model according to the jailbreak target and the visual language model. The method comprises the following steps: step S1, constructing corresponding system prompt words and dialogue templates according to different attack strategies; step S2, for each attack strategy, constructing input data according to the corresponding system prompt words and dialogue templates combined with the jailbreak target, and inputting the input data into the visual language model to generate the first round of attack text prompts and attack image prompts; step S3, generating a corresponding attack image according to the attack image prompts through the existing text-based graph model, and using the attack image and the corresponding attack text prompts of the round as the attack graph. text pair; step S4, in the current round, input each attack image-text pair corresponding to the round into the target model respectively to obtain the corresponding response reply; step S5, for each attack strategy, add the corresponding attack image-text pair and response reply to the historical data; step S6, analyze the historical data corresponding to each attack strategy through the visual language model to determine whether the jailbreak attack is successful. If so, the attack image pair of the last round corresponding to the attack strategy is used as the jailbreak image-text pair. If not, execute step S7; step S7, for the attack strategy that fails to jailbreak, analyze the corresponding historical data through the visual language model, generate the attack text prompt and attack image prompt for the next round according to the analysis results, and execute step S3.
[0007] The multimodal black-box attack method for a large visual language model provided by the present invention may also have the following features: wherein the attack strategies include typesetting attacks, role-playing, and emotion manipulation.
[0008] The multimodal black box attack method for a large visual language model provided by the present invention may also have the following features: wherein, in step S7, the visual language model constructs a CoT thinking chain based on historical data to perform reasoning and obtain analysis results.
[0009] The present invention also provides a multimodal black box attack device for a large visual language model, which is used to generate a jailbreak image-text pair for the target model according to the jailbreak target and the visual language model, and has the following characteristics: a model storage module, an initial text image generation module, an attack image generation module, a reply generation module, a historical data addition module, a judgment module and an optimization module, wherein the model storage module is used to store the visual language model and the target model; the initial text image generation module stores corresponding system prompt words and dialogue templates constructed according to different attack strategies, and is used to construct input data for each attack strategy according to the corresponding system prompt words and dialogue templates combined with the jailbreak target, and input the input data into the visual language model to generate the first round of attack text prompts and attack image prompts; the attack image generation module includes an existing text-based graph model, which is used to generate a first round of attack text prompts and attack image prompts according to the attack image prompts Generate a corresponding attack image, and use the attack image and the corresponding attack text prompt of this round as an attack image-text pair; the reply generation module is used to input each attack image-text pair corresponding to this round into the target model in the current round to obtain the corresponding response reply; the historical data addition module is used to add the corresponding attack image-text pair and response reply to the historical data for each attack strategy; the judgment module is used to analyze the historical data corresponding to each attack strategy through the visual language model to determine whether the jailbreak attack is successful. If so, the attack image pair of the last round corresponding to the attack strategy is used as the jailbreak image-text pair. If not, the optimization module is executed; the optimization module is used to analyze the corresponding historical data of the attack strategy that failed to jailbreak through the visual language model, and generate the attack text prompt and attack image prompt for the next round based on the analysis results, and execute the attack image generation module.
[0010] Functions and effects of the invention
[0011] According to the multimodal black-box attack method and device for large-scale visual language models involved in the present invention, on the one hand, attack text prompts and attack image prompts are generated according to different attack strategies, and then iteratively optimized in combination with the response reply of the target model, thereby generating more targeted attack text prompts and attack image prompts; on the other hand, attack text prompts and attack image prompts are generated synchronously, and then the attack image prompts are converted into corresponding attack images through the text graph, so that the semantics of the attack image are not distorted, reducing the possibility of detection or defense. Therefore, the multimodal black-box attack method and device for large-scale visual language models of the present invention can generate jailbreak image-text pairs that have a good effect on visual language models, further detecting and improving the security performance of visual language models. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 This is a schematic diagram of the working principle of generating jailbreak image-text pairs in an embodiment of the present invention;
[0013] Figure 2 is a block diagram of a multimodal black box attack device according to an embodiment of the present invention;
[0014] Figure 3 4 is a flowchart of a multimodal black-box attack method for a large visual language model in an embodiment of the present invention. DETAILED DESCRIPTION
[0015] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following embodiments and accompanying drawings specifically illustrate the multimodal black-box attack method and device of the present invention for large-scale visual language models.
[0016] This embodiment provides a multimodal black-box attack device for a large visual language model, which is used to generate a jailbreak image-text pair for the target model based on the jailbreak target and the visual language model.
[0017] Figure 1 It is a schematic diagram of the working principle of generating jailbreak image-text pairs in an embodiment of the present invention.
[0018] like Figure 1 As shown, the large-scale visual language red team model, i.e., the visual language model, generates text prompts and image prompts, and converts the image prompts into images through the text-to-image model. Subsequently, the image and text prompts are input together into the target visual language model, i.e., the target model, to generate a target response. The large-scale visual language red team model then analyzes the target response and further optimizes and generates new text prompts and image prompts. This cycle is iterated until the preset termination condition is reached, and the images corresponding to the finally optimized text prompts and image prompts are used as jailbreak text-image pairs.
[0019] Figure 2 4 is a block diagram of a multimodal black box attack device in an embodiment of the present invention.
[0020] like Figure 2 As shown, the multimodal black-box attack device 100 for a large visual language model includes a model storage module 11, an initial text image generation module 12, an attack image generation module 13, a reply generation module 14, a historical data addition module 15, a judgment module 16, an optimization module 17, and a control module 18 for controlling the operation of the above modules.
[0021] The model storage module 11 is used to store the visual language model and the target model.
[0022] The initial text and image generation module 12 stores system prompts and dialogue templates corresponding to different attack strategies. It constructs input data for each attack strategy based on the corresponding system prompts and dialogue templates, combined with the jailbreak target. This input data is then fed into the visual language model to generate the first round of attack text prompts (i.e., text prompts) and attack image prompts (i.e., image prompts). Attack strategies include typographic attacks, role-playing, and emotional manipulation.
[0023] In this embodiment, the initial text image generation module 12 generates the attack text prompt and attack image prompt according to each attack strategy as follows:
[0024]
[0025] Where M A is a visual language model, For image input is empty, For jailbreak goal, The analysis of the previous round of attacks is empty. This is the text prompt for the first round of attack. This is the image prompt for the first round of attacks.
[0026] The attack image generation module 13 includes an existing text-graph model, and is configured to generate a corresponding attack image according to the attack image prompt, and use the attack image and the corresponding attack text prompt of the round as an attack image-text pair.
[0027] The reply generation module 14 is used to input each attack graph-text pair corresponding to the current round into the target model respectively, and obtain a corresponding response reply, namely a target response.
[0028] The expression for generating the response reply in this embodiment is:
[0029]
[0030] In the formula is the response reply corresponding to the nth round, M V is the target model, I n is the attack image corresponding to the nth round, The attack text prompt corresponding to the nth round.
[0031] The historical data adding module 15 is used to add the corresponding attack image-text pair and response reply to the historical data for each attack strategy.
[0032] The judgment module 16 is used to analyze the historical data corresponding to each attack strategy through the visual language model to determine whether the jailbreak attack is successful. If so, the last round of attack image pairs corresponding to the attack strategy are used as jailbreak image-text pairs. If not, the optimization module 17 is executed.
[0033] Optimization module 17 analyzes the corresponding historical data for unsuccessful jailbreak attack strategies using the visual language model. Based on the analysis results, it generates the next round of attack text and image prompts, and then executes attack image generation module 13. The visual language model constructs a CoT (Coordinate of Thought) chain based on the historical data and performs reasoning to obtain the analysis results. In this embodiment, for any unsuccessful jailbreak attack strategy, the visual language model optimizes the previous round of attack text and image prompts within the scope of that attack strategy based on the analysis results, generating the next round of attack text and image prompts.
[0034] In this embodiment, the optimization module 17 generates the expression for the next round of attack text prompts and attack image prompts as follows:
[0035]
[0036] Where I n-1 is the attack image corresponding to the n-1th round, is the response reply corresponding to the n-1th round, is the visual language model M A The results of the analysis of historical data, The attack text prompt corresponding to the nth round, It is the attack image prompt corresponding to the nth round.
[0037] The control module 18 stores a control program for controlling the operation of each module.
[0038] The following describes the process of using the multimodal black box attack method device 100 to perform a multimodal black box attack method on a large visual language model with reference to the accompanying drawings.
[0039] Figure 3 4 is a flowchart of a multimodal black-box attack method for a large visual language model in an embodiment of the present invention.
[0040] like Figure 3 As shown in Figure 2, the multimodal black-box attack method for large-scale visual language models includes the following steps:
[0041] Step S1: construct corresponding system prompt words and dialogue templates according to different attack strategies and set them in the initial text image generation module 12.
[0042] In step S2, the initial text image generation module 12 constructs input data for each attack strategy according to the corresponding system prompt words and dialogue template combined with the jailbreak target, and inputs the input data into the visual language model to generate the first round of attack text prompts and attack image prompts.
[0043] In step S3, the attack image generation module 13 is used to generate a corresponding attack image according to the attack image prompt, and the attack image and the attack text prompt corresponding to the round are used as an attack image-text pair.
[0044] In step S4, the reply generation module 14 is used to input the attack graph-text pairs corresponding to the current round into the target model to obtain corresponding response replies.
[0045] In step S5, the historical data adding module 15 adds the corresponding attack graph-text pair and response reply to the historical data for each attack strategy.
[0046] In step S6, the judgment module 16 analyzes the historical data corresponding to each attack strategy through the visual language model to determine whether the jailbreak attack is successful. If so, the last round of attack image pairs corresponding to the attack strategy are used as jailbreak image-text pairs. If not, step S7 is executed.
[0047] In step S7, the optimization module 17 analyzes the corresponding historical data of the unsuccessful jailbreak attack strategy through the visual language model, generates the next round of attack text prompts and attack image prompts based on the analysis results, and executes step S3.
[0048] In this embodiment, a multimodal black-box attack method targeting large visual language models was used to jailbreak the existing large visual language model MiniGPT-4, achieving a 94% attack success rate. Furthermore, this method was used to perform migration attacks on the existing large models LLaVA and InstructBLIP, achieving success rates of 82% and 88%, respectively. This demonstrates that this embodiment's multimodal black-box attack method targeting large visual language models has excellent jailbreak attack performance.
[0049] In this embodiment, the jailbreak image-text pairs generated by the multimodal black-box attack method for large visual language models can be used to detect the target model's defense capability against jailbreak attacks, and then the target model can be optimized in combination with the jailbreak image-text pairs to improve the security performance of the target model.
[0050] Functions and Effects of the Embodiments
[0051] According to the multimodal black-box attack method and device for large-scale visual language models involved in this embodiment, on the one hand, attack text prompts and attack image prompts are generated according to different attack strategies, and then iteratively optimized in combination with the response of the target model to generate more targeted attack text prompts and attack image prompts. On the other hand, attack text prompts and attack image prompts are generated simultaneously, and then the attack image prompts are converted into corresponding attack images through the text-generated graph, so that the semantics of the attack images are not distorted, reducing the possibility of detection or defense. In short, this method can generate jailbreak image-text pairs that have a good effect on visual language models, further detecting and improving the security performance of visual language models.
[0052] Those skilled in the art will appreciate that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A multimodal black-box attack method for a large visual language model, which is used to generate a jailbreak image-text pair for the target model based on the jailbreak target and the visual language model, characterized in that: The following steps are involved: Step S1: construct corresponding system prompt words and dialogue templates according to different attack strategies; Step S2: for each attack strategy, construct input data based on the corresponding system prompt word and the dialogue template in combination with the jailbreak target, and input the input data into the visual language model to generate the first round of attack text prompts and attack image prompts; Step S3: Generate a corresponding attack image based on the attack image prompt using an existing text-based graph model, and use the attack image and the attack text prompt corresponding to the round as an attack image-text pair; Step S4: in the current round, input each of the attack graph-text pairs corresponding to the current round into the target model to obtain a corresponding response reply; Step S5: for each attack strategy, adding the corresponding attack image-text pair and the response reply to historical data; Step S6, analyzing the historical data corresponding to each attack strategy using the visual language model to determine whether the jailbreak attack is successful. If so, the attack image pair of the last round corresponding to the attack strategy is used as the jailbreak image-text pair. If not, executing step S7; Step S7: For the attack strategy that failed to jailbreak, the corresponding historical data is analyzed using the visual language model, and the attack text prompt and the attack image prompt for the next round are generated according to the analysis results, and step S3 is executed.
2. The multimodal black-box attack method for a large visual language model according to claim 1, characterized in that: in, Said attack strategies include typographical attacks, role-playing, and emotional manipulation.
3. The multimodal black-box attack method for a large visual language model according to claim 1, characterized in that: in, In step S7, the visual language model constructs a CoT thinking chain based on the historical data to perform reasoning to obtain the analysis result.
4. A multimodal black-box attack device for a large visual language model, which is used to generate a jailbreak image-text pair for the target model based on the jailbreak target and the visual language model, characterized in that: include: Model storage module, initial text image generation module, attack image generation module, reply generation module, historical data addition module, judgment module and optimization module, Wherein, the model storage module is used to store the visual language model and the target model; The initial text image generation module stores corresponding system prompt words and dialogue templates constructed according to different attack strategies, and is used to construct input data for each attack strategy based on the corresponding system prompt words and dialogue templates combined with the jailbreak target, and input the input data into the visual language model to generate the first round of attack text prompts and attack image prompts; The attack image generation module includes an existing text-based graph model, which is used to generate a corresponding attack image according to the attack image prompt, and use the attack image and the attack text prompt corresponding to the round as an attack image-text pair; The reply generation module is used to input each of the attack graph-text pairs corresponding to the current round into the target model to obtain a corresponding response reply; The historical data adding module is used to add the corresponding attack image-text pair and the response reply to the historical data for each attack strategy; The judgment module is used to analyze the historical data corresponding to each attack strategy through the visual language model to determine whether the jailbreak attack is successful. If so, the attack image pair of the last round corresponding to the attack strategy is used as the jailbreak image-text pair; if not, the optimization module is executed; The optimization module is used to analyze the corresponding historical data for the unsuccessful jailbreak attack strategy through the visual language model, generate the next round of attack text prompts and attack image prompts based on the analysis results, and execute the attack image generation module.