Prison break prompt word generation model training method and device, and prison break attack method and device

Abstract

The embodiment of the invention discloses a jailbreak prompt word generation model training method and device and a jailbreak attack method and device. According to the training method, the training data is automatically generated through prison break attack recognition detection, dependence on manual annotation is reduced, training cost is reduced, and meanwhile training efficiency is improved. Besides, the training method performs self-optimization in each detection and feedback by simulating an attack-detection-feedback closed-loop mechanism and combining a self-supervised learning strategy, so that the prison break cue word generation model can update a prison break strategy in real time and dynamically adapt to novel attacks. The jailbreak prompt word generation model training device, the jailbreak attack method and the jailbreak attack device in the embodiment of the specification also have the above beneficial effects.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a jailbreak prompt word generation model training, a jailbreak attack method and a device. Background Art

[0002] Current text-based graph models face the risk of jailbreak attacks, where attackers can trick the models into generating illegal content by inputting specific "jailbreak prompts." Current defenses against jailbreak attacks rely on static rules or single detection models, making them ineffective against the ever-changing and increasingly complex jailbreak attack vectors.

[0003] Therefore, the industry urgently needs attack testing tools that can effectively simulate different forms of jailbreak attacks in order to explore the security vulnerabilities of the cultural graph model and thus optimize the ability of the cultural graph model to resist jailbreak attacks. Summary of the Invention

[0004] One or more embodiments of this specification provide a jailbreak prompt word generation model training, jailbreak attack method and device, which can continuously upgrade attack strategies through a self-learning mechanism to adapt to the ever-changing jailbreak attack methods.

[0005] In a first aspect, a method for training a jailbreak prompt word generation model is provided, comprising:

[0006] Generate jailbreak prompt words using the pre-trained jailbreak prompt word generation model;

[0007] Inputting the jailbreak prompt word into a Wensheng graph model to obtain a generated image output by the Wensheng graph model;

[0008] performing jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images, respectively, constructing the jailbreak prompt words that successfully bypass the jailbreak attack recognition detection as positive preference samples, and constructing the jailbreak prompt words that fail to bypass the jailbreak attack recognition detection as negative preference samples;

[0009] The positive and negative preference samples are used to fine-tune the jailbreak prompt word generation model using a direct preference optimization method.

[0010] As an optional implementation of the method of the first aspect, generating jailbreak prompt words using a pre-trained jailbreak prompt word generation model specifically includes:

[0011] Get the topic prompt words;

[0012] The subject prompt word is input into the jailbreak prompt word generation model to obtain the jailbreak prompt word related to the subject prompt word generated by the jailbreak prompt word generation model.

[0013] As an optional implementation manner of the method of the first aspect, performing jailbreak attack identification and detection on the jailbreak prompt word and the corresponding generated image respectively includes:

[0014] Detecting the jailbreak prompt word using a pre-trained dangerous text language detection model to identify whether the jailbreak prompt word contains dangerous text; if the dangerous text language detection model does not detect that the jailbreak prompt word contains dangerous text, determining that the jailbreak prompt word has successfully bypassed the dangerous text language detection model;

[0015] The generated image is detected using a pre-trained image security detection model to identify whether the image is a prohibited image. If the image security detection model does not detect that the generated image is a prohibited image, it is determined that the generated image has successfully bypassed the image security detection model.

[0016] Specifically, the method further includes:

[0017] The generated image is detected using a pre-trained image content detection model to identify whether there are any illegal elements in the image. If the image content detection model does not detect the presence of illegal elements in the generated image, it is determined that the generated image has successfully bypassed the image content detection model.

[0018] As an optional implementation manner of the method of the first aspect, jailbreak attack recognition detection is performed on the jailbreak prompt word and the corresponding generated image, and the jailbreak prompt word that successfully bypasses the jailbreak attack recognition detection is constructed as a positive preference sample, and the jailbreak prompt word that does not bypass the jailbreak attack recognition detection is constructed as a negative preference sample, specifically including:

[0019] Using at least one preset jailbreak attack recognition and detection model to perform jailbreak attack recognition and detection on the jailbreak prompt word and the corresponding generated image respectively;

[0020] According to the number of jailbreak attack recognition and detection models bypassed by the jailbreak prompt words and the corresponding generated images, the jailbreak prompt words are given a preference score according to a preset scoring rule, and a preference label of the jailbreak prompt word is determined based on the scoring result.

[0021] As an optional implementation manner of the method of the first aspect, the method further includes: performing multiple rounds of iterative optimization on the fine-tuned jailbreak prompt word generation model until the jailbreak prompt word generation model meets preset requirements; wherein each round of iterative optimization includes:

[0022] Constructing the current round of topic prompts based on the positive preference samples used in the previous round of fine-tuning, and using the current round of topic prompts to guide the jailbreak prompt word generation model to generate the current round of jailbreak prompt words that tend to be positively preferred;

[0023] Inputting the current round of jailbreak prompt words into the Wensheng graph model to obtain a generated image output by the Wensheng graph model;

[0024] Perform the jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images in this round, and construct the jailbreak prompt words in this round that successfully bypass the jailbreak attack recognition detection as positive preference samples in this round, and construct the jailbreak prompt words in this round that fail to bypass the jailbreak attack recognition detection as negative preference samples in this round;

[0025] The positive and negative preference samples of this round are used to fine-tune the jailbreak prompt word generation model using a direct preference optimization method.

[0026] Specifically, based on the positive preference samples used in the previous round of fine-tuning, the topic prompt words for this round are constructed, including:

[0027] The positive preference samples used in the previous round of fine-tuning are sampled, and the sampling results are expanded to obtain the topic prompt words of this round.

[0028] In a second aspect, a jailbreak attack method is provided, including:

[0029] Using a jailbreak prompt word generation model to generate jailbreak prompt words for a target text graph model; the jailbreak prompt word generation model is trained using the jailbreak prompt word generation model training method described above;

[0030] The jailbreak prompt word is input into the target text graph model to perform a jailbreak attack.

[0031] In a third aspect, a jailbreak prompt word generation model training device is provided, comprising:

[0032] A first generation module is configured to generate a jailbreak prompt word using a pre-trained jailbreak prompt word generation model;

[0033] A first attack module is configured to input the jailbreak prompt word into a Wensheng graph model to obtain a generated image output by the Wensheng graph model;

[0034] a sample construction module configured to perform jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images, respectively, and construct the jailbreak prompt words that successfully bypass the jailbreak attack recognition detection as positive preference samples, and construct the jailbreak prompt words that fail to bypass the jailbreak attack recognition detection as negative preference samples;

[0035] The fine-tuning module is configured to use the positive and negative preference samples and adopt a direct preference optimization method to fine-tune the jailbreak prompt word generation model.

[0036] As an optional implementation manner of the apparatus according to the third aspect, the first generating module is specifically configured to:

[0037] Get the topic prompt words;

[0038] The subject prompt word is input into the jailbreak prompt word generation model to obtain the jailbreak prompt word related to the subject prompt word generated by the jailbreak prompt word generation model.

[0039] As an optional implementation manner of the apparatus according to the third aspect, the sample construction module is specifically configured to:

[0040] Detecting the jailbreak prompt word using a pre-trained dangerous text language detection model to identify whether the jailbreak prompt word contains dangerous text; if the dangerous text language detection model does not detect that the jailbreak prompt word contains dangerous text, determining that the jailbreak prompt word has successfully bypassed the dangerous text language detection model;

[0041] The generated image is detected using a pre-trained image security detection model to identify whether the image is a prohibited image. If the image security detection model does not detect that the generated image is a prohibited image, it is determined that the generated image has successfully bypassed the image security detection model.

[0042] Specifically, the sample construction module is further used to:

[0043] The generated image is detected using a pre-trained image content detection model to identify whether there are any illegal elements in the image. If the image content detection model does not detect the presence of illegal elements in the generated image, it is determined that the generated image has successfully bypassed the image content detection model.

[0044] As an optional implementation manner of the apparatus according to the third aspect, the sample construction module is specifically configured to:

[0045] Using at least one preset jailbreak attack recognition and detection model to perform jailbreak attack recognition and detection on the jailbreak prompt word and the corresponding generated image respectively;

[0046] According to the number of jailbreak attack recognition and detection models bypassed by the jailbreak prompt words and the corresponding generated images, the jailbreak prompt words are given a preference score according to a preset scoring rule, and a preference label of the jailbreak prompt word is determined based on the scoring result.

[0047] As an optional implementation manner of the apparatus of the third aspect, the fine-tuning module is further configured to perform multiple rounds of iterative optimization on the fine-tuned jailbreak prompt word generation model until the jailbreak prompt word generation model meets preset requirements; in each round of the iterative optimization, the fine-tuning module is configured to:

[0048] Constructing the current round of topic prompts based on the positive preference samples used in the previous round of fine-tuning, and using the current round of topic prompts to guide the jailbreak prompt word generation model to generate the current round of jailbreak prompt words that tend to be positively preferred;

[0049] Inputting the current round of jailbreak prompt words into the Wensheng graph model to obtain a generated image output by the Wensheng graph model;

[0050] Perform the jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images in this round, and construct the jailbreak prompt words in this round that successfully bypass the jailbreak attack recognition detection as positive preference samples in this round, and construct the jailbreak prompt words in this round that fail to bypass the jailbreak attack recognition detection as negative preference samples in this round;

[0051] The positive and negative preference samples of this round are used to fine-tune the jailbreak prompt word generation model using a direct preference optimization method.

[0052] Specifically, the fine-tuning module is specifically used to:

[0053] The positive preference samples used in the previous round of fine-tuning are sampled, and the sampling results are expanded to obtain the topic prompt words of this round.

[0054] In a fourth aspect, a jailbreak attack device is provided, comprising:

[0055] A second generation module is configured to generate a jailbreak prompt word for the target text graph model using a jailbreak prompt word generation model; the jailbreak prompt word generation model is trained using the jailbreak prompt word generation model training method described above;

[0056] The second attack module is configured to input the jailbreak prompt word into the target text graph model to perform a jailbreak attack.

[0057] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is run on an electronic device, the electronic device executes the above-mentioned jailbreak prompt word generation model training method, or executes the above-mentioned jailbreak attack method.

[0058] According to a sixth aspect, an electronic device is provided, including:

[0059] at least one memory for storing a program;

[0060] At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the above-mentioned jailbreak prompt word generation model training method, or execute the above-mentioned jailbreak attack method.

[0061] The jailbreak prompt word generation model training method described in the embodiment of this specification has the beneficial effect of:

[0062] This training method automatically generates training data using jailbreak attack detection and identification, reducing reliance on manual annotation, lowering training costs while improving training efficiency. Furthermore, by simulating a closed-loop attack-detection-feedback mechanism and incorporating a self-supervised learning strategy to optimize itself during each detection and feedback, the jailbreak prompt word generation model can update its jailbreak strategy in real time and dynamically adapt to new attacks.

[0063] The jailbreak prompt word generation model training device, jailbreak attack method and jailbreak attack device described in the embodiments of this specification also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0065] Figure 1 The flowchart of a jailbreak prompt word generation model training method described in an embodiment of this specification is exemplarily shown.

[0066] Figure 2 The structure diagram of a multimodal joint detection jailbreak attack identification and detection framework described in the embodiments of this specification is exemplarily shown.

[0067] Figure 3 The structure diagram of another multimodal joint detection jailbreak attack identification and detection framework described in the embodiments of this specification is exemplarily shown.

[0068] Figure 4 The following is a schematic diagram showing the structure of a jailbreak prompt word generation model training device described in an embodiment of this specification.

[0069] Figure 5 The flowchart of a jailbreak attack method described in an embodiment of this specification is exemplarily shown.

[0070] Figure 6The structure diagram of a jailbreak attack device described in an embodiment of this specification is exemplarily shown.

[0071] Figure 7 The schematic structural diagram of an electronic device provided in an embodiment of this specification is exemplarily shown. DETAILED DESCRIPTION

[0072] First, it should be noted that the terms used in the embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. As used in the embodiments of the present invention and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise.

[0073] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Therefore, it should be recognized by those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0074] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0075] Current text-based graph models (such as Stable Diffusion and Midjourney) are widely used in fields such as image generation and artistic creation, but they also face the risk of generating illegal content. To address this issue, these models typically rely on security detection mechanisms to filter out potentially illegal content. These detection mechanisms review generated images based on predefined rules or algorithms. However, as attack methods continue to evolve, attackers can carefully craft specific "jailbreak prompts" (for example, prompts with special characters or hidden meanings) to bypass existing security detection and generate illegal content.

[0076] Current defenses for text-based graph models mostly rely on static rules or single detection models, making them ineffective against the ever-changing and increasingly complex jailbreak attack vectors. Traditional solutions typically rely on fixed thresholds, keyword matching, and other techniques. These technologies often have significant limitations when facing new attack strategies and are unable to adapt to the dynamic changes in attack vectors.

[0077] In light of this, one or more embodiments of this specification propose a jailbreak prompt word generation model training, jailbreak attack method, and device. Within this jailbreak prompt word generation model training method, a dynamic optimization framework based on self-learning is proposed. The core concept of this framework is to continuously optimize the jailbreak prompt word generation model's ability to simulate different forms of jailbreak attack methods through a closed-loop mechanism of simulated attacks, detection, and feedback. This allows for effective exploration of the security boundaries of the text graph model, and further, by optimizing the text graph model, it can improve the text graph model's ability to identify and defend against complex and covert attacks over the long term.

[0078] The jailbreak prompt word generation model training, jailbreak attack method and device described in one or more embodiments of this specification will be further described in detail below in conjunction with the drawings and specific embodiments of the specification, but this detailed description does not constitute a limitation on the embodiments of this specification.

[0079] To clearly illustrate the jailbreak prompt word generation model training, jailbreak attack method, and device proposed in one or more embodiments of this specification, the following technical terms involved in the embodiments are first explained:

[0080] Self-supervised learning: Training is done through data labels generated by the model itself, without the need for manual labeling.

[0081] Direct Preference Optimization (DPO): A reinforcement learning method that directly optimizes model policies based on comparative preference data.

[0082] Jailbreak prompt word generation model: a large language model used to generate prompt words (prompt) that bypass the security restrictions of the text-based graph model.

[0083] Please refer to Figure 1 , Figure 1 This is a flow chart of a jailbreak prompt word generation model training method proposed in one or more embodiments of this specification. Figure 1 As shown, the training method includes steps S100-S106.

[0084] S100: Generate jailbreak prompt words using a pre-trained jailbreak prompt word generation model.

[0085] Before using the jailbreak prompt word generation model to generate jailbreak prompt words, the jailbreak prompt word generation model needs to be pre-trained to obtain a jailbreak prompt word generation model with the ability to generate jailbreak prompt words.

[0086] The jailbreak prompt word generation model can be implemented using a natural language generation model, such as GPT, or other models with natural language processing capabilities, such as the Clip model. It should be noted that this embodiment does not limit the choice of the model structure of the jailbreak prompt word generation model.

[0087] During the pre-training phase, existing jailbreak hint words that have successfully bypassed jailbreak attack detection and their corresponding topic hint words can be collected as an initial sample set. This initial sample set is then used to pre-train the jailbreak hint word generation model, enabling the pre-trained jailbreak hint word generation model to generate jailbreak hint words based on the input topic hint words.

[0088] S102: Input the jailbreak prompt word into the text-based graph model to obtain a generated image output by the text-based graph model.

[0089] This embodiment does not limit the specific category and structure of the above-mentioned text-generated graph model. For example, the above-mentioned text-generated graph model may be a model such as Stable Diffusion and Midjourney that generates an image based on input text.

[0090] In some implementations, a topic prompt word can be designed according to needs, and then the topic prompt word is input into a jailbreak prompt word generation model to obtain a jailbreak prompt word generated by the jailbreak prompt word generation model that is related to the topic of the topic prompt word.

[0091] S104: performing jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images respectively, and constructing the jailbreak prompt words that successfully bypass the jailbreak attack recognition detection as positive preference samples, and constructing the jailbreak prompt words that fail to bypass the jailbreak attack recognition detection as negative preference samples.

[0092] The jailbreak attack identification and detection mentioned above refers to determining whether the input of the jailbreak prompt word is a jailbreak attack based on the jailbreak prompt word and the generated image generated by the jailbreak prompt word. The jailbreak attack identification and detection mentioned above can be implemented using an existing detection method or a combination of multiple existing detection methods.

[0093] In some embodiments, a jailbreak attack identification and detection solution based on multimodal (text + image) joint detection is provided. Figure 2 , Figure 2 A multi-modal joint detection framework for jailbreak attack identification and detection is shown. Figure 2As shown in FIG, the detection framework includes a dangerous text language detection model and an image safety detection model.

[0094] The dangerous text language detection model can be pre-trained using an existing Transformer or BERT-based security classification model. The goal of the dangerous text language detection model is to detect whether the jailbreak prompt words generated by the jailbreak prompt word generation model contain malicious or illegal content. The jailbreak prompt words are detected using the pre-trained dangerous text language detection model to identify whether the jailbreak prompt words contain dangerous text. If the dangerous text language detection model does not detect that the jailbreak prompt words contain dangerous text, it is determined that the jailbreak prompt words have successfully bypassed the dangerous text language detection model. Otherwise, it is determined that the jailbreak prompt words have not successfully bypassed the dangerous text language detection model.

[0095] The image safety detection model can be pre-trained using an existing image classification model or convolutional neural network model. The image safety detection model's detection objective is to identify the images generated by the text-based image model and determine whether they are prohibited images, such as those containing violent or pornographic content. The pre-trained image safety detection model is used to detect the generated images and determine whether they are prohibited images. If the image safety detection model does not detect the generated images as prohibited, the generated images are determined to have successfully bypassed the image safety detection model. Otherwise, the generated images are determined to have failed to bypass the image safety detection model.

[0096] In some embodiments, another jailbreak attack identification and detection solution based on multimodal joint detection is also provided. Figure 3 , Figure 3 Another multi-modal joint detection framework for jailbreak attack identification and detection is shown. Figure 3 As shown in Figure 1, the detection framework includes a dangerous text language detection model, an image safety detection model, and an image content detection model. Figure 3 Compared with the jailbreak attack identification and detection framework of the multimodal joint detection Figure 2 For example, an image content detection model is added, so the following only introduces the image content detection model. Figure 3 The dangerous text language detection model and image safety detection model in are not described in detail here.

[0097] The image content detection model can be implemented using an element segmentation model and classifier based on target detection. Among them, the element segmentation model can segment the elements contained in the generated image (such as text, patterns, logos, foreground, background, etc.), and the segmented element images are sent to the classifier for classification, so as to determine whether there are any illegal elements in the generated image. The image content detection model can also be trained to automatically check whether the generated image meets the given theme requirements. For example, the jailbreak prompt word requires the generation of an image related to the theme "healthy diet". After pre-training, the image content detection model can determine whether the input generated image correctly expresses the theme of "healthy diet", or whether elements unrelated to the theme of the jailbreak prompt word appear in the generated image.

[0098] The generated image is detected using a pre-trained image content detection model to identify whether there are any illegal elements in the generated image. If the image content detection model does not detect the presence of illegal elements in the generated image, it is determined that the generated image has successfully bypassed the image content detection model; otherwise, it is determined that the generated image has not successfully bypassed the image content detection model.

[0099] It should be noted that the above-mentioned dangerous text language detection model, image safety detection model and image content detection model can be adaptively selected according to needs, and this embodiment does not impose any restrictions on this.

[0100] After performing jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images respectively, the jailbreak prompt words that successfully bypass the jailbreak attack recognition detection can be constructed as positive preference samples, and the jailbreak prompt words that fail to bypass the jailbreak attack recognition detection can be constructed as negative preference samples.

[0101] In some embodiments, the jailbreak prompt words can be scored according to a preset scoring rule based on the number of jailbreak attack recognition and detection models bypassed by the jailbreak prompt words and the corresponding generated images, and the preferred label of the jailbreak prompt word can be determined based on the scoring result. For example:

[0102] Jailbreak prompts that match the theme expressed by the theme prompt are scored 1 point.

[0103] If the jailbreak prompt word passes any jailbreak attack identification detection model (such as the dangerous text language detection model, the image security detection model, and the image content detection model), 2 points will be added to the jailbreak prompt word.

[0104] If the jailbreak prompt word passes all jailbreak attack identification and detection models, 5 points will be added to the jailbreak prompt word.

[0105] Finally, calculate the total score of this jailbreak clue word.

[0106] For the same jailbreak clue word, we can perform image generation and jailbreak attack recognition tests dozens of times, recording the scores for each. By calculating the expected score for the jailbreak clue word based on the results of multiple experiments (for example, we can take the average or weighted sum of the results), this helps to comprehensively evaluate the quality of the jailbreak clue word.

[0107] When constructing positive and negative preference samples, different jailbreak cue words within the same topic can be sorted in ascending order based on their scores. The dataset can be constructed as (Topic Cue Word 1: Positive Preference Score a, Negative Preference Score b). This data construction method can help the jailbreak cue word generation model better understand the pros and cons of different jailbreak cue words within the same topic and improve its generation capabilities through optimization.

[0108] S106: Using positive and negative preference samples, a direct preference optimization method is used to fine-tune the jailbreak prompt word generation model.

[0109] Construct preference optimization loss function:

[0110]

[0111] Among them, π ref represents the generation strategy of the jailbreak prompt word generation model before fine-tuning, π θ represents the generation strategy of the jailbreak prompt word generation model after fine-tuning. σ represents the logic function, and β is the constraint term used to control the generation strategy after each fine-tuning not to deviate too far from the generation strategy before fine-tuning. x represents the topic prompt word, y w Indicates positive preference samples, y l Represents a negative preference sample.

[0112] Based on the aforementioned preference optimization loss function, the policy parameter θ is optimized using gradient descent. This allows the jailbreak prompt word generation model to learn how to generate more appropriate jailbreak prompt words for a given topic, thereby bypassing existing security detection mechanisms.

[0113] In some implementations, the fine-tuned jailbreak prompt word generation model may be subjected to multiple rounds of iterative optimization to further improve the generation quality of the jailbreak prompt word generation model. Each round of iterative optimization may include:

[0114] Based on the positive preference samples used in the previous round of fine-tuning, construct the current round of theme prompt words, and use the current round of theme prompt words to guide the jailbreak prompt word generation model to generate the current round of jailbreak prompt words that tend to be positively preferred;

[0115] Input the jailbreak prompt words of this round into the Wensheng graph model to obtain the generated image output by the Wensheng graph model;

[0116] Perform jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images in this round, and construct the jailbreak prompt words in this round that successfully bypass the jailbreak attack recognition detection as the positive preference samples in this round, and construct the jailbreak prompt words in this round that fail to bypass the jailbreak attack recognition detection as the negative preference samples in this round;

[0117] Using the positive and negative preference samples from this round, the direct preference optimization method is used to fine-tune the jailbreak prompt word generation model.

[0118] When constructing the topic cues for this round, we can sample the positively biased samples used in the previous round of fine-tuning and expand upon these samples, resulting in more diverse and targeted topic cues for this round. This sampling strategy can help expand the exploration scope of the jailbreak cues generation model and generate cues of varying styles and complexity.

[0119] In summary, steps S100-S106 illustrate the jailbreak prompt word generation model training method. The above training method has the following advantages:

[0120] By simulating the closed-loop mechanism of attack-detection-feedback and combining it with a self-supervised learning strategy, the large model for generating jailbreak prompt words can dynamically adapt to new attacks and self-optimize in each detection and feedback.

[0121] We employ a multi-layered detection mechanism, combining text detection, post-image generation security testing, and content compliance testing to comprehensively review jailbreak warning words and generated images at different levels. This multimodal detection mechanism improves the ability to identify complex and illegal content, thereby improving the accuracy of the constructed preference sample data, prompting the jailbreak warning word generation model to generate higher-quality jailbreak warning words, and further exploring the security boundaries of the text-image model.

[0122] Through multiple rounds of iterative optimization, new jailbreak prompts are continuously generated to maximize the coverage of the attack space. This iterative optimization method helps the jailbreak prompt generation model effectively generate jailbreak prompts of varying complexity and styles, allowing it to effectively simulate attackers' ever-innovative jailbreak strategies.

[0123] Corresponding to the above-mentioned jailbreak prompt word generation model training method, one or more embodiments of this specification propose a jailbreak prompt word generation model training device. Figure 4 , Figure 4 The schematic diagram of the structure of a jailbreak prompt word generation model training device proposed in one or more embodiments of this specification is given schematically. It should be noted that the jailbreak prompt word generation model training method can rely on Figure 4The jailbreak prompt word generation model training device shown is implemented, but not limited to this device.

[0124] like Figure 4 As shown, the jailbreak prompt word generation model training device includes:

[0125] The first generation module 401 is configured to generate jailbreak prompt words using a pre-trained jailbreak prompt word generation model.

[0126] The first attack module 402 is configured to input the jailbreak prompt word into the Wensheng graph model to obtain a generated image output by the Wensheng graph model.

[0127] The sample construction module 403 is configured to perform jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images respectively, and construct the jailbreak prompt words that successfully bypass the jailbreak attack recognition detection as positive preference samples, and construct the jailbreak prompt words that fail to bypass the jailbreak attack recognition detection as negative preference samples.

[0128] The fine-tuning module 404 is configured to use the positive and negative preference samples and adopt a direct preference optimization method to fine-tune the jailbreak prompt word generation model.

[0129] Regarding the above-mentioned first generating module 401, the first generating module 401 is specifically configured to:

[0130] A topic prompt word is obtained, and then the topic prompt word is input into a jailbreak prompt word generation model to obtain a jailbreak prompt word related to the topic prompt word generated by the jailbreak prompt word generation model.

[0131] It should be noted that before the first generation module 401 uses the pre-trained jailbreak prompt word generation model to generate the jailbreak prompt word, it is also necessary to load the pre-trained jailbreak prompt word generation model. The pre-training method of the jailbreak prompt word generation model has been described in the above-mentioned jailbreak prompt word generation model training method and will not be repeated here.

[0132] Regarding the above-mentioned first attack module 402, the first attack module 402 can obtain the theme prompt words designed by the user according to the needs, and then input the theme prompt words into the jailbreak prompt word generation model to obtain the jailbreak prompt words generated by the jailbreak prompt word generation model and related to the theme of the theme prompt words.

[0133] Regarding the above-mentioned sample construction module 403, the sample construction module 403 can be implemented as follows: Figure 2 The multimodal joint detection jailbreak attack recognition and detection framework shown performs jailbreak attack recognition and detection on jailbreak prompt words and corresponding generated images respectively.

[0134] Specifically, the sample construction module 403 can use a pre-trained dangerous text language detection model to detect the jailbreak prompt word to identify whether the jailbreak prompt word contains dangerous text. If the dangerous text language detection model does not detect that the jailbreak prompt word contains dangerous text, it is determined that the jailbreak prompt word has successfully bypassed the dangerous text language detection model. Otherwise, it is determined that the jailbreak prompt word has not successfully bypassed the dangerous text language detection model. The sample construction module 403 can use a pre-trained picture security detection model to detect the generated image to identify whether the generated image is a prohibited image. If the picture security detection model does not detect that the generated image is a prohibited image, it is determined that the generated image has successfully bypassed the picture security detection model. Otherwise, it is determined that the generated image has not successfully bypassed the picture security detection model.

[0135] The sample construction module 403 can also be used as follows Figure 3 The multimodal joint detection jailbreak attack recognition and detection framework shown performs jailbreak attack recognition and detection on jailbreak prompt words and corresponding generated images respectively.

[0136] Specifically, the sample construction module 403 can use a pre-trained dangerous text language detection model to detect the jailbreak prompt word to identify whether the jailbreak prompt word contains dangerous text. If the dangerous text language detection model does not detect that the jailbreak prompt word contains dangerous text, it is determined that the jailbreak prompt word has successfully bypassed the dangerous text language detection model. Otherwise, it is determined that the jailbreak prompt word has not successfully bypassed the dangerous text language detection model. The sample construction module 403 can use a pre-trained picture security detection model to detect the generated image to identify whether the generated image is a prohibited image. If the picture security detection model does not detect that the generated image is a prohibited image, it is determined that the generated image has successfully bypassed the picture security detection model. Otherwise, it is determined that the generated image has not successfully bypassed the picture security detection model. The sample construction module 403 can use a pre-trained picture content detection model to detect the generated image to identify whether there are illegal elements in the generated image. If the picture content detection model does not detect that there are illegal elements in the generated image, it is determined that the generated image has successfully bypassed the picture content detection model. Otherwise, it is determined that the generated image has not successfully bypassed the picture content detection model.

[0137] After performing jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images respectively, the sample construction module 403 can construct the jailbreak prompt words that successfully bypass the jailbreak attack recognition detection as positive preference samples, and construct the jailbreak prompt words that fail to bypass the jailbreak attack recognition detection as negative preference samples.

[0138] In some embodiments, the sample construction module 403 can use at least one preset jailbreak attack recognition and detection model to perform jailbreak attack recognition and detection on the jailbreak prompt word and the corresponding generated image. The sample construction module 403 can assign a preference score to the jailbreak prompt word according to a preset scoring rule based on the number of jailbreak attack recognition and detection models bypassed by the jailbreak prompt word and the corresponding generated image, and determine a preference label for the jailbreak prompt word based on the scoring result. For example:

[0139] Jailbreak prompts that match the theme expressed by the theme prompt are scored 1 point.

[0140] If the jailbreak prompt word passes any jailbreak attack identification detection model (such as the dangerous text language detection model, the image security detection model, and the image content detection model), 2 points will be added to the jailbreak prompt word.

[0141] If the jailbreak prompt word passes all jailbreak attack identification and detection models, 5 points will be added to the jailbreak prompt word.

[0142] Finally, calculate the total score of this jailbreak clue word.

[0143] For the same jailbreak warning word, the sample construction module 403 can perform image generation and jailbreak attack recognition detection dozens of times, and record the score each time. Based on the results of multiple experiments, the expected score of the jailbreak warning word is calculated (for example, the average or weighted sum of the multiple experimental results can be taken), which helps to comprehensively evaluate the quality of the jailbreak warning word.

[0144] When constructing the positive and negative preference samples, the sample construction module 403 can arrange different prompt words under the same theme in ascending order based on the scores of the jailbreak prompt words. The data set is constructed in the form of (theme prompt word 1: positive preference score a, negative preference score b). This data construction method can help the jailbreak prompt word generation model better understand the advantages and disadvantages of different jailbreak prompt words under the same theme and improve generation capabilities through the optimization process.

[0145] For the above-mentioned fine-tuning module 404, the fine-tuning module 404 can construct a preference optimization loss function:

[0146]

[0147] Among them, π ref represents the generation strategy of the jailbreak prompt word generation model before fine-tuning, π θ represents the generation strategy of the jailbreak prompt word generation model after fine-tuning. σ represents the logic function, and β is the constraint term used to control the generation strategy after each fine-tuning not to deviate too far from the generation strategy before fine-tuning. x represents the topic prompt word, y w Indicates positive preference samples, y lRepresents a negative preference sample.

[0148] The fine-tuning module 404 optimizes the policy parameter θ by gradient descent based on the above-mentioned preference optimization loss function, so that the jailbreak prompt word generation model can learn how to generate more appropriate jailbreak prompt words under a given topic to achieve the purpose of bypassing the existing security detection mechanism.

[0149] In some embodiments, the fine-tuning module 404 may further perform multiple rounds of iterative optimization on the fine-tuned jailbreak prompt word generation model to further improve the generation quality of the jailbreak prompt word generation model. Each round of iterative optimization performed by the fine-tuning module 404 may include:

[0150] Based on the positive preference samples used in the previous round of fine-tuning, construct the current round of theme prompt words, and use the current round of theme prompt words to guide the jailbreak prompt word generation model to generate the current round of jailbreak prompt words that tend to be positively preferred;

[0151] Input the jailbreak prompt words of this round into the Wensheng graph model to obtain the generated image output by the Wensheng graph model;

[0152] Perform jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images in this round, and construct the jailbreak prompt words in this round that successfully bypass the jailbreak attack recognition detection as the positive preference samples in this round, and construct the jailbreak prompt words in this round that fail to bypass the jailbreak attack recognition detection as the negative preference samples in this round;

[0153] Using the positive and negative preference samples from this round, the direct preference optimization method is used to fine-tune the jailbreak prompt word generation model.

[0154] When constructing the current round of topic prompts, the fine-tuning module 404 can sample the positive preference samples used in the previous round of fine-tuning and expand the sampled results to obtain more diverse and targeted topic prompts for this round. This sampling strategy can help expand the exploration scope of the jailbreak prompt generation model and generate jailbreak prompts of different styles and complexities.

[0155] For the above-mentioned jailbreak prompt word generation model training device, taking the module as an example of a software functional unit, the first generation module 401 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the first generation module 401 may include code running on multiple hosts / virtual machines / containers. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, and each AZ includes one data center or multiple data centers with close geographical locations. Among them, usually a region can include multiple AZs.

[0156] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0157] As an example of a hardware functional unit, the first generation module 401 may include at least one computing device, such as a server. Alternatively, the first generation module 401 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0158] The multiple computing devices included in the first generation module 401 can be distributed in the same region or in different regions. The multiple computing devices included in the first generation module 401 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the first generation module 401 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0159] In other embodiments, the first generation module 401 can be used to execute any step in the above-mentioned jailbreak prompt word generation model training method, the first attack module 402 can be used to execute any step in the above-mentioned jailbreak prompt word generation model training method, the sample construction module 403 can be used to execute any step in the above-mentioned jailbreak prompt word generation model training method, and the fine-tuning module 404 can be used to execute any step in the above-mentioned jailbreak prompt word generation model training method. The steps implemented by the first generation module 401, the first attack module 402, the sample construction module 403, and the fine-tuning module 404 can be specified as needed. The first generation module 401, the first attack module 402, the sample construction module 403, and the fine-tuning module 404 respectively implement different steps in the above-mentioned jailbreak prompt word generation model training method to achieve the full functions of the above-mentioned jailbreak prompt word generation model training device.

[0160] In this implementation, the jailbreak prompt word generation model training device can also be applied to computing devices such as computers and servers, or to a computing device cluster including at least one computing device, to realize the specific functions of the jailbreak prompt word generation model training device.

[0161] Based on the above jailbreak prompt word generation model training method, one or more embodiments of this specification also propose a jailbreak attack method. Figure 5 , the method includes steps S500 to S502.

[0162] S500: Generate jailbreak prompt words for the target text graph model using a jailbreak prompt word generation model.

[0163] The jailbreak prompt word generation model is trained using the jailbreak prompt word generation model training method, which will not be described in detail in this step.

[0164] When using the above-mentioned jailbreak prompt word generation model to generate jailbreak prompt words for the target text graph model, the subject prompt words input by the user can be obtained, and then the subject prompt words can be input into the jailbreak prompt word generation model to obtain the jailbreak prompt words for the target text graph model generated by the jailbreak prompt word generation model.

[0165] S502: Inputting jailbreak prompt words into the target text graph model to perform a jailbreak attack.

[0166] The jailbreak prompt word is input into the target text-graph model to obtain a generated image generated by the target text-graph model.

[0167] Using the jailbreak attack identification and detection model deployed on the target cultural graph model side or the jailbreak attack identification and detection model set separately from the target cultural graph model, the generated image is subjected to jailbreak attack identification and detection, and the detection results and the corresponding jailbreak prompt words are recorded as the attack record of this jailbreak attack.

[0168] Using these attack records as sample data to optimize the target text graph model can improve the target text graph model's ability to resist jailbreak attacks.

[0169] Corresponding to the above jailbreak attack method, one or more embodiments of this specification propose a jailbreak attack device. Figure 6 , Figure 6 The schematic diagram of the structure of a jailbreak attack device proposed in one or more embodiments of this specification is given schematically. It should be noted that the jailbreak attack method can rely on Figure 6 The jailbreak attack shown is implemented on a specific device, but is not limited to that device.

[0170] like Figure 6 As shown, the jailbreak attack device includes:

[0171] The second generating module 601 is configured to generate a jailbreak prompt word for a target text graph model using a jailbreak prompt word generation model.

[0172] The jailbreak prompt word generation model is trained using the jailbreak prompt word generation model training method, which will not be described in detail in this step.

[0173] When the second generation module 601 uses the above-mentioned jailbreak prompt word generation model to generate jailbreak prompt words for the target text graph model, it can obtain the topic prompt words input by the user, and then input the topic prompt words into the jailbreak prompt word generation model to obtain the jailbreak prompt words for the target text graph model generated by the jailbreak prompt word generation model.

[0174] The second attack module 602 is configured to input the jailbreak prompt word into the target text graph model to perform a jailbreak attack.

[0175] Specifically, the second attack module 602 inputs the jailbreak prompt word into the target text graph model to obtain a generated image generated by the target text graph model.

[0176] Next, the second attack module 602 can use the jailbreak attack identification and detection model deployed on the target cultural graph model side or the jailbreak attack identification and detection model set separately from the target cultural graph model to perform jailbreak attack identification and detection on the generated image, and record the detection results and the corresponding jailbreak prompt words as the attack record of this jailbreak attack.

[0177] Using these attack records as sample data to optimize the target text graph model can improve the target text graph model's ability to resist jailbreak attacks.

[0178] For the above-mentioned jailbreak attack device, taking a module as an example of a software functional unit, the second generation module 601 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the above-mentioned computing instance may be one or more. For example, the second generation module 601 may include code running on multiple hosts / virtual machines / containers. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Among them, usually a region may include multiple AZs.

[0179] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0180] As an example of a hardware functional unit, the second generation module 601 may include at least one computing device, such as a server. Alternatively, the second generation module 601 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0181] The multiple computing devices included in the second generation module 601 can be distributed in the same region or in different regions. The multiple computing devices included in the second generation module 601 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the second generation module 601 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0182] In other embodiments, the second generation module 601 can be used to execute any step in the above-mentioned jailbreak attack method, and the second attack module 602 can be used to execute any step in the above-mentioned jailbreak attack method. The steps that the second generation module 601 and the second attack module 602 are responsible for implementing can be specified as needed. The second generation module 601 and the second attack module 602 respectively implement different steps in the above-mentioned jailbreak attack method to achieve the full functionality of the above-mentioned jailbreak attack device.

[0183] In this implementation, the jailbreak attack device can also be applied to computing devices such as computers and servers, or to a computing device cluster including at least one computing device, to realize the specific functions of the jailbreak attack device.

[0184] In some embodiments, an electronic device is also provided. Figure 7 The electronic device includes a bus 701, a processor 702, a memory 703, and a communication interface 704. The processor 702, the memory 703, and the communication interface 704 communicate with each other via the bus 701. The electronic device can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the electronic device.

[0185] The bus 701 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus 701 may include a path for transmitting information between various components of the electronic device (eg, the processor 702, the memory 703, and the communication interface 704).

[0186] The processor 702 may include any one or more processors such as a CPU, a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0187] The memory 703 may include a volatile memory, such as a random access memory (RAM). The memory 703 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0188] The memory 703 stores executable program code, and the processor 702 executes the executable program code to implement the aforementioned jailbreak prompt word generation model training method, or to implement the aforementioned jailbreak attack method.

[0189] The communication interface 704 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the electronic device and other devices or a communication network.

[0190] One or more embodiments in this specification provide a computer-readable storage medium, which stores a computer program. When the computer program runs on an electronic device, the electronic device executes the above-mentioned jailbreak prompt word generation model training method, or executes the above-mentioned jailbreak attack method.

[0191] The computer-readable storage medium can be any available medium that can be stored by an electronic device, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the electronic device to execute the above-mentioned jailbreak prompt word generation model training method, or to execute the above-mentioned jailbreak attack method.

[0192] It should be understood that the structures illustrated in the embodiments of this specification do not constitute specific limitations on the systems of the embodiments of this specification. In other embodiments of the specification, the above-mentioned system may include more or fewer components than shown in the figure, or some components may be combined, some components may be separated, or the components may be arranged differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0193] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0194] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0195] It should be noted that the above examples are merely specific embodiments of the present invention. Obviously, the present invention is not limited to the above examples, and many similar variations are possible. All variations directly derived from or associating with the present invention by those skilled in the art are intended to fall within the scope of protection of the present invention.

Claims

1. A jailbreak prompt word generation model training method, comprising: Generate jailbreak prompt words using the pre-trained jailbreak prompt word generation model; Inputting the jailbreak prompt word into a Wensheng graph model to obtain a generated image output by the Wensheng graph model; performing jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images, respectively, constructing the jailbreak prompt words that successfully bypass the jailbreak attack recognition detection as positive preference samples, and constructing the jailbreak prompt words that fail to bypass the jailbreak attack recognition detection as negative preference samples; The positive and negative preference samples are used to fine-tune the jailbreak prompt word generation model using a direct preference optimization method.

2. The method according to claim 1, wherein generating a jailbreak prompt word using a pre-trained jailbreak prompt word generation model specifically comprises: Get the topic prompt words; The subject prompt word is input into the jailbreak prompt word generation model to obtain the jailbreak prompt word related to the subject prompt word generated by the jailbreak prompt word generation model.

3. The method according to claim 1, further comprising performing jailbreak attack identification and detection on the jailbreak prompt word and the corresponding generated image, specifically comprising: Detecting the jailbreak prompt word using a pre-trained dangerous text language detection model to identify whether the jailbreak prompt word contains dangerous text; if the dangerous text language detection model does not detect that the jailbreak prompt word contains dangerous text, determining that the jailbreak prompt word has successfully bypassed the dangerous text language detection model; The generated image is detected using a pre-trained image security detection model to identify whether the image is a prohibited image. If the image security detection model does not detect that the generated image is a prohibited image, it is determined that the generated image has successfully bypassed the image security detection model.

4. The method of claim 3, further comprising: The generated image is detected using a pre-trained image content detection model to identify whether there are any illegal elements in the generated image. If the image content detection model does not detect the presence of illegal elements in the generated image, it is determined that the generated image has successfully bypassed the image content detection model.

5. The method according to claim 1, wherein the jailbreak prompt word and the corresponding generated image are respectively subjected to jailbreak attack recognition detection, and the jailbreak prompt word that successfully bypasses the jailbreak attack recognition detection is constructed as a positive preference sample, and the jailbreak prompt word that fails to bypass the jailbreak attack recognition detection is constructed as a negative preference sample, specifically comprising: Using at least one preset jailbreak attack recognition and detection model to perform jailbreak attack recognition and detection on the jailbreak prompt word and the corresponding generated image respectively; According to the number of jailbreak attack recognition and detection models bypassed by the jailbreak prompt words and the corresponding generated images, the jailbreak prompt words are given a preference score according to a preset scoring rule, and a preference label of the jailbreak prompt word is determined based on the scoring result.

6. The method of claim 1, further comprising: The fine-tuned jailbreak prompt word generation model is subjected to multiple rounds of iterative optimization until the jailbreak prompt word generation model meets preset requirements; wherein each round of iterative optimization includes: Constructing the current round of topic prompts based on the positive preference samples used in the previous round of fine-tuning, and using the current round of topic prompts to guide the jailbreak prompt word generation model to generate the current round of jailbreak prompt words that tend to be positively preferred; Inputting the current round of jailbreak prompt words into the Wensheng graph model to obtain a generated image output by the Wensheng graph model; Perform the jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images in this round, and construct the jailbreak prompt words in this round that successfully bypass the jailbreak attack recognition detection as positive preference samples in this round, and construct the jailbreak prompt words in this round that fail to bypass the jailbreak attack recognition detection as negative preference samples in this round; The positive and negative preference samples of this round are used to fine-tune the jailbreak prompt word generation model using a direct preference optimization method.

7. The method according to claim 6, wherein the topic prompt words of this round are constructed based on the positive preference samples used in the previous round of fine-tuning, specifically comprising: The positive preference samples used in the previous round of fine-tuning are sampled, and the sampling results are expanded to obtain the topic prompt words of this round.

8. A jailbreak attack method comprising: Using the jailbreak prompt word generation model to generate jailbreak prompt words for the target text graph model; The jailbreak prompt word generation model is trained using the method described in any one of claims 1 to 7; The jailbreak prompt word is input into the target text graph model to perform a jailbreak attack.

9. A jailbreak prompt word generation model training device, comprising: A first generation module is configured to generate a jailbreak prompt word using a pre-trained jailbreak prompt word generation model; A first attack module is configured to input the jailbreak prompt word into a Wensheng graph model to obtain a generated image output by the Wensheng graph model; a sample construction module configured to perform jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images, respectively, and construct the jailbreak prompt words that successfully bypass the jailbreak attack recognition detection as positive preference samples, and construct the jailbreak prompt words that fail to bypass the jailbreak attack recognition detection as negative preference samples; The fine-tuning module is configured to use the positive and negative preference samples and adopt a direct preference optimization method to fine-tune the jailbreak prompt word generation model.

10. The apparatus according to claim 9, wherein the first generating module is specifically configured to: Get the topic prompt words; The subject prompt word is input into the jailbreak prompt word generation model to obtain the jailbreak prompt word related to the subject prompt word generated by the jailbreak prompt word generation model.

11. The apparatus according to claim 9, wherein the sample construction module is specifically configured to: Detecting the jailbreak prompt word using a pre-trained dangerous text language detection model to identify whether the jailbreak prompt word contains dangerous text; if the dangerous text language detection model does not detect that the jailbreak prompt word contains dangerous text, determining that the jailbreak prompt word has successfully bypassed the dangerous text language detection model; The generated image is detected using a pre-trained image security detection model to identify whether the image is a prohibited image. If the image security detection model does not detect that the generated image is a prohibited image, it is determined that the generated image has successfully bypassed the image security detection model.

12. The apparatus according to claim 11, wherein the sample construction module is further configured to: The generated image is detected using a pre-trained image content detection model to identify whether there are any illegal elements in the image. If the image content detection model does not detect the presence of illegal elements in the generated image, it is determined that the generated image has successfully bypassed the image content detection model.

13. The apparatus according to claim 9, wherein the sample construction module is specifically configured to: Using at least one preset jailbreak attack recognition and detection model to perform jailbreak attack recognition and detection on the jailbreak prompt word and the corresponding generated image respectively; According to the number of jailbreak attack recognition and detection models bypassed by the jailbreak prompt words and the corresponding generated images, the jailbreak prompt words are given a preference score according to a preset scoring rule, and a preference label of the jailbreak prompt word is determined based on the scoring result.

14. The apparatus of claim 9 , wherein the fine-tuning module is further configured to perform multiple rounds of iterative optimization on the fine-tuned jailbreak prompt word generation model until the jailbreak prompt word generation model meets preset requirements; in each round of iterative optimization, the fine-tuning module is configured to: Constructing the current round of topic prompts based on the positive preference samples used in the previous round of fine-tuning, and using the current round of topic prompts to guide the jailbreak prompt word generation model to generate the current round of jailbreak prompt words that tend to be positively preferred; Inputting the current round of jailbreak prompt words into the Wensheng graph model to obtain a generated image output by the Wensheng graph model; Perform the jailbreak attack recognition detection on the jailbreak prompt words and the corresponding generated images in this round, and construct the jailbreak prompt words in this round that successfully bypass the jailbreak attack recognition detection as positive preference samples in this round, and construct the jailbreak prompt words in this round that fail to bypass the jailbreak attack recognition detection as negative preference samples in this round; The positive and negative preference samples of this round are used to fine-tune the jailbreak prompt word generation model using a direct preference optimization method.

15. The apparatus according to claim 14, wherein the fine-tuning module is specifically configured to: The positive preference samples used in the previous round of fine-tuning are sampled, and the sampling results are expanded to obtain the topic prompt words of this round.

16. A jailbreak attack device, comprising: A second generation module is configured to generate a jailbreak prompt word for the target text graph model using the jailbreak prompt word generation model; The jailbreak prompt word generation model is trained using the method described in any one of claims 1 to 7; The second attack module is configured to input the jailbreak prompt word into the target text graph model to perform a jailbreak attack.

17. A computer-readable storage medium storing a computer program, wherein when the computer program is executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 7, or the method according to claim 8.

18. An electronic device comprising: at least one memory for storing a program; At least one processor is used to execute the program stored in the memory, when the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 7, or execute the method according to claim 8.

Images