Jailbreak prompt generation method, device, storage medium and electronic device

Abstract

The embodiment of the present application discloses a jailbreak prompt generation method, device, storage medium and electronic device, the method comprising: determining a jailbreak scenario construction strategy, obtaining a prompt template format and each feature sentence library corresponding to the jailbreak scenario construction strategy, selecting target feature sentences belonging to different feature types from the feature sentence library, so that the jailbreak prompt generation process can be performed based on each target feature sentence and the prompt template format to obtain an initial jailbreak prompt, performing jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt, and performing jailbreak prompt evaluation processing based on the target jailbreak prompt using at least one test large model. Thus, the embodiment of the present application achieves the effect of intelligently generating a relatively high-quality universal jailbreak prompt.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a jailbreak prompt generation method, device, storage medium and electronic device. Background Art

[0002] In related technologies, Large Language Models (LLM), such as ChatGPT, Gemini, and Claude, have demonstrated impressive capabilities in natural language understanding and generation, which provides important support for content creation, education, decision-making and other fields. However, the widespread application of LLM has also raised concerns about the robustness and security of the model. LLM provides detailed guidance on harmful or illegal activities in some cases. Although model developers have adopted security measures such as security alignment and content filtering, the model is still vulnerable to certain adversarial strategies. Among them, the most notable type of adversarial strategy is jailbreak attack, which can bypass the security mechanism of LLM through carefully designed prompts, thereby guiding LLM to generate harmful, discriminatory, violent or sensitive content. As the performance and application scope of LLM continue to improve, managing its abuse risks becomes increasingly critical. Summary of the invention

[0003] The embodiment of the present application provides a jailbreak prompt generation method, device, computer storage medium and electronic device. The technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a method for generating a jailbreak prompt, the method comprising:

[0005] Determine a jailbreak scenario construction strategy, and obtain a prompt template format and feature sentence libraries corresponding to the jailbreak scenario construction strategy;

[0006] Selecting target feature sentences corresponding to a plurality of feature types from the feature sentence library, and performing jailbreak prompt generation processing based on each of the target feature sentences and the prompt template format to obtain an initial jailbreak prompt;

[0007] The initial jailbreak prompt is subjected to jailbreak prompt adjustment processing to obtain a target jailbreak prompt, and based on the target jailbreak prompt, at least one test large model is used to perform jailbreak prompt evaluation processing.

[0008] In some possible implementations, the jailbreak prompt generation process is performed based on each of the target feature sentences and the prompt template format to obtain an initial jailbreak prompt, including:

[0009] Performing language conversion processing based on each of the target feature sentences to obtain each variant feature sentence;

[0010] A jailbreak prompt generation process is performed based on the variation feature sentence and the prompt template format to obtain an initial jailbreak prompt.

[0011] In some possible implementations, the language conversion process is performed based on each of the target feature sentences to obtain each variant feature sentence, including:

[0012] Determining a feature word to be mutated in the target feature sentence, and determining a variant language corresponding to the feature word to be mutated;

[0013] In the target feature sentence, the feature words to be mutated are subjected to language variation processing by using the variant language to obtain a variant feature sentence.

[0014] In some possible implementations, performing jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt includes:

[0015] A prompt adjustment strategy for the initial jailbreak prompt is determined, and the prompt adjustment strategy is used to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt.

[0016] In some possible implementations, the adopting the prompt adjustment strategy to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt includes:

[0017] Get a large model fine-tuned for attacking jailbreak prompt generation scenarios;

[0018] Obtain the jailbreak prompt adjustment prompt word corresponding to the prompt adjustment strategy;

[0019] The jailbreak prompt adjustment prompt word and the initial jailbreak prompt are input into the attack fine-tuning large model to perform jailbreak prompt adjustment processing to obtain a target jailbreak prompt.

[0020] In some possible implementations, the using at least one test large model to perform jailbreak prompt evaluation processing based on the target jailbreak prompt includes:

[0021] Determine the attack success rate of the target jailbreak prompt in at least one test large model, and determine the number of targets whose attack success rate is greater than a success rate threshold;

[0022] If the number of targets is greater than a preset threshold, outputting a target jailbreak prompt;

[0023] If the target number is less than a preset threshold, the steps of determining the jailbreak scenario construction strategy and obtaining the prompt template format and each feature sentence library corresponding to the jailbreak scenario construction strategy are executed.

[0024] In some possible implementations, after outputting the target jailbreak prompt, the method further includes:

[0025] The target jailbreak prompt is saved to a jailbreak prompt library, a target jailbreak scenario construction strategy corresponding to the target jailbreak prompt is determined, feature sentence splitting processing is performed on the target jailbreak prompt to obtain a target feature sentence corresponding to at least one feature type, and the target feature sentence is added to a target feature sentence library corresponding to the target jailbreak scenario construction strategy.

[0026] In some possible implementations, determining the attack success rate of the target jailbreak prompt in at least one test large model includes:

[0027] Acquire multiple malicious test questions, and combine each malicious test question with the target jailbreak prompt to obtain each jailbreak attack prompt word;

[0028] Input each jailbreak attack prompt word into at least one test large model to obtain jailbreak attack response content;

[0029] An attack fine-tuning macro model is used to perform attack evaluation processing based on the jailbreak attack response content to obtain an attack success rate of the target jailbreak prompt in the at least one test macro model.

[0030] In some possible implementations, the step of performing attack evaluation processing based on the jailbreak attack response content using an attack fine-tuning macro model to obtain an attack success rate of the target jailbreak prompt in the at least one test macro model includes:

[0031] Generate an attack assessment prompt word based on the jailbreak attack prompt word corresponding to the jailbreak attack response content and the jailbreak attack response content;

[0032] The attack assessment prompt word is input into the attack fine-tuning large model, and the attack assessment processing is performed on the target jailbreak prompt through the attack fine-tuning large model to obtain the attack success rate of the target jailbreak prompt in the at least one test large model.

[0033] In some possible implementations, the method further comprises:

[0034] Acquire multiple jailbreak prompt samples, and perform jailbreak strategy analysis and processing based on each of the jailbreak prompt samples to obtain multiple reference jailbreak scenario construction strategies;

[0035] Based on the jailbreak prompt sample corresponding to the reference jailbreak scenario construction strategy, determine the prompt template format corresponding to the reference jailbreak scenario construction strategy;

[0036] Based on the jailbreak prompt sample corresponding to the reference jailbreak scenario construction strategy, feature sentence splitting processing is performed to obtain a reference feature sentence corresponding to at least one feature type, and a feature sentence library corresponding to the at least one feature type is created based on the reference feature sentence.

[0037] In a second aspect, an embodiment of the present application provides a jailbreak prompt generating device, the device comprising:

[0038] A data acquisition module, used to determine a jailbreak scenario construction strategy, and obtain a prompt template format and feature sentence libraries corresponding to the jailbreak scenario construction strategy;

[0039] A prompt generation module, used for selecting target feature sentences corresponding to a plurality of feature types from the feature sentence library, and performing jailbreak prompt generation processing based on each of the target feature sentences and the prompt template format to obtain an initial jailbreak prompt;

[0040] The prompt processing module is used to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt, and perform jailbreak prompt evaluation processing based on the target jailbreak prompt using at least one test large model.

[0041] Optionally, the prompt generation module includes a sentence variation unit and a prompt generation unit, which is specifically used to:

[0042] The sentence variation unit is used to perform language conversion processing based on each of the target feature sentences to obtain each variation feature sentence;

[0043] The prompt generating unit is used to perform jailbreak prompt generating processing based on the variation feature sentence and the prompt template format to obtain an initial jailbreak prompt.

[0044] Optionally, the sentence variation unit includes:

[0045] The first variation processing subunit is used to determine the feature words to be varied in the target feature sentence, and determine the variant language corresponding to the feature words to be varied;

[0046] The second variation processing subunit is used to perform language variation processing on the to-be-varied feature words in the target feature sentence by using the variant language to obtain a variant feature sentence.

[0047] Optionally, the prompt processing module includes:

[0048] The prompt adjustment unit is used to determine a prompt adjustment strategy for the initial jailbreak prompt, and use the prompt adjustment strategy to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt.

[0049] Optionally, the prompt adjustment unit includes:

[0050] A first adjustment subunit is used to obtain a fine-tuned attack model for a jailbreak prompt generation scenario;

[0051] The second adjustment subunit is used to obtain the jailbreak prompt adjustment prompt word corresponding to the prompt adjustment strategy;

[0052] The third adjustment subunit is used to input the jailbreak prompt adjustment prompt word and the initial jailbreak prompt into the attack fine-tuning large model to perform jailbreak prompt adjustment processing to obtain a target jailbreak prompt.

[0053] Optionally, the prompt processing module includes:

[0054] A first processing unit is used to determine the attack success rate of the target jailbreak prompt in at least one test large model, and determine the number of targets whose attack success rate is greater than a success rate threshold;

[0055] A second processing unit, configured to output a target jailbreak prompt if the number of targets is greater than a preset threshold;

[0056] The third processing unit is used to execute the steps of determining the jailbreak scenario construction strategy and obtaining the prompt template format and each feature sentence library corresponding to the jailbreak scenario construction strategy if the target number is less than a preset threshold.

[0057] Optionally, the prompt processing module further includes:

[0058] A prompt saving unit is used to save the target jailbreak prompt to a jailbreak prompt library, determine a target jailbreak scenario construction strategy corresponding to the target jailbreak prompt, perform feature sentence splitting processing on the target jailbreak prompt to obtain a target feature sentence corresponding to at least one feature type, and add the target feature sentence to a target feature sentence library corresponding to the target jailbreak scenario construction strategy.

[0059] Optionally, the first processing unit includes:

[0060] A combination processing unit, used for obtaining a plurality of test malicious questions, and combining each of the test malicious questions with the target jailbreak prompt to obtain each jailbreak attack prompt word;

[0061] A test processing unit, used for inputting each jailbreak attack prompt word into at least one test large model to obtain jailbreak attack response content;

[0062] The attack evaluation unit is used to perform attack evaluation processing based on the jailbreak attack response content using an attack fine-tuning large model to obtain the attack success rate of the target jailbreak prompt in the at least one test large model.

[0063] Optional, Attack Assessment Unit, including:

[0064] A first evaluation subunit, configured to generate an attack evaluation prompt word based on a jailbreak attack prompt word corresponding to the jailbreak attack response content and the jailbreak attack response content;

[0065] The second evaluation sub-unit is used to input the attack evaluation prompt word into the attack fine-tuning large model, perform attack evaluation processing through the attack fine-tuning large model, and obtain the attack success rate of the target jailbreak prompt in the at least one test large model.

[0066] Optionally, the jailbreak prompt generating device further includes:

[0067] A jailbreak strategy creation module, used to obtain multiple jailbreak prompt samples, and perform jailbreak strategy analysis and processing based on each of the jailbreak prompt samples to obtain multiple reference jailbreak scenario construction strategies;

[0068] A prompt template creation module, used to determine the prompt template format corresponding to the reference jailbreak scenario construction strategy based on the jailbreak prompt sample corresponding to the reference jailbreak scenario construction strategy;

[0069] The feature sentence library creation module is used to perform feature sentence splitting processing based on the jailbreak prompt sample corresponding to the reference jailbreak scenario construction strategy, obtain reference feature sentences corresponding to at least one feature type, and create a feature sentence library corresponding to the at least one feature type based on the reference feature sentences.

[0070] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium has a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the above-mentioned method.

[0071] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a memory and a processor; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the memory and executing the above method.

[0072] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0073] The jailbreak prompt generation method provided in the embodiment of the present application determines the jailbreak scene construction strategy, obtains the prompt template format and each feature sentence library corresponding to the jailbreak scene construction strategy, selects the target feature sentences belonging to different feature types from the feature sentence library, and thus can perform jailbreak prompt generation processing based on each target feature sentence and prompt template format to obtain the initial jailbreak prompt, and obtains the target jailbreak prompt by performing jailbreak prompt adjustment processing on the initial jailbreak prompt, and uses at least one test large model to perform jailbreak prompt evaluation processing based on the target jailbreak prompt. Thus, the initial jailbreak prompt is generated by selecting the target feature sentence and prompt template format corresponding to the different feature types, and the intelligent generation of the general jailbreak prompt is realized. After that, the initial jailbreak prompt is optimized to obtain the target jailbreak prompt, and the semantic integrity and fluency of the generated jailbreak prompt are guaranteed. Then, the generated jailbreak prompt is evaluated by using the test large model to verify the quality of the generated jailbreak prompt, so as to ensure that the generated jailbreak prompt is a high-quality jailbreak prompt. Therefore, the embodiment of the present application achieves the effect of intelligently generating a high-quality general jailbreak prompt. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0075] Figure 1 It is a flowchart of a method for generating a jailbreak prompt provided in an embodiment of the present application;

[0076] Figure 2 It is a flowchart of another jailbreak prompt generation method provided in an embodiment of the present application;

[0077] Figure 3 It is a structural schematic diagram of a jailbreak prompt generating device provided in an embodiment of the present application;

[0078] Figure 4 It is a structural diagram of a prompt generation module provided in an embodiment of the present application;

[0079] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0080] In order to make the purpose, features, and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0081] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood in specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are an "or" relationship.

[0082] In related technologies, large models are vulnerable to certain adversarial strategies. A notable adversarial strategy is jailbreak attacks, which can bypass the security mechanism of LLMs through carefully designed prompts, thereby guiding LLMs to generate harmful, discriminatory, violent or sensitive content. Therefore, as the performance and application scope of LLMs continue to improve, it is crucial to evaluate the security and robustness of LLMs. When evaluating the security and robustness of LLMs, using jailbreak attack prompts to test LLMs is an effective testing method. Therefore, how to construct a universal and effective jailbreak attack prompt is a technical problem that needs to be solved urgently.

[0083] In order to solve the above technical problems, the present application is described in detail below in conjunction with specific embodiments.

[0084] In one embodiment, Figure 1 As shown, a jailbreak prompt generation method is proposed, which can be implemented by a computer program and can be run on a jailbreak prompt generation device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application.

[0085] Specifically, the jailbreak prompt generation method includes:

[0086] S101, determine a jailbreak scenario construction strategy, and obtain a prompt template format and feature sentence libraries corresponding to the jailbreak scenario construction strategy.

[0087] Among them, the jailbreak scenario construction strategy refers to the jailbreak strategy required to generate jailbreak prompts applied to one or more large language models (LLMs). LLM is an artificial intelligence content generation model designed to understand and generate human language. Optionally, LLM can be a generative artificial intelligence (AIGC) model such as ChatGPT, Gemini, and Claude. Jailbreak prompts refer to jailbreak attack prompts used in conjunction with malicious questions to bypass the security mechanism of LLM and guide LLM to generate harmful, discriminatory, violent or sensitive content based on malicious questions.

[0088] For example, jailbreak scenario construction strategies may include but are not limited to role-playing, text continuation, scientific research experiments, code execution, and oppositional responses. Role-playing aims to allow the large language model to play a specific role or identity, thereby guiding the large language model to respond from the perspective of the role or identity; role-playing allows the large language model to generate content more freely. Text continuation aims to provide a large language model with a text so that the large language model can continue writing based on the provided text; text continuation can guide the model to bypass security mechanisms for content generation in some cases. Scientific research experiments aim to guide the large language model to generate content related to scientific research by proposing hypotheses or research questions and simulating the experimental process; scientific research experiments are used to allow the model to provide more information and discussion in certain areas. Code execution aims to allow the large language model to execute specific code snippets, or interpret and generate code; code execution can be used to guide the model to generate technical content, or to guide the model to try to implement some functions that are not allowed. Oppositional responses aim to stimulate reflection or discussion in the large language model by proposing views that are opposite to the default position of the large language model; oppositional responses can be used to guide the model to explore different viewpoints, and may even break certain restrictions.

[0089] The prompt template format refers to the prompt format that the jailbreak prompt generated by using the jailbreak scenario construction strategy needs to meet. The prompt template format can specify: what content is filled in different positions in the jailbreak prompt to be generated.

[0090] The feature sentence library refers to the production materials required when the jailbreak scenario construction strategy is used to generate the jailbreak prompt. For each jailbreak scenario construction strategy, it corresponds to a feature sentence library of different feature types. The feature sentences contained in the feature sentence library of each feature type can be part of the sentences in the generated jailbreak prompt. Optionally, the feature types of the feature sentence library corresponding to each jailbreak scenario construction strategy may include but are not limited to scene introduction, role definition, role enhancement, task operation, and output specification. In the feature sentence library of each feature type, it may include a variety of different forms of feature sentences for realizing the same feature function. For example, in the feature sentence library of the scene introduction, it includes a variety of different expressions for introducing the scene. Feature sentences.

[0091] In some embodiments, a jailbreak scenario construction strategy is determined. A specific implementation method may be: obtaining multiple pre-built reference jailbreak scenario construction strategies, obtaining the priority corresponding to each reference jailbreak scenario construction strategy, and determining the reference jailbreak scenario construction strategy with the highest priority as the jailbreak scenario strategy for the jailbreak prompt generation scenario. Alternatively, another specific implementation method may be: obtaining multiple pre-built reference jailbreak scenario construction strategies, determining the jailbreak prompt construction requirements under the jailbreak prompt generation scenario, and matching the jailbreak scenario construction strategy for the jailbreak prompt generation scenario among multiple reference jailbreak scenario construction strategies based on the jailbreak prompt construction requirements. Alternatively, another specific implementation method may be: receiving a jailbreak strategy selection operation input by a user, and determining the jailbreak scenario construction strategy for the jailbreak prompt generation scenario indicated by the jailbreak strategy selection operation.

[0092] Obtain the prompt template format and each feature sentence library corresponding to the jailbreak scenario construction strategy. The specific implementation method can be: obtain the policy identifier of the jailbreak scenario construction strategy, query the prompt template format associated with the policy identifier from the prompt template format library, and query each feature sentence library associated with the policy identifier in the feature sentence total library. It can be understood that when constructing multiple reference jailbreak scenario construction strategies, the prompt template format and each feature sentence library corresponding to each reference jailbreak scenario construction strategy are also constructed, and the prompt template and each feature sentence library corresponding to each reference jailbreak scenario construction strategy are associated with the policy identifier of each reference jailbreak scenario construction strategy and stored.

[0093] S102, selecting target feature sentences corresponding to a plurality of feature types from a feature sentence library, performing jailbreak prompt generation processing based on each target feature sentence and a prompt template format, and obtaining an initial jailbreak prompt.

[0094] It can be understood that feature types may include but are not limited to scene introduction, role definition, role reinforcement, behavior description, task operation, and output specifications, wherein the feature sentence library of the scene introduction type is used to store sentences describing the scene, and the sentences describing the scene are used to describe a specific situation or background to help the large language model understand the environment or conditions; the feature sentence library of the role definition type is used to store sentences describing the role or identity played by the large language model in the scene, so as to tell the large language model what role or identity to think about the problem; the feature sentence library of the role reinforcement type is used to store sentences describing role characteristics or personality, so that the large language model can better understand the role or identity played; the feature sentence library of the task operation type is used to store sentences describing the specific tasks or operations that the large language model needs to perform; the feature sentence library of the output specification type is used to store sentences describing the requirements or standards for the content generated by the large language model, for example, it may include sentences describing the format, style, length and other regulations.

[0095] In some embodiments, a feature selection strategy is used to select one or more target feature sentences corresponding to each feature type from a feature sentence library of each feature type. The feature selection strategy can be a random selection strategy, or a strategy that selects in order of priority of the feature type. After selecting the target feature sentences corresponding to each feature type, the target feature sentences corresponding to different feature types are combined according to the prompt template format to obtain an initial jailbreak prompt.

[0096] S103, performing jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt, and performing jailbreak prompt evaluation processing based on the target jailbreak prompt using at least one test large model.

[0097] In some embodiments, the prompt defect of the initial jailbreak prompt can be analyzed, the prompt adjustment strategy for the prompt defect can be determined, and then the prompt adjustment strategy can be used to adjust the initial jailbreak prompt to obtain the target jailbreak prompt. For example, the prompt defect of the initial jailbreak prompt can be determined by multi-angle analysis and processing such as semantic analysis, grammatical analysis, and logical analysis. After obtaining the target jailbreak prompt, the attack success rate of the target jailbreak prompt in at least one test large model can be determined. When the attack success rate meets the preset conditions, the target jailbreak prompt is output so that the developer in the jailbreak prompt generation scenario can obtain the target jailbreak prompt. When the attack success rate does not meet the preset conditions, it can return to execute steps S101-S103 and regenerate a new target jailbreak prompt. For example, the attack success rate meets the preset conditions and can be set to, the number of targets is greater than the preset threshold, and the number of target number attack success rate is greater than the number of success rate thresholds.

[0098] Optionally, after outputting the target jailbreak prompt, the target jailbreak prompt can be saved to a jailbreak prompt library, a target jailbreak scenario construction strategy corresponding to the target jailbreak prompt is determined, the target jailbreak prompt is subjected to feature sentence splitting processing to obtain a target feature sentence corresponding to at least one feature type, and the target feature sentence is added to a target feature sentence library corresponding to the target jailbreak scenario construction strategy. In this way, the scale and diversity of the feature sentence library can be continuously expanded.

[0099] The jailbreak prompt generation method provided in the embodiment of the present application determines the jailbreak scene construction strategy, obtains the prompt template format and each feature sentence library corresponding to the jailbreak scene construction strategy, selects the target feature sentences belonging to different feature types from the feature sentence library, and thus can perform jailbreak prompt generation processing based on each target feature sentence and prompt template format to obtain the initial jailbreak prompt, and obtains the target jailbreak prompt by performing jailbreak prompt adjustment processing on the initial jailbreak prompt, and uses at least one test large model to perform jailbreak prompt evaluation processing based on the target jailbreak prompt. Thus, the initial jailbreak prompt is generated by selecting the target feature sentence and prompt template format corresponding to the different feature types, and the intelligent generation of the general jailbreak prompt is realized. After that, the initial jailbreak prompt is optimized to obtain the target jailbreak prompt, and the semantic integrity and fluency of the generated jailbreak prompt are guaranteed. Then, the generated jailbreak prompt is evaluated by using the test large model to verify the quality of the generated jailbreak prompt, so as to ensure that the generated jailbreak prompt is a high-quality jailbreak prompt. Therefore, the embodiment of the present application achieves the effect of intelligently generating a high-quality general jailbreak prompt.

[0100] See also Figure 2 , Figure 2 It is a flowchart of another embodiment of a jailbreak prompt generation method proposed in the present application.

[0101] Specifically, the jailbreak prompt generation method includes:

[0102] S201, determine a jailbreak scenario construction strategy, and obtain a prompt template format and feature sentence libraries corresponding to the jailbreak scenario construction strategy.

[0103] Specifically, the implementation of step S201 can be found in Figure 1 The description of the relevant parts in the illustrated embodiment will not be repeated here.

[0104] Optionally, before executing step S201, the method of the embodiment of the present application further executes the following steps:

[0105] A2: Obtain multiple jailbreak prompt samples, analyze and process jailbreak strategies based on each jailbreak prompt sample, and obtain multiple reference jailbreak scenario construction strategies;

[0106] The jailbreak prompt sample refers to the carefully designed jailbreak prompts collected from the Internet. The jailbreak prompt sample is also the jailbreak prompt with a high model attack success rate. The reference jailbreak scenario construction strategy refers to the jailbreak scenario construction strategy used as a standard strategy obtained by analyzing, sorting and refining the jailbreak prompt samples. The jailbreak scenario construction strategy is the jailbreak strategy used to generate the jailbreak prompt.

[0107] Specifically, multiple jailbreak prompt samples are collected from the Internet, feature classification is performed on each jailbreak prompt sample, and multiple reference jailbreak scenario construction strategies are extracted from the jailbreak prompt samples of different feature classifications.

[0108] A4: Based on the jailbreak prompt sample corresponding to the reference jailbreak scenario construction strategy, determine the prompt template format corresponding to the reference jailbreak scenario construction strategy;

[0109] Specifically, the format of the jailbreak prompt sample corresponding to each reference jailbreak scenario construction strategy is analyzed, multiple prompt constituent submodules of the jailbreak prompt sample corresponding to each reference jailbreak scenario construction strategy are determined, the module position of each prompt constituent submodule in the jailbreak prompt sample is determined, the module position of each prompt constituent submodule is optimized to obtain the target module position corresponding to each prompt constituent submodule, the target module position is the filling position of the filling sentence in each prompt constituent submodule, the feature type of the filling sentence in each prompt constituent submodule is determined, and the mapping relationship between the target module position and the feature type corresponding to each prompt constituent submodule is suggested to obtain the prompt template format. Among them, the prompt template format can include each feature type and its corresponding target module position. After determining the feature type corresponding to the feature sentence, the filling position (i.e., the target module position) of the feature sentence can be determined from the prompt template format.

[0110] A6: Perform feature sentence splitting processing based on the jailbreak prompt sample corresponding to the reference jailbreak scenario construction strategy to obtain a reference feature sentence corresponding to at least one feature type, and create a feature sentence library corresponding to at least one feature type based on the reference feature sentence.

[0111] Specifically, for each reference jailbreak scenario construction strategy, the jailbreak prompt sample corresponding to the reference jailbreak scenario construction strategy is decomposed into reference feature sentences corresponding to at least one feature type through grammatical analysis and semantic analysis, so as to obtain reference feature sentences corresponding to the scene introduction, reference feature sentences corresponding to the role definition, reference feature sentences corresponding to the role reinforcement, reference feature sentences corresponding to the task operation, and reference feature sentences corresponding to the output specification. A feature sentence library including reference feature sentences corresponding to the scene introduction is created, a feature sentence library including reference feature sentences corresponding to the role definition is created, a feature sentence library including reference feature sentences corresponding to the role reinforcement is created, a feature sentence library including reference feature sentences corresponding to the task operation is created, and a feature sentence library including reference feature sentences corresponding to the output specification is created.

[0112] Therefore, by pre-building a jailbreak scenario construction strategy and its corresponding prompt template format and each feature sentence library, it is convenient to directly extract feature sentences of different feature types from the feature sentence library and combine them with the prompt template format to automatically generate the initial jailbreak prompt.

[0113] S202, selecting target feature sentences corresponding to multiple feature types from a feature sentence library.

[0114] In some embodiments, since each feature sentence library includes a reference feature sentence of a feature type, and different feature sentence libraries include reference feature sentences of different feature types, the priority of each feature type can be determined, and the target feature sentence corresponding to each feature type is selected from each feature sentence library in order of priority from high to low. For example, the priority order of scene introduction, role definition, role enhancement, task operation, and output specification is: scene introduction> role definition> role enhancement> task operation> output specification, then first extract one or more target feature sentences from the feature sentence library of the scene introduction type, then extract one or more target feature sentences from the feature sentence library of the role definition type, then extract one or more target feature sentences from the feature sentence library of the role enhancement type, then extract one or more target feature sentences from the feature sentence library of the character operation type, and finally extract one or more target feature sentences from the feature sentence library of the output specification type.

[0115] Optionally, a random selection strategy may be used to select target feature sentences corresponding to different feature types from different feature sentence libraries, or a polling selection strategy may be used to select target feature sentences corresponding to different feature types from different feature sentence libraries, or a heuristic selection strategy may be used to select target feature sentences corresponding to different feature types from different feature sentence libraries.

[0116] S203, performing language conversion processing based on each target feature sentence to obtain each variant feature sentence.

[0117] The variant feature sentence refers to a feature sentence obtained by changing the language category of at least one word in the target feature sentence.

[0118] In some embodiments, executing step S203 may include: determining the feature words to be mutated in the target feature sentence, determining the variant language corresponding to the feature words to be mutated; and performing language variation processing on the feature words to be mutated in the target feature sentence using the variant language to obtain the variant feature sentence.

[0119] It can be understood that the feature words to be mutated refer to the words whose language categories need to be changed in the target feature sentences, and the variant language refers to the new language different from the existing language that the feature words to be mutated will use. Specifically, the feature words to be mutated can be randomly determined in the target feature sentences, and the variant language corresponding to the feature words to be mutated can be randomly determined according to the existing language categories of the feature words to be mutated; or, the target feature sentences can be processed by performing sensitive word detection to obtain the existing sensitive words appearing in the target feature sentences, and the existing sensitive words are used as the feature words to be mutated, and the variant language corresponding to the feature words to be mutated is selected; then, the feature words to be mutated are replaced with the target feature words translated using the variant language in the target feature sentences to obtain the variant feature sentences.

[0120] For example, if the target feature sentence contains a sensitive word described in English or Chinese, the security mechanism of the large language model will usually restrict the sensitive word and prohibit the large language model from outputting content related to the sensitive word. In this case, Arabic can be used as the variant language corresponding to the sensitive word, and the sensitive word can be translated into Arabic to obtain the target sensitive word. In the target feature sentence, the sensitive word described in English can be replaced with the target sensitive word to obtain the variant feature sentence.

[0121] S204, performing jailbreak prompt generation processing based on the variant feature sentence and the prompt template format to obtain an initial jailbreak prompt.

[0122] In some embodiments, it can be seen from the description of the above-mentioned relevant parts in the embodiments of the present application that the prompt template format may include each feature type and its corresponding target module position. After determining the feature type corresponding to the feature sentence, the filling position (i.e., the target module position) of the feature sentence can be determined from the prompt template format. Therefore, after determining the variant feature sentence, obtain the feature type corresponding to the variant feature sentence, find the target module position corresponding to the feature type from the prompt template format, and fill in the variant feature sentences of the corresponding feature type in different target module positions to obtain the initial jailbreak prompt. In this way, the initial jailbreak prompt may include variant feature sentences of scene introduction type, variant feature sentences of role definition type, variant feature sentences of role reinforcement type, variant feature sentences of task operation type, and variant feature sentences of output specification type.

[0123] S205, determining a prompt adjustment strategy for the initial jailbreak prompt, and using the prompt adjustment strategy to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt.

[0124] Among them, the prompt adjustment strategy refers to the strategy adopted to adjust the initial jailbreak prompt. For example, the prompt adjustment strategy may include but is not limited to imitation, reconstruction, fusion, abbreviation, expansion, conversion, etc. Imitation refers to a strategy for generating a jailbreak prompt with the same semantics as the initial jailbreak prompt but different expressions. Reconstruction refers to a strategy for rewriting the original feature sentences in the initial jailbreak prompt while retaining the semantics of the original feature sentences. Fusion refers to a strategy for fusing two different initial jailbreak prompts to generate a new jailbreak prompt. Abbreviation refers to a strategy for simplifying the initial jailbreak prompt while retaining the original semantics of the initial jailbreak prompt. Expansion refers to a strategy for adding new content to the initial jailbreak prompt while retaining the semantics of the initial jailbreak prompt. Conversion refers to a strategy for using colloquialisms or dialects to change the expression style of the initial jailbreak prompt.

[0125] Specifically, the implementation method of determining the prompt adjustment strategy for the initial jailbreak prompt can be: randomly selecting the prompt adjustment strategy for the initial jailbreak prompt from the preset prompt adjustment strategies, or, determining the prompt defect existing in the initial jailbreak prompt, and selecting the prompt adjustment strategy for the initial jailbreak prompt from the preset prompt adjustment strategies according to the prompt defect.

[0126] In some embodiments, executing a prompt adjustment strategy to perform jailbreak prompt adjustment processing on an initial jailbreak prompt to obtain a target jailbreak prompt may specifically include: obtaining a jailbreak prompt adjustment prompt word corresponding to the prompt adjustment strategy; inputting the jailbreak prompt adjustment prompt word and the initial jailbreak prompt into the attack fine-tuning large model to perform jailbreak prompt adjustment processing to obtain a target jailbreak prompt.

[0127] It is understandable that the attack fine-tuning large model can be an open source large model obtained after model training for the jailbreak prompt generation scenario. The jailbreak prompt adjustment prompt word includes prompting the attack fine-tuning large model to use the prompt adjustment strategy to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain the task description information of the target jailbreak prompt.

[0128] Specifically, the jailbreak prompt adjustment prompt words and the initial jailbreak prompt are input into the attack fine-tuning large model, and the attack fine-tuning large model uses the prompt adjustment strategy indicated by the jailbreak prompt adjustment prompt words to adjust the initial jailbreak prompt to obtain the target jailbreak prompt, so that the target jailbreak prompt has diversity.

[0129] In some other embodiments, executing step S205 may include: obtaining an attack fine-tuning large model for a jailbreak prompt generation scenario; generating a prompt adjustment prompt word based on the initial jailbreak prompt, inputting the prompt adjustment prompt word and the initial jailbreak prompt into the attack fine-tuning large model, performing prompt detection processing on the initial jailbreak prompt through the attack fine-tuning large model to obtain a prompt adjustment strategy for the initial jailbreak prompt, and performing jailbreak prompt adjustment processing on the initial jailbreak prompt based on the prompt adjustment strategy to obtain a target jailbreak prompt.

[0130] Specifically, the prompt adjustment prompt words and the initial jailbreak prompt are input into the attack fine-tuning large model, and the attack fine-tuning large model first performs prompt detection processing on the initial jailbreak prompt to obtain a prompt adjustment strategy for the initial jailbreak prompt. For example, the attack fine-tuning large model performs multiple prompt detection processing such as semantic analysis, grammatical analysis, and logical analysis on the initial jailbreak prompt to obtain prompt defects in the initial jailbreak prompt, and selects a prompt adjustment strategy for the prompt defect from a variety of preset adjustment strategies. Then, the attack fine-tuning large model uses the prompt adjustment strategy to adjust the jailbreak prompt to obtain a target jailbreak prompt, so that the target jailbreak prompt has diversity.

[0131] Before obtaining the attack fine-tuning large model for the jailbreak prompt generation scenario, it also includes: creating a target risk data set based on reference malicious questions and reference response content for the reference malicious questions; creating an initial attack fine-tuning large model, and performing at least one round of model training processing on the initial attack fine-tuning large model based on the target risk data set to obtain the attack fine-tuning large model. The target risk data set may include an input-output data pair consisting of a reference malicious question and a reference response content for the reference malicious question, and the reference response content for the reference malicious question may be obtained by inputting the reference malicious question into large models of different architectures and scales. The training process of the attack fine-tuning large model may be: creating an initial attack fine-tuning large model for the jailbreak prompt generation scenario, using the reference malicious question as a sample input question, marking the reference response content for the reference malicious question as the output response label of the sample input question, inputting the sample input question into the initial fine-tuning attack large model for at least one round of model training to obtain a predicted output response, using a model loss function based on the predicted output response and the output response label to determine a model loss value, and adjusting the model parameters of the initial attack fine-tuning large model based on the model loss value until the model training end condition is met to obtain the attack fine-tuning large model.

[0132] S206, determining the attack success rate of the target jailbreak prompt in at least one test large model, and determining the number of targets whose attack success rate is greater than the success rate threshold.

[0133] Among them, the attack success rate refers to the probability of determining whether the target jailbreak prompt is successfully attacked based on the response content of the test large model to the target jailbreak prompt. The test large model can use large models of different architectures and scales.

[0134] In some embodiments, determining the attack success rate of a target jailbreak prompt in at least one test large model includes: obtaining a plurality of test malicious questions, and combining each test malicious question with the target jailbreak prompt to obtain each jailbreak attack prompt word; inputting each jailbreak attack prompt word into at least one test large model to obtain jailbreak attack response content; and performing attack evaluation processing using an attack fine-tuning large model based on the jailbreak attack response content to obtain the attack success rate of the target jailbreak prompt in at least one test large model.

[0135] It is understandable that the test malicious question refers to a malicious, sensitive, or harmful, discriminatory, violent question. Each test malicious question is combined with the target jailbreak prompt to obtain each jailbreak attack prompt word, which means that each test malicious question is filled in the designated position in the target jailbreak prompt to obtain multiple jailbreak attack prompt words, and the designated position is the position in the target jailbreak prompt for filling the malicious question. All jailbreak attack prompt words can be input into each test large model respectively to obtain the jailbreak attack response content of the test large model to each jailbreak attack prompt word. Alternatively, the first number of jailbreak attack prompt words can also be input into a certain test large model, the second number of jailbreak attack prompt words can be input into another test large model, and the third number of jailbreak prompt input content can be input into another test large model to obtain the jailbreak attack response content of each jailbreak attack prompt word. Wherein, the first number, the second number, and the third number can be partially the same or completely the same or completely different.

[0136] The attack success rate of the target jailbreak prompt in at least one test large model can be understood as: after attacking the test large model by combining the target jailbreak prompt with the test malicious question, the attack success rate of the attack on the test large model is obtained by evaluating the response content of the test large model to the attack. The attack success rate is the attack success rate of the target jailbreak prompt in the test large model.

[0137] Specifically, based on the jailbreak attack response content, an attack fine-tuning large model is used to perform attack evaluation processing to obtain the attack success rate of the target jailbreak prompt in at least one test large model, including: generating an attack evaluation prompt word based on the jailbreak attack prompt word and the jailbreak prompt response content corresponding to the jailbreak attack response content; inputting the attack evaluation prompt word into the attack fine-tuning large model, and performing attack evaluation processing through the attack fine-tuning large model to obtain the attack success rate of the target jailbreak prompt in at least one test large model. Among them, the attack evaluation prompt word may include preset attack evaluation rules, jailbreak attack prompt words, and jailbreak prompt response content corresponding to the jailbreak attack prompt word. The attack evaluation prompt word refers to the task description information that instructs the attack fine-tuning large model to use the preset attack evaluation rules to analyze the jailbreak attack prompt word and the jailbreak attack response content corresponding to the jailbreak attack prompt word, and determine the attack success rate of the jailbreak attack prompt word in the test large model. Among them, the preset attack evaluation rules may include rules such as correlation analysis, value alignment, semantic analysis, malicious content detection, and jailbreak attack detection. It can be understood that the attack success rate of the jailbreak attack prompt word determined by attacking the fine-tuning large model in the test large model is the attack success rate of the target jailbreak prompt constituting the jailbreak attack prompt word in the test large model.

[0138] Optionally, after inputting the attack evaluation prompt word into the attack fine-tuning large model and obtaining the attack success rate of the target jailbreak prompt in at least one test large model, the target jailbreak prompt in the test large model outputting the jailbreak prompt response content can be generated based on the jailbreak attack prompt word corresponding to the jailbreak attack response content, the jailbreak attack response content, and the attack success rate of the target jailbreak prompt in the test large model outputting the jailbreak attack response content. A prompt evaluation result of the target jailbreak prompt in the test large model outputting the jailbreak prompt response content can be generated based on the prompt evaluation result. The first attack evaluation prompt word and the prompt evaluation result can be input into the attack fine-tuning large model, and the prompt evaluation result can be verified by the attack fine-tuning large model to obtain the target attack success rate of the target jailbreak prompt in at least one test large model. The first attack evaluation prompt word is used to instruct the attack fine-tuning large model to reasonably verify the prompt evaluation result, instruct the attack fine-tuning large model to carefully review the explanation given in the prompt evaluation result, find out any illogical reasoning or inconsistency caused by security restrictions, and overturn the prompt evaluation result when the verification prompt evaluation result is unreasonable, and correct the attack success rate in the prompt evaluation result to obtain the correct target attack success rate.

[0139] S207, determining whether the number of targets is greater than a preset threshold.

[0140] S208: If the number of targets is greater than a preset threshold, a target jailbreak prompt is output.

[0141] In some embodiments, if the number of targets whose attack success rate is greater than the success rate threshold is greater than a preset threshold, it indicates that the target jailbreak prompt has a good attack effect, can be used in combination with a variety of malicious problems, and can successfully attack most large models. Then the target jailbreak prompt can be output so that developers in the jailbreak prompt scenario can obtain the target jailbreak prompt.

[0142] If the number of targets is less than the preset threshold, then execute S201: determine the jailbreak scenario construction strategy, obtain the prompt template format and each feature sentence library corresponding to the jailbreak scenario construction strategy. If the number of targets whose attack success rate is greater than the success rate threshold is greater than the preset threshold, indicating that the target jailbreak prompt has a poor attack effect on the large model, then return to execute steps S201-S207 to regenerate a new jailbreak prompt and verify the attack effect of the new jailbreak prompt.

[0143] Optionally, after outputting the target jailbreak prompt, the target jailbreak prompt can also be saved to a jailbreak prompt library, a target jailbreak scenario construction strategy corresponding to the target jailbreak prompt can be determined, feature sentence splitting processing can be performed on the target jailbreak prompt to obtain a target feature sentence corresponding to at least one feature type, and the target feature sentence can be added to a target feature sentence library corresponding to the target jailbreak scenario construction strategy.

[0144] It can be understood that each target jailbreak scenario construction strategy has a target feature sentence library belonging to at least one feature type. Therefore, the target feature sentences obtained from the target jailbreak prompts can be stored in the target feature sentence library of the same feature type to expand the scale and diversity of the feature sentence library.

[0145] In the jailbreak prompt generation method provided in the embodiment of the present application, a jailbreak scenario construction strategy is determined, a prompt template format and each feature sentence library corresponding to the jailbreak scenario construction strategy are obtained, language conversion processing is performed based on each target feature sentence to obtain each variant feature sentence, thereby, by performing language conversion processing on the target feature sentence selected from the feature sentence library, the sensitive words in the variant feature sentence can be packaged; then, a jailbreak prompt generation processing is performed based on the variant feature sentence and the prompt template format to obtain an initial jailbreak prompt, a prompt adjustment strategy for the initial jailbreak prompt is determined, and the prompt adjustment strategy is used to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt, thereby, by adjusting the initial jailbreak prompt, In order to obtain a target jailbreak prompt that does not lose the original semantics of the initial jailbreak prompt and has logical coherence, fluent expression, and a strong guiding effect; finally, determine the attack success rate of the target jailbreak prompt in at least one test large model, determine the number of targets whose attack success rate is greater than the success rate threshold, and judge whether the number of targets is greater than the preset threshold. If the number of targets is greater than the preset threshold, output the target jailbreak prompt; if the number of targets is less than the preset threshold, regenerate the jailbreak prompt. Thus, the attack success rate of the target jailbreak prompt is detected through multiple test large models to ensure that a general and effective jailbreak prompt with a good attack effect on the large model can be generated, so that the reliability of the security function of the large model can be tested using the general and effective jailbreak prompt with a good attack effect.

[0146] The following will be combined Figure 3 , the jailbreak prompt generating device provided in the embodiment of the present application is introduced in detail. It should be noted that, Figure 3 The jailbreak prompt generating device shown is used to execute the present application Figure 1-2 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 1-2 The embodiment shown.

[0147] See also Figure 3 , which shows a schematic diagram of the structure of the jailbreak prompt generating device of the embodiment of the present application. The jailbreak prompt generating device 1 can be implemented as all or part of the device through software, hardware or a combination of both. According to some embodiments, the jailbreak prompt generating device 1 includes a data acquisition module 11, a prompt generating module 12 and a prompt processing module 13, which are specifically used to:

[0148] The data acquisition module 11 is used to determine the jailbreak scenario construction strategy and obtain the prompt template format and each characteristic sentence library corresponding to the jailbreak scenario construction strategy;

[0149] A prompt generating module 12 is used to select target feature sentences corresponding to a plurality of feature types from the feature sentence library, and perform jailbreak prompt generating processing based on each of the target feature sentences and the prompt template format to obtain an initial jailbreak prompt;

[0150] The prompt processing module 13 is used to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt, and perform jailbreak prompt evaluation processing based on the target jailbreak prompt using at least one test large model.

[0151] Optional, see Figure 4 , is a schematic diagram of the structure of a prompt generation module 12 provided in an embodiment of the present application. The prompt generation module 12 may include a sentence variation unit 121 and a prompt generation unit 122, which are specifically used for:

[0152] A sentence variation unit 121, configured to perform language conversion processing based on each of the target feature sentences to obtain each variation feature sentence;

[0153] The prompt generating unit 122 is used to generate a jailbreak prompt based on the variation feature sentence and the prompt template format to obtain an initial jailbreak prompt.

[0154] Optionally, the sentence variation unit 121 includes:

[0155] The first variation processing subunit is used to determine the feature words to be varied in the target feature sentence, and determine the variant language corresponding to the feature words to be varied;

[0156] The second variation processing subunit is used to perform language variation processing on the to-be-varied feature words in the target feature sentence by using the variant language to obtain a variant feature sentence.

[0157] Optionally, the prompt processing module 13 includes:

[0158] The prompt adjustment unit is used to determine a prompt adjustment strategy for the initial jailbreak prompt, and use the prompt adjustment strategy to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt.

[0159] Optional, prompt adjustment unit, including:

[0160] A first adjustment subunit is used to obtain a fine-tuned attack model for a jailbreak prompt generation scenario;

[0161] The second adjustment subunit is used to obtain the jailbreak prompt adjustment prompt word corresponding to the prompt adjustment strategy;

[0162] The third adjustment subunit is used to input the jailbreak prompt adjustment prompt word and the initial jailbreak prompt into the attack fine-tuning large model to perform jailbreak prompt adjustment processing to obtain a target jailbreak prompt.

[0163] Optionally, the prompt processing module 13 includes:

[0164] A first processing unit is used to determine the attack success rate of the target jailbreak prompt in at least one test large model, and determine the number of targets whose attack success rate is greater than a success rate threshold;

[0165] A second processing unit, configured to output a target jailbreak prompt if the number of targets is greater than a preset threshold;

[0166] The third processing unit is used to execute the steps of determining the jailbreak scenario construction strategy and obtaining the prompt template format and each feature sentence library corresponding to the jailbreak scenario construction strategy if the target number is less than a preset threshold.

[0167] Optionally, the prompt processing module 13 further includes:

[0168] A prompt saving unit is used to save the target jailbreak prompt to a jailbreak prompt library, determine a target jailbreak scenario construction strategy corresponding to the target jailbreak prompt, perform feature sentence splitting processing on the target jailbreak prompt to obtain a target feature sentence corresponding to at least one feature type, and add the target feature sentence to a target feature sentence library corresponding to the target jailbreak scenario construction strategy.

[0169] Optionally, the first processing unit includes:

[0170] A combination processing unit, used for obtaining a plurality of malicious test questions, and combining each of the malicious test questions with the target jailbreak prompt to obtain each jailbreak attack prompt word;

[0171] A test processing unit, used for inputting each jailbreak attack prompt word into at least one test large model to obtain jailbreak attack response content;

[0172] The attack evaluation unit is used to perform attack evaluation processing based on the jailbreak attack response content using an attack fine-tuning large model to obtain the attack success rate of the target jailbreak prompt in the at least one test large model.

[0173] Optional, Attack Assessment Unit, including:

[0174] A first evaluation subunit, configured to generate an attack evaluation prompt word based on a jailbreak attack prompt word corresponding to the jailbreak attack response content and the jailbreak attack response content;

[0175] The second evaluation sub-unit is used to input the attack evaluation prompt word into the attack fine-tuning large model, perform attack evaluation processing through the attack fine-tuning large model, and obtain the attack success rate of the target jailbreak prompt in the at least one test large model.

[0176] Optionally, the jailbreak prompt generating device 1 further includes:

[0177] A jailbreak strategy creation module, used to obtain multiple jailbreak prompt samples, and perform jailbreak strategy analysis and processing based on each of the jailbreak prompt samples to obtain multiple reference jailbreak scenario construction strategies;

[0178] A prompt template creation module, used to determine the prompt template format corresponding to the reference jailbreak scenario construction strategy based on the jailbreak prompt sample corresponding to the reference jailbreak scenario construction strategy;

[0179] The feature sentence library creation module is used to perform feature sentence splitting processing based on the jailbreak prompt sample corresponding to the reference jailbreak scenario construction strategy, obtain reference feature sentences corresponding to at least one feature type, and create a feature sentence library corresponding to the at least one feature type based on the reference feature sentences.

[0180] The jailbreak prompt generation device provided by the embodiment of the present application determines the jailbreak scene construction strategy, obtains the prompt template format and each feature sentence library corresponding to the jailbreak scene construction strategy, selects the target feature sentences belonging to different feature types from the feature sentence library, so that the jailbreak prompt generation process can be performed based on each target feature sentence and the prompt template format to obtain the initial jailbreak prompt, and the target jailbreak prompt is obtained by performing jailbreak prompt adjustment processing on the initial jailbreak prompt, and the jailbreak prompt evaluation process is performed based on the target jailbreak prompt using at least one test large model. Thus, the initial jailbreak prompt is generated by the selected target feature sentences and prompt template formats corresponding to different feature types, and the intelligent generation of a universal jailbreak prompt is realized. After that, the initial jailbreak prompt is optimized to obtain the target jailbreak prompt, and the semantic integrity and fluency of the generated jailbreak prompt are guaranteed. Then, the generated jailbreak prompt is evaluated using the test large model to verify the quality of the generated jailbreak prompt, so as to ensure that the generated jailbreak prompt is a high-quality jailbreak prompt. Therefore, the embodiment of the present application achieves the effect of intelligently generating a high-quality universal jailbreak prompt.

[0181] Please refer to Figure 5 , Figure 5A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Exemplarily, the electronic device in the embodiment of the present application may be a server, and the electronic device may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected via the bus 150.

[0182] The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 110 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0183] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (e.g., a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The operating system may be an Android system, including a system deeply developed based on the Android system, an IOS system developed by Apple, including a system deeply developed based on the IOS system or other systems.

[0184] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0185] The input device 130 is used to receive input commands or data, and includes but is not limited to a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is used to output commands or data, and includes but is not limited to a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 are touch screen displays.

[0186] The touch display screen can be designed as a full screen, a curved screen or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of the present application.

[0187] In addition, those skilled in the art will appreciate that the structure of the electronic device shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the electronic device also includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0188] exist Figure 5In the electronic device shown, the processor 110 can be used to call the program of the jailbreak prompt generation method stored in the memory 120, and specifically perform the following operations:

[0189] Determine a jailbreak scenario construction strategy, and obtain a prompt template format and feature sentence libraries corresponding to the jailbreak scenario construction strategy;

[0190] Selecting target feature sentences corresponding to a plurality of feature types from the feature sentence library, and performing jailbreak prompt generation processing based on each of the target feature sentences and the prompt template format to obtain an initial jailbreak prompt;

[0191] The initial jailbreak prompt is subjected to jailbreak prompt adjustment processing to obtain a target jailbreak prompt, and based on the target jailbreak prompt, at least one test large model is used to perform jailbreak prompt evaluation processing.

[0192] In one embodiment, when the processor 110 performs the step of generating a jailbreak prompt based on each of the target feature sentences and the prompt template format to obtain an initial jailbreak prompt, the processor 110 specifically performs the following operations:

[0193] Performing language conversion processing based on each of the target feature sentences to obtain each variant feature sentence;

[0194] A jailbreak prompt generation process is performed based on the variation feature sentence and the prompt template format to obtain an initial jailbreak prompt.

[0195] In one embodiment, when the processor 110 performs the step of performing language conversion processing based on each of the target feature sentences to obtain each variant feature sentence, the processor 110 specifically performs the following operations:

[0196] Determining a feature word to be mutated in the target feature sentence, and determining a variant language corresponding to the feature word to be mutated;

[0197] In the target feature sentence, the feature words to be mutated are subjected to language variation processing by using the variant language to obtain a variant feature sentence.

[0198] In one embodiment, when the processor 110 performs the step of performing jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain the target jailbreak prompt, the processor 110 specifically performs the following operations:

[0199] A prompt adjustment strategy for the initial jailbreak prompt is determined, and the prompt adjustment strategy is used to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt.

[0200] In one embodiment, when the processor 110 executes the step of using the prompt adjustment strategy to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt, the processor 110 specifically performs the following operations:

[0201] Get a large model fine-tuned for attacking jailbreak prompt generation scenarios;

[0202] Obtain the jailbreak prompt adjustment prompt word corresponding to the prompt adjustment strategy;

[0203] The jailbreak prompt adjustment prompt word and the initial jailbreak prompt are input into the attack fine-tuning large model to perform jailbreak prompt adjustment processing to obtain a target jailbreak prompt.

[0204] In one embodiment, when the processor 110 executes the step of using at least one test large model to perform jailbreak prompt evaluation processing based on the target jailbreak prompt, the processor 110 specifically performs the following operations:

[0205] Determine the attack success rate of the target jailbreak prompt in at least one test large model, and determine the number of targets whose attack success rate is greater than a success rate threshold;

[0206] If the number of targets is greater than a preset threshold, outputting a target jailbreak prompt;

[0207] If the target number is less than a preset threshold, the steps of determining the jailbreak scenario construction strategy and obtaining the prompt template format and each feature sentence library corresponding to the jailbreak scenario construction strategy are executed.

[0208] In one embodiment, after outputting the target jailbreak prompt, the processor 110 further performs the following operations:

[0209] The target jailbreak prompt is saved to a jailbreak prompt library, a target jailbreak scenario construction strategy corresponding to the target jailbreak prompt is determined, feature sentence splitting processing is performed on the target jailbreak prompt to obtain a target feature sentence corresponding to at least one feature type, and the target feature sentence is added to a target feature sentence library corresponding to the target jailbreak scenario construction strategy.

[0210] In one embodiment, when the processor 110 executes the step of determining the attack success rate of the target jailbreak prompt in at least one test large model, the processor 110 specifically performs the following operations:

[0211] Acquire multiple malicious test questions, and combine each malicious test question with the target jailbreak prompt to obtain each jailbreak attack prompt word;

[0212] Input each of the jailbreak attack prompt words into at least one test large model to obtain jailbreak attack response content;

[0213] An attack fine-tuning macro model is used to perform attack evaluation processing based on the jailbreak attack response content to obtain an attack success rate of the target jailbreak prompt in the at least one test macro model.

[0214] In one embodiment, when the processor 110 performs the step of performing attack evaluation processing based on the jailbreak attack response content using the attack fine-tuning large model to obtain the attack success rate of the target jailbreak prompt in the at least one test large model, the processor 110 specifically performs the following operations:

[0215] Generate an attack assessment prompt word based on the jailbreak prompt input content corresponding to the jailbreak attack response content and the jailbreak prompt response content;

[0216] The attack assessment prompt word is input into the attack fine-tuning large model, and the attack assessment processing is performed through the attack fine-tuning large model to obtain the attack success rate of the target jailbreak prompt in the at least one test large model.

[0217] In one embodiment, the processor 110 further performs the following operations:

[0218] Acquire multiple jailbreak prompt samples, and perform jailbreak strategy analysis and processing based on each of the jailbreak prompt samples to obtain multiple reference jailbreak scenario construction strategies;

[0219] Based on the jailbreak prompt sample corresponding to the reference jailbreak scenario construction strategy, determine the prompt template format corresponding to the reference jailbreak scenario construction strategy;

[0220] Based on the jailbreak prompt sample corresponding to the reference jailbreak scenario construction strategy, feature sentence splitting processing is performed to obtain a reference feature sentence corresponding to at least one feature type, and a feature sentence library corresponding to the at least one feature type is created based on the reference feature sentence.

[0221] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the jailbreak prompt generation method as described in the above embodiments.

[0222] An embodiment of the present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the jailbreak prompt generation method described in the above embodiments.

[0223] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0224] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A jailbreak prompt generation method, characterized in that: The method comprises: Determine a jailbreak scenario construction strategy, and obtain a prompt template format and feature sentence libraries corresponding to the jailbreak scenario construction strategy; Selecting target feature sentences corresponding to a plurality of feature types from the feature sentence library, and performing jailbreak prompt generation processing based on each of the target feature sentences and the prompt template format to obtain an initial jailbreak prompt; The initial jailbreak prompt is subjected to jailbreak prompt adjustment processing to obtain a target jailbreak prompt, and based on the target jailbreak prompt, at least one test large model is used to perform jailbreak prompt evaluation processing.

2. The method according to claim 1, characterized in that The jailbreak prompt generation process is performed based on each of the target feature sentences and the prompt template format to obtain an initial jailbreak prompt, including: Performing language conversion processing based on each of the target feature sentences to obtain each variant feature sentence; A jailbreak prompt generation process is performed based on the variation feature sentence and the prompt template format to obtain an initial jailbreak prompt.

3. The method according to claim 2, characterized in that The language conversion process is performed based on each of the target feature sentences to obtain each variant feature sentence, including: Determining a feature word to be mutated in the target feature sentence, and determining a variant language corresponding to the feature word to be mutated; In the target feature sentence, the feature words to be mutated are subjected to language variation processing by using the variant language to obtain a variant feature sentence.

4. The method according to claim 1, characterized in that The step of performing jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt includes: A prompt adjustment strategy for the initial jailbreak prompt is determined, and the prompt adjustment strategy is used to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt.

5. The method according to claim 4, characterized in that The step of using the prompt adjustment strategy to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt includes: Get a large model fine-tuned for attacking jailbreak prompt generation scenarios; Obtain the jailbreak prompt adjustment prompt word corresponding to the prompt adjustment strategy; The jailbreak prompt adjustment prompt word and the initial jailbreak prompt are input into the attack fine-tuning large model to perform jailbreak prompt adjustment processing to obtain a target jailbreak prompt.

6. The method according to claim 1, characterized in that The step of using at least one test large model to perform jailbreak prompt evaluation based on the target jailbreak prompt includes: Determine the attack success rate of the target jailbreak prompt in at least one test large model, and determine the number of targets whose attack success rate is greater than a success rate threshold; If the number of targets is greater than a preset threshold, outputting a target jailbreak prompt; If the target number is less than a preset threshold, the steps of determining the jailbreak scenario construction strategy and obtaining the prompt template format and each feature sentence library corresponding to the jailbreak scenario construction strategy are executed.

7. The method according to claim 6, characterized in that After outputting the target jailbreak prompt, the method further includes: The target jailbreak prompt is saved to a jailbreak prompt library, a target jailbreak scenario construction strategy corresponding to the target jailbreak prompt is determined, feature sentence splitting processing is performed on the target jailbreak prompt to obtain a target feature sentence corresponding to at least one feature type, and the target feature sentence is added to a target feature sentence library corresponding to the target jailbreak scenario construction strategy.

8. A jailbreak prompt generating device, characterized in that: The device comprises: A data acquisition module, used to determine a jailbreak scenario construction strategy, and obtain a prompt template format and feature sentence libraries corresponding to the jailbreak scenario construction strategy; A prompt generation module, used for selecting target feature sentences corresponding to a plurality of feature types from the feature sentence library, and performing jailbreak prompt generation processing based on each of the target feature sentences and the prompt template format to obtain an initial jailbreak prompt; The prompt processing module is used to perform jailbreak prompt adjustment processing on the initial jailbreak prompt to obtain a target jailbreak prompt, and perform jailbreak prompt evaluation processing based on the target jailbreak prompt using at least one test large model.

9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method according to any one of claims 1 to 7.

Images